W3C DOM Level 1 Conformance Test Suite Released

Dimitris Dimitriadis of Ontologicon , W3C Representative to the DOM Test Suite Group, writes with an announcement also signed by Philippe Le Hégaret, W3C DOM Activity lead, and Mary Brady of NIST: "The DOM Test Suite Group releases the first version of the DOM Conformance Test Suite, Level 1 Core. The first version of the DOM Conformance Test Suite for Level 1 Core has been released by the DOM Test Suite Group. The DOM TS aims at helping implementors test their implementations' conformance with the W3C DOM Level 1 specification. This work, launched by W3C and NIST, is a publically developed and open framework to test the DOM Level 1 Core implementations. Read about the Document Object Model (DOM) Conformance Test Suites at http://www.w3.org/DOM/Test, where you can also download the DOM TS distribution. Comments are appreciated and need to be sent to [e-mail] (online archive at http://lists.w3.org/Archives/Public/www-dom-ts)."

IE is way more compatible than Mozilla?

[svu]

Just tested ECMAScript on Mozilla 0.9.8. I got 25 failures and 63 errors out of 290 runs! It is not very good, isn't it? IE6 gives 18 failures and 10 errors. Really sad... Hope mozilla.org will do something about it...

Re:IE is way more compatible than Mozilla?

[Shimbo]

Just tested ECMAScript on Mozilla 0.9.8. I got 25 failures and 63 errors out of 290 runs! It is not very good, isn't it? IE6 gives 18 failures and 10 errors. Really sad...

Well that's one way of looking at it. Support for the DOM in IE has been pretty good for a while; it's just that they didn't draw attention to it by dropping their proprietary DOM. CSS on the other hand...

However, if IE/Mozilla/Opera/etc. are competing to get these errors down then everyone wins. It's a virtuous circle; I'm mildly optimistic about the web when I see vendors compete to improve support for standard features.

Yes, it's slightly sad for Mozilla that it lags in these tests. However, it's good for everyone in the long run if there are independent tests published and people care about the results.

It's not a zero sum game. Better standard support in IE makes it easier for developers to write corss-platform code.

Re:IE is way more compatible than Mozilla?

[Anonymous Coward]

For what its worth,
I tested todays mozilla build (2-15)
and it only has 19 failures and 58 errors,
so IE and Moz are quickly converging.
I'm willing to bet that the next IE release
and mozilla 1.0 will both have 0 failures, 0 warnings.

More complicated than just counting failed tests

[CArnold]

Were you never warned not to compare apples and oranges. The raw test failure count is not a good metric (unless it is zero). Several points: 1) The DOM L1 test suite currently only contains tests that exercise the XML and SVG implementations and contains no HTML tests. For MS IE, this means testing whatever is loaded to MSXML2.DomDocument or the Adobe SVG control. For Mozilla, this means the internal XML DOM implementation. We are awaiting submission of additional tests that will test HTML implementations of DOM L1. 2) Many of the Mozilla test failures are due to Mozilla not parsing the DTD and providing default values for attributes. 3) This is the first of several test suites for different levels and modules of the DOM. Don't be making conclusions just on the basis of the L1 DOM Core suite. 4) Exceptions throw to the JScript binding from MSXML do not adhere to the DOMException interface (from the public documentation for Windows Scripting, it does not appear possible to craft the exception class that is generated for a COM failure HRESULT). The test adapter for MSXML compensates for this. The compensation is justified by the third paragraph in http://www.w3.org/TR/2000/WD-DOM-Level-1-20000929/ level-one-core.html#ID-17189187 which allows for the limitation of object system. If this compensation is eliminated, then MSXML will fail every test that checks the code for an exception which will radically change the raw numbers (find MSXMLDocumentBuilder_isDOMExceptionCode(ex, code) in DOMTestSuite.js and add "return (ex.code == code);" in case you are curious.) So, don't compare apples to oranges, don't rely on default attribute values for Mozilla's XML DOM and don't rely on DOMException.code on MSXML. p.s. I've participated in the DOM Test Suite work, but the opinions expressed in this message are my personal opinions and not that of the DOM Test Suite group or any other group of the W3C. p.s.s. A unofficial superficial analysis of the test failures for an earlier version of Mozilla against an working draft versions is available at http://lists.w3.org/Archives/Public/www-dom-ts/200 1Nov/0011.html