Clashing Scores In the HTML5 Compatibility Test Wars

Andreas(R) writes "Microsoft has published a set of HTML5 tests comparing Internet Explorer 9 to other web browsers. In Microsoft's own tests, IE9 performs 100% on all tests. However, the Internet Explorer 9 HTML5 Canvas Campaign has published results that show that Internet Explorer gets 0% on all their tests." The results reported here are selected with tongue in cheek: "Therefore, we'll also present shameless results from tests which have been carefully selected to give the results that the PR department has demanded."

Build Your Own Test

[eldavojohn]

First off if this is a technical discussion, we should probably be talking about layout engines -- not browsers. Secondly their HTML5 capabilities are well documented [wikipedia.org]. You can come up with whatever perventage you want from those charts as some things (Video) might be deal breakers compared to others (MathML).

It's also worth mentioning...

[The MAZZTer]

...that they benchmarked IE trunk against OLD versions of other browsers. They didn't even use Chrome 5.0!

In some places it's a significant difference. [withinwindows.com]

I also did some benchmarks of my own on non-Microsoft controlled sites. See the first comment on that page for results. Suffice it to say IE9 has improved since IE8 but still has a ways to go.

Re:Here's how to solve the impasse

[maxume]

There is plenty of agreement in the html5 draft, lots of it is not controversial.

There certainly is not complete agreement.

Re:Here's how to solve the impasse

[thestudio_bob]

Hi. Have you heard of this thing called "Google". It's pretty amazing really.

http://html5test.com/ [html5test.com]

HTML5TEST

[v1]

http://html5test.com/ [html5test.com]

things like this will have to do until we see something like ACID support HTML5.

Re:IE has 100% compatability...

[Anonymous Coward]

Or false reading, the microsoft page in question says:

This website contains several collections of new test pages that we developed in conjunction with the World Wide Web Consortium (W3C) working groups. These 192 test pages have been updated based on feedback and now include some new HTML5 test pages.

Re:test results are largely irrelevant anyway

[Ralish]

Wait, what? No Windows Service Pack has ever forced an update of Internet Explorer; maybe NT 4.0 did as I can't remember that far back, but definitely nothing since Windows 2000 onwards. Windows XP SP3 will install fine with IE 6.0 (XP bundled version). They'd be breaking their own support policy by even doing so, as Microsoft commits to supporting the version of IE that is shipped with every Windows version for the lifetime of support for that OS release. Seriously, where do you trolls get your garbage? You're not picking exceptions, you're claiming shit that has never happened.

Re:IE has 100% compatability...

[CannonballHead]

Is this an ad? What are they selling?

Re:test results are largely irrelevant anyway

[CannonballHead]

As another user pointed out, IE upgrades are not forced. They are perhaps put into the "recommended" (but I think they are in "optional," now I don't remember) updates, but you are not forced to upgrade. I can run XP SP3 and click "No" when it asks if I want IE8, and nothing bad has happened.

Re:IE has 100% compatability...

[Anonymous Coward]

Bad software.

Re:Here's how to solve the impasse

[petermgreen]

Note that site tests things that aren't actally required by the current drafts of the spec. e.g. support for particular audio/video codecs.

Re:The Difference

[UnknowingFool]

At best this demonstrates misrepresentation. MS is like a student who says that they passed did better in Math than their peers. They even have a handy chart. What MS doesn't tell you is that they only tested specific skills like quadratic equations. What missing from this is that Chrome/Opera/Firefox/Safari took the whole Math test from algebra/geometry/trig/calculus/etc while MS only attempted the handful of questions it knew it solve.

Re:Build Your Own Test

[matthewv789]

Here is a better listing of specific capabilities for different browser versions: http://caniuse.com/ [caniuse.com] It's still unclear how competitive IE9 will be at the time it comes out, but it will clearly be a huge leap forward from IE8. (The big problem will be getting all the existing IE6, 7, and 8 users to migrate to IE9 or other modern browsers.)

Re:test results are largely irrelevant anyway

[dangitman]

Like IE8, IE7 and IE6 before them, windows users will be forced to upgrade to 9 sooner or later anyway.

Yeah, what the hell? Most Windows users I deal with who aren't running Firefox, are running IE6 on XP.

I took the time to read the source of the tests.

[GNUALMAFUERTE]

The reason why most tests failed with browsers other than IE:

1st) Since HTML5 is still in a very early state, many browsers (AKA Webkit, Gecko, Presto) used prefixes for most tags and CSS properties. Example: round borders is -moz-border-radius in gecko, and -webkit-border-radius in Chrome. Some latest versions have taken some out of beta and also read border-radius, but most still don't. IE obviously uses border-radius, and that's why other's don't work.
2nd) The JS is tricky at best. Go and check it out. Lots of lines of code to perform a simple task, and those lines are carefully selected to fail in other browsers. I downloaded the tests, and they work on ALL browsers (I tested Chrome, Firefox and Opera, all on GNU/Linux, all on their latest version). That JS was crafted to fail on all browsers and work only on IE
3rd) I took the time to run the source of many of their scripts through the W3C validator. Most scripts have several warnings, some errors, etc. They DO NOT VALIDATE.
4th) The tests aren't really HTML5. Only the HTML5 tests are actual HTML5, the others are XHTML 1.0 strict ... except they are not, because they use HTML5 styles and tags, and they do not validate. Validator says: The document located at was tentatively checked as XHTML 1.0 Strict. This means that with the use of some fallback or override mechanism, we successfully performed a formal validation using an SGML, HTML5 and/or XML Parser(s). In other words, the document would validate as XHTML 1.0 Strict if you changed the markup to match the changes we have performed automatically, but it will not be valid until you make these changes.

It's microsoft ... never forget about that. This is business as usual.

Re:I took the time to read the source of the tests

[GNUALMAFUERTE]

Sorry to reply to myself, but I forgot a few things:

1st: The actual ietestcenter fails validation with 12 errors and 6 warnings: http://validator.w3.org/check?uri=http://samples.msdn.microsoft.com/ietestcenter/&charset=(detect+automatically)&doctype=Inline&group=0 [w3.org]

Including some serious ones, like no Character encoding specified.

None of the tests specify a character encoding either.

Re:Here's how to solve the impasse

[GoochOwnsYou]

There isn't full agreement but most of it is pretty complete. The only non nitpicking issue that people cant agree on is video/audio. Microsoft and Apple want push h.264 into their browsers and push h.264 as a de-facto standard so they advocate against defining a codec in HTML5 (an open standard). Of course they dont support anything else in IE and Safari(for HTML5 video tags)

The other camp: mostly the open source community push for Ogg containers (Theora/Vorbis), despite h.264 is a superior codec Microsoft and Apple have mostly been attakcing it with patent FUD. Opera and Firefox are in the Ogg camp.

While Google has been cooking up VP8, an open codec that is supposed to be on par with h.264. Chrome contains support for Ogg and h.264 and likely in the future VP8 will be adopted by Chrome, Firefox and Opera.

At the end of the day we need an open specific video standard: otherwise it fails to solve the problem Flash started (breaking standards).

Re:It's also worth mentioning...

[RAMMS+EIN]

``Chrome is an immature browser based on one of the newer rendering engines, so we expect it to mature rapidly, but hardly can expect it to match it's cousin Safari in most areas, thous we expect it would in a short time.''

I think that depends on how you look at it. Chrome's rendering engine is based on an older version of the rendering engine from Safari, which is in turn based on KHTML from Konqueror, which was forked from khtmlw in 1998, making it about as old as the Gecko engine used by Mozilla. While Chrome's engine has supported the multi-process model for some time, Safari's only started on that in April 2010, I think. So if you are looking for maturity, I am not so sure Safari is a better bet than Chrome.

Re:The Difference

[notrandomly]

If I am not mistaken, Acid3 was created by an Opera employee at the time (who now works for Google), and was meant to be extremely tough to pass for all browsers.

Re:IE has 100% compatability...

[DragonWriter]

In Australia (where I am from) an advertisement needs to either have a disclaimer (normally small text at the bottom) if there is vagueness about what it is saying, or what the advertisement says needs to be taken at face value - meaning "what it implies".

Surely the US would have that too?

No, the US instead has a strong Constitutional guarantee of free speech which sharply limits government prior restraint of speech, even for commercial speech; disclaimers are rarely required except for products in certain particularly tightly regulated industries.

A misleading ad might provide a basis for a fraud claim from someone who bought a product based on the advertisement, which is -- rather than any specific requirement for disclaimers -- most of the disclaimers seen in US ads are provided, but even then (due to the fact that, e.g., "puffery" is found to be protected) the ad has to be pretty blatantly false to provide a strong likelihood that such a claim would succeed.