[whatwg] [editing] New conformance tests
ayg at aryeh.name
Wed Sep 14 14:24:20 PDT 2011
On Fri, Sep 9, 2011 at 12:48 PM, Bjartur Thorlacius
<svartman95 at gmail.com> wrote:
> It freezes surf as well, and then crashes it (though that's clearly a bug in
> down severely. The results are quite a bit bigger than my copy buffer, and
> polite ML messages, though :) Maybe my Opera build is just plain outdated,
> but I felt the numbers might be interesting either way.
> Opera 10.60 Internal. Build 6386 for Linux.
> Compiled on Jun 30 2010 by gcc 4.3.2 (ABI: 1002) for GNU libc 2.7.
> Time elapsed: 2:38.857 min.
> Found 56720 tests
> 13685 Pass
> 43035 Fail
Apparently I need a faster CPU:
Time elapsed: 15:38.332 min.Summary
Found 56734 tests14107 Pass42627 Fail
By way of comparison, in Firefox 8.0a2:
Time elapsed: 4:51.723 min.
Found 56734 tests
Annoyingly, it looks like Opera is failing some tests solely due to
not following the HTML5 serialization algorithm. I rely on innerHTML
heavily for the tests. I could try to work around these failures, but
hopefully IE and Opera will migrate to HTML5 serialization soon enough
that it won't be worth the effort.
On Tue, Sep 13, 2011 at 6:25 PM, Ehsan Akhgari <ehsan at mozilla.com> wrote:
> This looks really interesting. Do you think this conformance suite is in a
> good stage for browser vendors to integrate it in their test suites? I'm
> interested in integrating this into Mozilla's test suite to make sure that
> we don't regress anything covered by these tests without realizing it.
It's very likely that some of the tests are wrong, or that they match
the spec but the spec is wrong. With that kept in mind, yes, it
should be suitable for integration into browser test suites. It uses
James Graham's testharness.js, so I should think you'd want to make
some bridge that lets that integrate into your test suites, then plug
these in. testharness.js is designed for this use-case, but as far as
I know, only Opera uses it that way at this point. You might want to
speak to James about it if you want to work on integrating it. There
are lots more tests that use testharness.js, such as these:
And more are being written regularly. This is now the preferred test
format for the HTML and WebApps WGs, so it would be great if browsers
More information about the whatwg