[Implementors] Moving tests to an implementation-agnostic place

[Thomas Broyer]

Hi all,

We've been discussing in th html5lib-discuss list some minor problems
with tests as they are stored in the html5lib repository and kept in
synch with the python and ruby implementations there in, and as they
are shared with other projects (at least encoding, tokenizer and
tree-construction tests).

Recently, there has been a change to align test files format with
their documentation on the WhatWG wiki [1], which made them fail in
html5lib python (and probably ruby, though I haven't checked)
implementation.

Given that those tests are shared by other implementations, I propose
they're moved to the html5 project on Google Code [2] (which already
has an empty tests/ directory in its repository [3]), or maybe even a
new html5-tests project.

Sam Ruby said on the html5lib-discuss list:
> what I think is that the tests should reflect how we expect the software
> to behave TODAY.  If there are things that we would like to see behave
> different in the future, the tests can be coded and commented out with
> # TODO or # XXX or the like.

I totally agree with him, and it would then be easier for an
implementation to "say" (svn:external or a README file) that it passes
tests as of their revision X (and each implementation repository can
be then more easily kept in a "coherent" state; we now have tests in
the html5lib repository with people changing them –on their rights–
and thus make them fail in html5lib).

What do people think? is it realistic?

[1] http://wiki.whatwg.org/wiki/Parser_tests
[2] http://code.google.com/p/html5/
[3] http://html5.googlecode.com/svn/trunk/tests/

-- 
Thomas Broyer