[whatwg] HTML5 Tokenizer Test Cases and Correct Output

[James Graham]

On 03/28/2013 12:06 PM, Mohammad Al Houssami (Alumni) wrote:
> Hello everyone.
>
> I was wondering if there is some sort of tests for the Tokenizer along with the correct output of tokens as well as a way of representing tokens.
> What I have in mind is running the tokenizer on some HTML input and printing the tokens in the same way the correct output is written.
> I will  then be comparing the result I have with the correct one provided character by character. :)
>
http://code.google.com/p/html5lib/source/browse/#hg%2Ftestdata%2Ftokenizer

http://wiki.whatwg.org/wiki/Parser_tests has some documentation of the 
format.