[whatwg] HTML5 Tokenizer Test Cases and Correct Output

[Mohammad Al Houssami (Alumni)]

Hello everyone.

I was wondering if there is some sort of tests for the Tokenizer along with the correct output of tokens as well as a way of representing tokens.
What I have in mind is running the tokenizer on some HTML input and printing the tokens in the same way the correct output is written.
I will  then be comparing the result I have with the correct one provided character by character. :)

Thanks in advance.
Mohammad