[imps] Cross browser automated tests
sylvain.pasche at gmail.com
Wed May 21 16:06:01 PDT 2008
Adam Barth wrote:
> The WebKit LayoutTestController might be a reasonable starting point
> for a testing API. Just calling parent.reportResults might be
> limiting for security testing because a frame might not be able to
> access its parent. For example, the LayoutTestController has a
> globally readable and writable boolean flag that helps cross-domain
> frames signal that the test is complete.
One fact to keep in mind is that all browsers won't be able to provide
something like a LayerTestController object to web pages (but having
both WebKit and Firefox do that is already a good coverage).
However, testing some features requires that a privileged API is
available for the testcases (synthesizing privileged events, clearing
caches, ...). One idea would be to offer a generic privileged API that
browsers could implement to some degree. Then, tests should be flagged
in some way to describe what privileged API they require to be run.
They would be skipped on browsers not implementing that. To have the
widest browser coverage, tests should only require these privileged API
For the WebKit security tests, being able to run them on a browser where
something like the LayoutTestController is not available seems
difficult. In the experiments I did to run the WebKit tests, I loaded
them in a frame, and extracted the page text to compare against the
-expected.txt file. This approach won't work directly for testing top
>> ...and using <iframe>s to embed the tests one after another.
> This is also limiting for security tests because some behavior must be
> tested in a top-level frame.
>> This makes the test even easier to maintain, and also makes it a lot
>> easier to reimplement the harness or to share tests between hardnesses.
> One of our goals for the project is to integrate with Firefox's
> testing harness. Ideally, the Firefox source tree could check out the
> WebKit LayoutTests unmodified and run them as part of their
> per-checkin regression testing.
I had a look at your extension. Out of curiosity, did you develop some
tool like run-webkit-tests for loading pages inside Firefox and
comparing the actual and expected .txt files?
More information about the Implementors