[whatwg] HTML resource packages

[Justin Lebar]

> Can you provide the content of the page which you used in your whitepaper?
> (https://bug529208.bugzilla.mozilla.org/attachment.cgi?id=455820)

I'll post this to the bug when I get home tonight.  But your comments
are astute -- the page I used is a pretty bad benchmark for a variety
of reasons.  It sounds like you probably could hack up a much better
one.

>    a) Looks like pages were loaded exactly once, as per your notes?  How
> hard is it to run the tests long enough to get to a 95% confidence interval?

Since I was running on a simulated network with no random parameters
(e.g. no packet loss), there was very little variance in load time
across runs.

>    d) What did you do about subdomains in the test?  I assume your test
> loaded from one subdomain?

That's correct.

> I'm betting time-to-paint goes through the roof with resource bundles:-)

It does right now because we don't support incremental extraction,
which is why I didn't bother measuring time-to-paint.  The hope is
that with incremental extraction, we won't take too much of a hit.

-Justin

On Mon, Aug 9, 2010 at 1:30 PM, Mike Belshe <mike at belshe.com> wrote:
> Justin -
> Can you provide the content of the page which you used in your whitepaper?
> (https://bug529208.bugzilla.mozilla.org/attachment.cgi?id=455820)
> I have a few concerns about the benchmark:
>    a) Looks like pages were loaded exactly once, as per your notes?  How
> hard is it to run the tests long enough to get to a 95% confidence interval?
>    b) As you note in the report, slow start will kill you.  I've verified
> this so many times it makes me sick.  If you try more combinations, I
> believe you'll see this.
>    c) The 1.3MB of subresources in a single bundle seems unrealistic to me.
>  On one hand you say that its similar to CNN, but note that CNN has
> JS/CSS/images, not just thumbnails like your test.  Further, note that CNN
> pulls these resources from multiple domains; combining them into one domain
> may work, but certainly makes the test content very different from CNN.  So
> the claim that it is somehow representative seems incorrect.   For more
> accurate data on what websites look like,
> see http://code.google.com/speed/articles/web-metrics.html
>    d) What did you do about subdomains in the test?  I assume your test
> loaded from one subdomain?
>    e) There is more to a browser than page-load-time.  Time-to-first-paint
> is critical as well.  For instance, in WebKit and Chrome, we have specific
> heuristics which optimize for time-to-render instead of total page load.
>  CNN is always cited as a "bad page", but it's really not - it just has a
> lot of content, both below and above the fold.  When the user can interact
> with the page successfully, the user is happy.  In other words, I know I can
> make webkit's PLT much faster by removing a couple of throttles.  But I also
> know that doing so worsens the user experience by delaying the time to first
> paint.  So - is it possible to measure both times?  I'm betting
> time-to-paint goes through the roof with resource bundles:-)
> If you provide the content, I'll try to run some tests.  It will take a few
> days.
> Mike
>
> On Mon, Aug 9, 2010 at 9:52 AM, Justin Lebar <justin.lebar at gmail.com> wrote:
>>
>> On Mon, Aug 9, 2010 at 9:47 AM, Aryeh Gregor <Simetrical+w3c at gmail.com>
>> wrote:
>> > If UAs can assume that files with the same path
>> > are the same regardless of whether they came from a resource package
>> > or which, and they have all but a couple of the files cached, they
>> > could request those directly instead of from the resource package,
>> > even if a resource package is specified.
>>
>> These kinds of heuristics are far beyond the scope of resource
>> packages as we're planning to implement them.  Again, I think this
>> type of behavior is the domain of a large change to the networking
>> stack, such as SPDY, not a small hack like resource packages.
>>
>> -Justin
>>
>> On Mon, Aug 9, 2010 at 9:47 AM, Aryeh Gregor <Simetrical+w3c at gmail.com>
>> wrote:
>> > On Fri, Aug 6, 2010 at 7:40 PM, Justin Lebar <justin.lebar at gmail.com>
>> > wrote:
>> >> I think this is a fair point.  But I'd suggest we consider the
>> >> following:
>> >>
>> >> * It might be confusing for resources from a resource package to show
>> >> up on a page which doesn't "opt-in" to resource packages in general or
>> >> to that specific resource package.
>> >
>> > Only if the resource package contains a different file from the real
>> > one.  I suggest we treat this as a pathological case and accept that
>> > it will be broken and confusing -- or at least we consider how many
>> > extra optimizations we could make if we did accept that, before
>> > deciding whether the extra performance is worth the confusion.
>> >
>> >> * There's no easy way to opt out of this behavior.  That is, if I
>> >> explicitly *don't* want to load content cached from a resource
>> >> package, I have to name that content differently.
>> >
>> > Why would you want that, if the files are the same anyway?
>> >
>> >> * The avatars-on-a-forum use case is less convincing the more I think
>> >> about it.  Certainly you'd want each page which displays many avatars
>> >> to package up all the avatars into a single package.  So you wouldn't
>> >> benefit from the suggested caching changes on those pages.
>> >
>> > I don't see why not.  If UAs can assume that files with the same path
>> > are the same regardless of whether they came from a resource package
>> > or which, and they have all but a couple of the files cached, they
>> > could request those directly instead of from the resource package,
>> > even if a resource package is specified.  So if twenty different
>> > people post on the page, and you've been browsing for a while and have
>> > eighteen of their avatars (this will be common, a handful of people
>> > tend to account for most posts in a given forum):
>> >
>> > 1) With no resource packages, you fetch two separate avatars (but on
>> > earlier page views you suffered).
>> >
>> > 2) With resource packages as you suggest, you fetch a whole resource
>> > package, 90% of which you don't need.  In fact, you have to fetch a
>> > resource package even if you have 100% of the avatars on the page!  No
>> > two pages will be likely to have the same resource package, so you
>> > can't share cache at all.
>> >
>> > 3) With resource packages as I suggest, you fetch only two separate
>> > avatars, *and* you got the benefits of resource packages on earlier
>> > pages.  The UA gets to guess whether using resource packages would be
>> > a win on a case-by-case basis, so in particular, it should be able to
>> > perform strictly better than either (1) or (2), given decent
>> > heuristics.  E.g., the heuristic "fetch the resource package if I need
>> > at least two files, fetch the file if I only need one" will perform
>> > better than either (1) or (2) in any reasonable circumstance.
>> >
>> > I think this sort of situation will be fairly common.  Has anyone
>> > looked at a bunch of different types of web pages and done a breakdown
>> > of how many assets they have, and how they're reused across pages?  If
>> > we're talking about assets that are used only on one page (image
>> > search) or all pages (logos, shared scripts), your approach works
>> > fine, but not if they're used on a random mix of pages.  I think a lot
>> > of files will wind up being used on only particular subsets of pages.
>> >
>> >> In general, I think we need something like SPDY to really address the
>> >> problem of duplicated downloads.  I don't think resource packages can
>> >> fix it with any caching policy.
>> >
>> > Certainly there are limits to what resource packages can do, but we
>> > can wind up closer to the limits or farther from them depending on the
>> > implementation details.
>> >
>
>