[whatwg] Restarting the media element resource fetch algorithm after "load" event

[Philip Jägenstedt]

On Thu, 08 Oct 2009 19:42:14 +0200, Eric Carlson <eric.carlson at apple.com>  
wrote:

>
> On Oct 8, 2009, at 5:32 AM, Philip Jägenstedt wrote:
>
>> On Thu, 08 Oct 2009 12:10:01 +0200, Robert O'Callahan  
>> <robert at ocallahan.org
>> > wrote:
>>
>>> Another issue is that it's not completely clear to me what is meant
>>> by
>>> "While the user agent might still need network access to obtain
>>> parts of the
>>> media  
>>> resource<http://www.whatwg.org/specs/web-apps/current-work/#media-resource
>>> >..."
>>> What if there is data in the resource that we don't need in order to
>>> play through normally, but which might be needed in some special
>>> situations
>>> (e.g., enabling subtitles, or seeking using an index), and we
>>> optimize to
>>> not load that data unless/until we need it? In that case would we
>>> never
>>> reach NETWORK_LOADED?
>>
>> As I understand it, NETWORK_LOADED means that all bytes of the
>> resource have been loaded, regardless of whether they will be used
>> or not. Are there any formats that would actually allow not
>> downloading parts of the resource in a meaningful way?
>
>    Yes. A disabled track in an MPEG-4 or QuickTime file is not
> rendered so the data is not used when presenting the movie. Media data
> for an enabled but invisible video track (eg. size 0x0, or not within
> the visible region) or an enabled but muted audio track isn't
> technically needed for the presentation either.

The question was rather of such tracks can be multiplexed in a way that  
make it possible to skip them efficiently. As per below, probably yes.

>> Subtitles and indexes are too small to bother, and multiplexed audio/
>> video tracks can hardly be skipped without zillions of HTTP Range
>> requests. It seems to me that kind of thing would have to be done
>> either with a server side media fragment request (using the 'track'
>> dimension) or with an external audio/video track somehow synced to
>> the master track (much like external subtitles).
>>
>    I don't agree that this is necessarily best done on a server. Some
> file formats include tables with the location of every sample, so a
> media engine that uses range requests anyway can easily read just the
> data needed. It might be wise for such an engine to optimize the size
> of chunks read from the server, but that is an implementation detail.
>
>    Also remember that "multiplexed" is a relative term, different
> chunking/interleaving schemes make sense for different media types and
> use cases so not all multiplexed files interleave data in small chunks.

Right, if the chunks are large enough (at least 100s of KB I suppose) it  
might be worthwhile using range requests.

>>> In general NETWORK_LOADED and the "load" event seem rather useless
>>> and
>>> dangerous IMHO. If you're playing a resource that doesn't fit in
>>> your cache
>>> then you'll certainly never reach NETWORK_LOADED, and since authors
>>> can't
>>> know the cache size they can never rely on "load" firing. And if
>>> you allow
>>> the cache discarding behavior I described above, authors can't rely
>>> on data
>>> actually being present locally even after "load" has fired.
>
>    If data can be evicted from the cache you can never reach
> NETWORK_LOADED because "Network connectivity could be lost without
> affecting the media playback".
>
>
>>> I suspect many
>>> authors will make invalid assumptions about "load" being sure to
>>> fire and
>>> about what "load" means if it does fire. Does anyone have any use
>>> cases that
>>> "load" actually solves?
>>
>    I also agree that the 'load' event and the NETWORK_LOADED state are
> not terribly useful and will likely cause a great deal of confusion
> for developers. We have have seen a number of cases where experienced
> web developers have used the 'load' event when they should have used
> the 'canplaythough', and I fear that this will be a common mistake.
>
>
>> I agree, sites that depend on the load event sites will likely break
>> randomly for file sizes that usually barely fit into the cache of
>> the browser they were tested with. If browsers are conservative with
>> bandwidth and only send the load event when it's true, I think we
>> will have less of a problem however.
>>
>
>    I don't agree that it will be any less of a problem if browsers are
> conservative, users will still not *ever* be able to depend on the
> 'load' event firing (except perhaps for local files).

Precisely, if browsers don't fire 'load' until the last data is loaded  
when playback is closing in on completion, people can't accidentally use  
it instead of 'canplay'. Still, for local files, very small files or in  
combination with autobuffer the problem will still be the same.

>> Note that the load event isn't strictly needed, waiting for a
>> progress event with loaded==total would achieve the same thing.
>>
>    Actually, a progress event with loaded==total tells you even less
> than the 'load' event because it doesn't guarantee that the data won't
> be evicted from the cache.

Good point.

>>  Aesthetically, however, I think it would be strange to not have the
>> load event.
>
>    I am not worried about the aesthetics of not having the event.  I
> am somewhat concerned about existing content that uses it (including
> many of the WebKit layout tests :-( ), but I think we will be better
> off in the long run if we get rid of the event and network state now.

I wouldn't be particularly opposed to dropping the load event, unless  
there's a use case for the guarantee that the resource won't be evicted  
 from cache as long as the page is loaded. Also I'm not sure if it would  
contradict the progress events spec to not have a load event (even though  
arguably that spec shouldn't have those kinds of requirements).

-- 
Philip Jägenstedt
Opera Software