[whatwg] <video> resource selection algorithm and NETWORK_NO_SOURCE

[Philip Jägenstedt]

On Mon, 26 Jul 2010 15:02:34 +0200, Silvia Pfeiffer  
<silviapfeiffer1 at gmail.com> wrote:

> On Mon, Jul 26, 2010 at 7:49 PM, Philip Jägenstedt  
> <philipj at opera.com>wrote:
>
>> On Mon, 26 Jul 2010 02:58:07 +0200, Silvia Pfeiffer <
>> silviapfeiffer1 at gmail.com> wrote:
>>
>>  On Sun, Jul 25, 2010 at 3:31 PM, Maciej Stachowiak <mjs at apple.com>  
>> wrote:
>>>
>>>
>>>> On Jul 23, 2010, at 7:16 AM, Philip Jägenstedt wrote:
>>>>
>>>> Silvia made we aware of discrepancy in how browsers implement the
>>>> resource
>>>> selection algorithm, see forwarded message. It's my assessment that  
>>>> Opera
>>>> is
>>>> the only browser following the spec. I've filed this bug with Mozilla:
>>>>
>>>> https://bugzilla.mozilla.org/show_bug.cgi?id=581355
>>>>
>>>> I've also reported bugs in Chrome and Safari, but can't see where they
>>>> ended up.
>>>>
>>>> The reason I'm writing this email is that apparently everyone but  
>>>> myself
>>>> has a different interpretation of the spec, so perhaps this is  
>>>> something
>>>> we
>>>> need to discuss. Does any other browser ever set the state
>>>> NETWORK_NO_SOURCE
>>>> at all? I speculated that perhaps other browsers aren't very strict  
>>>> about
>>>> which parts of the algorithm are run synchronously and not, but even
>>>> checking the networkState after a setTimeout it still isn't
>>>> NETWORK_NO_SOURCE.
>>>>
>>>> Test case:  
>>>> http://people.opera.com/philipj/2010/07/23/networkState.html
>>>>
>>>> Please fix implementation or spec :)
>>>>
>>>>
>>>> 1) Which behavior is more useful?
>>>> 2) Sylvia's original issue was with play() - should we ensure that any
>>>> time
>>>> you call play(), it will cause the media resource to start playing  
>>>> once
>>>> loaded? That seems like the real spec bug.
>>>>
>>>>
>>> The problem there with play in Opera was that the @src was changed, but
>>> Opera doesn't run the resource selection algorithm to load it for  
>>> play(),
>>> but instead expects to run load() first. The description of play()  
>>> however
>>> clearly states as a first step:
>>> 1. If the media
>>> element<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>> >'s
>>> networkState<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#dom-media-networkstate
>>> >attribute
>>> has the value
>>> NETWORK_EMPTY<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#dom-media-network_empty
>>> >,
>>> invoke the media
>>> element<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>> >'s
>>> resource selection
>>> algorithm<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#concept-media-load-algorithm
>>> >
>>>
>>> .
>>>
>>> This is why we are discussing whether the networkState needs to be
>>> NETWORK_EMPTY or NETWORK_NO_SOURCE after initialisation of a media
>>> element.
>>>
>>> Incidentally, that has effects on other elements, too, such as for
>>> <source>
>>> it is stated:
>>> If a source<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#the-source-element
>>> >element
>>>
>>> is inserted as a child of a media
>>> element<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>> >that
>>> has no
>>> src<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#attr-media-src
>>> >attribute
>>> and whose
>>> networkState<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#dom-media-networkstate
>>> >has
>>> the value
>>> NETWORK_EMPTY<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#dom-media-network_empty
>>> >,
>>>
>>> the user agent must invoke the media
>>> element<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>> >'s
>>> resource selection
>>> algorithm<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#concept-media-load-algorithm
>>> >
>>>
>>> .
>>>
>>> And for @src it is said:
>>> If a src<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#attr-media-src
>>> >attribute
>>> of a media
>>> element<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>> >is
>>>
>>> set or changed, the user agent must invoke the media
>>> element<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>> >'s
>>> media element load
>>> algorithm<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element-load-algorithm
>>> >.
>>> (*Removing* the
>>> src<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#attr-media-src
>>> >attribute
>>>
>>> does not do this, even if there are
>>> source<
>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#the-source-element
>>> >elements
>>>
>>> present.)
>>>
>>> That is kinda strange, too, because really when @src is changed,  
>>> resource
>>> selection has to be run first rather than media element load.
>>>
>>
>> The load algorithm is just a wrapper for the resource selection  
>> algorithm
>> that makes sure that a previously running resource selection algorithm  
>> is
>> aborted in an orderly fashion and that the state is reset. What's  
>> strange?
>>
>
> I misread that, too. I thought it was pointing to the resource fetch
> algorithm and not the media element load algorithm. You're right - this  
> part
> is fine.
>
>
> Incidentally, your test isn't complete. In my test file, I was actually
> setting the @src values of the <source> elements to resources that  
> existed
> and then called the play() function. That activity should, IMO, work. All
> browser accept it, except for Opera, which never leaves step 21 because
> play() doesn't accept NETWORK_NO_SOURCE for re-running the resource
> selection algorithm. That was really my original problem.

I agree that it'd be good if this worked, all else equal. Unfortunately,  
making it work would require adding hooks for when the src attribute of  
source elements are changed. Since you want to change all of the src  
attributes before resource selection runs, you'd either have to delay its  
invocation (a.k.a. await a stable state) or have a situation where the  
first source is potentially processed 3 times, the second 2 times and the  
last 1 time.

My solution:

     <video controls width="400px">
     </video>
     <script type="text/javascript">
       var video = document.querySelector("video");
       var exts = ["mp4", "webm", "ogv"];
       exts.forEach(function(ext) {
         var source = document.createElement("source");
         source.src = "HelloWorld."+ext;
         source.type = "video/"+ext;
         video.appendChild(source);
       });
       video.play();
     </script>

Of course, there must be some good reason to use scripts to begin with, as  
the result of this is always the same, so you might just as well use  
static markup.

-- 
Philip Jägenstedt
Core Developer
Opera Software