[whatwg] <video> resource selection algorithm and NETWORK_NO_SOURCE
Philip Jägenstedt
philipj at opera.com
Mon Jul 26 08:37:11 PDT 2010
On Mon, 26 Jul 2010 16:53:43 +0200, Silvia Pfeiffer
<silviapfeiffer1 at gmail.com> wrote:
> On Mon, Jul 26, 2010 at 11:25 PM, Philip Jägenstedt
> <philipj at opera.com>wrote:
>
>> On Mon, 26 Jul 2010 15:02:34 +0200, Silvia Pfeiffer <
>> silviapfeiffer1 at gmail.com> wrote:
>>
>> On Mon, Jul 26, 2010 at 7:49 PM, Philip Jägenstedt <philipj at opera.com
>>> >wrote:
>>>
>>> On Mon, 26 Jul 2010 02:58:07 +0200, Silvia Pfeiffer <
>>>> silviapfeiffer1 at gmail.com> wrote:
>>>>
>>>> On Sun, Jul 25, 2010 at 3:31 PM, Maciej Stachowiak <mjs at apple.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Jul 23, 2010, at 7:16 AM, Philip Jägenstedt wrote:
>>>>>>
>>>>>> Silvia made we aware of discrepancy in how browsers implement the
>>>>>> resource
>>>>>> selection algorithm, see forwarded message. It's my assessment that
>>>>>> Opera
>>>>>> is
>>>>>> the only browser following the spec. I've filed this bug with
>>>>>> Mozilla:
>>>>>>
>>>>>> https://bugzilla.mozilla.org/show_bug.cgi?id=581355
>>>>>>
>>>>>> I've also reported bugs in Chrome and Safari, but can't see where
>>>>>> they
>>>>>> ended up.
>>>>>>
>>>>>> The reason I'm writing this email is that apparently everyone but
>>>>>> myself
>>>>>> has a different interpretation of the spec, so perhaps this is
>>>>>> something
>>>>>> we
>>>>>> need to discuss. Does any other browser ever set the state
>>>>>> NETWORK_NO_SOURCE
>>>>>> at all? I speculated that perhaps other browsers aren't very strict
>>>>>> about
>>>>>> which parts of the algorithm are run synchronously and not, but even
>>>>>> checking the networkState after a setTimeout it still isn't
>>>>>> NETWORK_NO_SOURCE.
>>>>>>
>>>>>> Test case:
>>>>>> http://people.opera.com/philipj/2010/07/23/networkState.html
>>>>>>
>>>>>> Please fix implementation or spec :)
>>>>>>
>>>>>>
>>>>>> 1) Which behavior is more useful?
>>>>>> 2) Sylvia's original issue was with play() - should we ensure that
>>>>>> any
>>>>>> time
>>>>>> you call play(), it will cause the media resource to start playing
>>>>>> once
>>>>>> loaded? That seems like the real spec bug.
>>>>>>
>>>>>>
>>>>>> The problem there with play in Opera was that the @src was changed,
>>>>> but
>>>>> Opera doesn't run the resource selection algorithm to load it for
>>>>> play(),
>>>>> but instead expects to run load() first. The description of play()
>>>>> however
>>>>> clearly states as a first step:
>>>>> 1. If the media
>>>>> element<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>>>> >'s
>>>>> networkState<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#dom-media-networkstate
>>>>> >attribute
>>>>> has the value
>>>>> NETWORK_EMPTY<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#dom-media-network_empty
>>>>> >,
>>>>> invoke the media
>>>>> element<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>>>> >'s
>>>>> resource selection
>>>>> algorithm<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#concept-media-load-algorithm
>>>>> >
>>>>>
>>>>> .
>>>>>
>>>>> This is why we are discussing whether the networkState needs to be
>>>>> NETWORK_EMPTY or NETWORK_NO_SOURCE after initialisation of a media
>>>>> element.
>>>>>
>>>>> Incidentally, that has effects on other elements, too, such as for
>>>>> <source>
>>>>> it is stated:
>>>>> If a source<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#the-source-element
>>>>> >element
>>>>>
>>>>> is inserted as a child of a media
>>>>> element<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>>>> >that
>>>>> has no
>>>>> src<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#attr-media-src
>>>>> >attribute
>>>>> and whose
>>>>> networkState<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#dom-media-networkstate
>>>>> >has
>>>>> the value
>>>>> NETWORK_EMPTY<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#dom-media-network_empty
>>>>> >,
>>>>>
>>>>> the user agent must invoke the media
>>>>> element<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>>>> >'s
>>>>> resource selection
>>>>> algorithm<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#concept-media-load-algorithm
>>>>> >
>>>>>
>>>>> .
>>>>>
>>>>> And for @src it is said:
>>>>> If a src<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#attr-media-src
>>>>> >attribute
>>>>> of a media
>>>>> element<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>>>> >is
>>>>>
>>>>> set or changed, the user agent must invoke the media
>>>>> element<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element
>>>>> >'s
>>>>> media element load
>>>>> algorithm<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#media-element-load-algorithm
>>>>> >.
>>>>> (*Removing* the
>>>>> src<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#attr-media-src
>>>>> >attribute
>>>>>
>>>>> does not do this, even if there are
>>>>> source<
>>>>>
>>>>> http://www.whatwg.org/specs/web-apps/current-work/complete/video.html#the-source-element
>>>>> >elements
>>>>>
>>>>> present.)
>>>>>
>>>>> That is kinda strange, too, because really when @src is changed,
>>>>> resource
>>>>> selection has to be run first rather than media element load.
>>>>>
>>>>>
>>>> The load algorithm is just a wrapper for the resource selection
>>>> algorithm
>>>> that makes sure that a previously running resource selection
>>>> algorithm is
>>>> aborted in an orderly fashion and that the state is reset. What's
>>>> strange?
>>>>
>>>>
>>> I misread that, too. I thought it was pointing to the resource fetch
>>> algorithm and not the media element load algorithm. You're right - this
>>> part
>>> is fine.
>>>
>>>
>>> Incidentally, your test isn't complete. In my test file, I was actually
>>> setting the @src values of the <source> elements to resources that
>>> existed
>>> and then called the play() function. That activity should, IMO, work.
>>> All
>>> browser accept it, except for Opera, which never leaves step 21 because
>>> play() doesn't accept NETWORK_NO_SOURCE for re-running the resource
>>> selection algorithm. That was really my original problem.
>>>
>>
>> I agree that it'd be good if this worked, all else equal. Unfortunately,
>> making it work would require adding hooks for when the src attribute of
>> source elements are changed. Since you want to change all of the src
>> attributes before resource selection runs, you'd either have to delay
>> its
>> invocation (a.k.a. await a stable state) or have a situation where the
>> first
>> source is potentially processed 3 times, the second 2 times and the
>> last 1
>> time.
>>
>> My solution:
>>
>>
>> <video controls width="400px">
>> </video>
>> <script type="text/javascript">
>> var video = document.querySelector("video");
>> var exts = ["mp4", "webm", "ogv"];
>> exts.forEach(function(ext) {
>> var source = document.createElement("source");
>> source.src = "HelloWorld."+ext;
>> source.type = "video/"+ext;
>> video.appendChild(source);
>> });
>> video.play();
>> </script>
>>
>
>
> Does this actually work in Opera now?
Yes, when I have a HelloWorld.webm file available it starts playing. It
also works in Firefox 4b1 and it should work in Chrome and Safari too
unless they are buggy.
>> Of course, there must be some good reason to use scripts to begin with,
>> as
>> the result of this is always the same, so you might just as well use
>> static
>> markup.
>
>
>
> Sure, but this is only a snippet of an actual application. If, e.g., you
> want to step through a list of videos (maybe an automated playlist) using
> script and you need to provide at least two different formats with
> <source>,
> you'd want to run this algorithm frequently. While this is a good
> solution,
> authors may still end up doing it in sequence, in which case I guess you
> may
> indeed need to run resource selection three times: once every time a @src
> attribute is changed on a <source> element.
Right, I realize there are actually situations where it isn't a pointless
exercise as in the above. If you're already using scripts, though, you
could actually call canPlayType yourself and use the first one that works:
exts.forEach(function(ext) {
if (video.canPlayType("video/"+ext)) {
video.src = "movie_300."+ext;
return false;
}
});
I expect this will be interoperable right now and it's easier to
understand exactly what's going on.
Looking again at the resource selection algorithm, the second step is to
await a stable state, i.e. wait until the current script has finished.
Given that, it wouldn't be a big problem to let modification of src
attributes on source elements trigger resource selection, you won't get
the 3-2-1 problem I mentioned earlier. However, then I would argue that
modifying type and media should also have the same effect, as those affect
the outcome of resource selection. In the end, my suggestion is still no
spec change, except for editorial changes to clarify.
--
Philip Jägenstedt
Core Developer
Opera Software
More information about the whatwg
mailing list