[whatwg] ***DHSPAM*** re-thinking "cue ranges"

[Ian Hickson]

On Thu, 22 May 2008, Dave Singer wrote:
>
> WARNING:  this email is sent to both the WhatWG and W3C Public HTML 
> list, as it is a proposal.  Please be careful about where you 
> reply/follow-up to.  The editors may have a preference (and if they do, 
> I hope they express it).

My preference is to one list, either is fine. I've only cc'ed whatwg on 
this one, chosen merely because that's where most of the video discussion 
has been recently.


> In the current HTML5 draft cue ranges are available using a DOM API.
> 
> This way of doing ranges is less than ideal.
> 
> First of all, it is hard to use. The ranges must be added by script, 
> can't be supplied with the media, and the callbacks are awkward to 
> handle. The only way to identify the range a received callback applies 
> to is by creating not one but two separate functions for each range: one 
> for enter, one for exit. While creating functions on-demand is easy in 
> JavaScript it does fall under advanced techniques that most authors will 
> be unfamiliar with.

One of the features proposed for the next version of the video API is 
chapter markers and other embedded timed metadata, with corresponding 
callbacks for authors to hook into. Would that resolve the problem you 
mention?


> This kind of feature is also not available in all languages that might 
> provide access to the DOM API.

JavaScript is really the only concern from HTML5's point of view; if other 
languages become relevant, they should get specially-crafted APIs for 
them when it comes to this kind of issue.


> Secondly this mechanism is not very powerful. You can't do anything else 
> with the ranges besides receiving callbacks and removing them. You can't 
> modify them. They are not visible to scripts or CSS. You can't link to 
> them. You can't link out from them.

I'm not sure what it would really mean to link to or from a range, unless 
you turned the entire video into a link, in which case you can just wrap 
the <video> in an <a href=""> element for the duration of the range, using 
script.


> Thirdly, a script is somewhat strange place to define the ranges. A set 
> of ranges usually relates closely to some particular piece of media 
> content. The same set of ranges rarely makes much sense in the context 
> of some other content. It seems that ranges should be defined or 
> supplied along with the media content.

For in-band data, callbacks for chapter markers as mentioned earlier seem 
like the best solution.

For out-of-band data, if the ranges are just intended to trigger script, I 
don't think we gain much from providing a way to mark up ranges semi- 
declaratively as opposed to just having HTML-based media players define 
their own range markup and have them implement it using this API. It 
wouldn't be especially hard.


> Fourth, this kind of callback API is pretty strange creature in the HTML 
> specification. The only other callback APIs are things like setTimeout() 
> and the new SQL API which don't have associated elements. Events are the 
> callback mechanism for everything else.

Events use callbacks themselves, so it's not that unusual.

I don't really think events would be a good interface for this. 
Consistency is good, but if one can come up with a better API, it's better 
to use that than just be consistent for the sake of it.



> In SMIL the equivalent concept is the <area> element which is used like this:
> <video src="http://www.example.org/CoolStuff">
>            <area id="area1" begin="0s" end="5s"/>
>            <area id="area2" begin="5s" end="10s"/>
> </video>
> 
> This kind of approach has several advantages.
> * Ranges are defined as part of the document, in the context of a particular
> media stream.

I'm not sure why that is an advantage in the context of HTML.


> * This uses events, a more flexible and more appropriate callback mechanism.

I don't really see why the flexibility of events is useful here, and I 
don't see why it's more appropriate.


> * The callbacks have a JavaScript object associated with them, namely a DOM
> element, which carries information about the range.

That's useful, yes. Should we include some data with the callback? We 
could include the class name, the start time, and the end time. Having 
said that, it's easy to use currying here to hook callbacks that know what 
they're expecting.


> We would like to suggest a <timerange> element that can be used as a 
> child of the <video> and <audio> elements.

It's not clear to me that this is solving any problems worth solving.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'