[whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

On Wed, 07 Mar 2012 11:56:42 +0100, Odin Hørthe Omdal <odinho at opera.com>  
wrote:

> startOffsetTime seem to leave people confused, I often have to explain  
> it, and yesterday I read the spec[5] and old emails and got confused  
> myself. It hasn't been implemented after almost 2 years.

We (Opera) have wanted to implement this for a long time, but it has been  
stalled by the fact that the spec is confusing to the point that we  
haven't been able to agree on what it's actually trying to say. Let's fix  
that.

> Having the UTC time of the clip you're getting would be very useful. But  
> it'd be really nice to get the start of the non-normalized timestamp  
> exposed to javascript for synchronizing out-of-band metadata with the  
> live streamed media.
>
> Browsers are currently supposed to take the timestamp and normalize it  
> to 0 for currentTime. Chromium currently does not do that; it starts at  
> 3:15, if I join a streamed video that I started streaming 3 minutes, 15  
> seconds ago.
>
> I don't think using the UTC time as the sync point is very stable at the  
> moment. It'd be a much quicker, stable, and easier win to get a  
> startTime, timelineStartTime or timeSinceStart or similar that exposes  
> the NON-normalized timestamp value at the start of the stream. So that,  
> if you do
>
>     startTime + currentTime
>
> you're able to get the actual timestamp that the stream is at, at that  
> point. And in contrast with startOffsetTime this one won't ever change,  
> so startTime + currentTime will always be continuously increasing.

I agree that it would be useful to expose the constant by which timestamps  
are adjusted to guarantee that that currentTime starts at 0 and ends at  
duration. I think that both a name like startTime (or initialTime) would  
suggest that it is the initial value of currentTime, which it is not.

I suggest the property offsetTime, defined as the stream time in seconds  
which currentTime and duration are relative to. In practice it would often  
be understood as the "time since the server began streaming" and would be  
useful to sync live streams with out-of-band content simply by letting the  
out-of-band content be relative to the start of the stream. No round-trip  
with Date representations should be necessary in the common case.

> The Date UTC which startOffsetTime would use, seems to be varying quite  
> a bit. You need to know your streaming server and what it does in order  
> to understand the result. Even different media from the same server  
> might give different results if the streaming server implementation just  
> reads the UTC time directly from the file. The information could be  
> useful, but for more advanced uses.
>
>
> startOffsetTime and initialTime came out of this conversation in 2010:
>    <http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-May/thread.html#26342>
>
> And introduced here:
>    <http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/028004.html>
>
>
> Sean O'Halpin of BBC recently mentioned[2] some of the confusion:
>
>> There seems to be some confusion here in how the HTML5 media elements  
>> specification is dealing with logical stream addressing versus physical  
>> stream addressing. The excerpt above talks about a user agent being  
>> able to "seek to an earlier point than the first frame originally  
>> provided by the server" but does not explain how this could possibly  
>> happen without communication back to the server, in which case we are  
>> effectively dealing with a request for a different physical resource.  
>> At the very least, the fact that the Firefox and Chrome teams came up  
>> with different interpretations shows that this part of the  
>> specification would benefit from clarification.
>
>
> And an earlier blog post about startOffsetTime specifically[3]:
>
>> The reason for setting this out is that we'd like to see consistent  
>> support for startOffsetTime across all commonly used codecs and for  
>> browser vendors to bring their implementations into line with the  
>> published HTML5 media elements specification. There are ambiguities in  
>> the specification itself, such as the interpretation of 'earliest  
>> seekable position', which could be clarified, especially with respect  
>> to continuous live streaming media. Browser vendors need to agree on a  
>> common interpretation of attributes such as currentTime so others can  
>> experiment with the exciting possibilities this new technology is  
>> opening up.
>
>
>
> Sooo... It would be nice to get some real cleanups to the whole media +  
> time thing. :D
>
>
>
> 1.  
> <http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#offsets-into-the-media-resource>
> 2.  
> <http://www.bbc.co.uk/blogs/researchanddevelopment/2012/02/what-does-currenttime-mean-in.shtml>
> 3.  
> <http://www.bbc.co.uk/blogs/researchanddevelopment/2012/01/implementing-startoffsettime-f.shtml>

As hinted above, I don't think that startOffsetTime should really be the  
first choice for trying to sync live streams. However, knowing the date of  
a video is still useful, potentially even for the streaming case, so we do  
want to expose the DateUTC field from WebM. However, startOffsetTime is a  
bad name for it, since it's not using the same unit as currentTime. I  
suggest offsetDate, to go with offsetTime.

Finally, what about initialTime? It can be set to a non-zero value at two  
points in the spec:

"Establish the media timeline for the purposes of the current playback  
position, the earliest possible position, and the initial playback  
position, based on the media data."

"If either the media resource or the address of the current media resource  
indicate a particular start time, then set the initial playback position  
to that time and"

Does any format expose something like this in-band? I don't know of any  
that do and how to implement this, so the only thing that remains is  
exposing the start time of media fragments. This seems rather useless to  
me, so unless someone has already implemented initialTime and explain what  
it means, I suggest dropping it from the spec.

-- 
Philip Jägenstedt
Core Developer
Opera Software