[whatwg] [media] startOffsetTime, also add startTime?

Fri Mar 9 06:40:26 PST 2012

On Thu, 08 Mar 2012 19:16:40 +0100, Ian Hickson <ian at hixie.ch> wrote:

>
> (Oops, sorry. Missed these e-mails in my earlier reply.)
>
> On Thu, 8 Mar 2012, Philip Jägenstedt wrote:
>> On Wed, 07 Mar 2012 11:56:42 +0100, Odin Hørthe Omdal
>> <odinho at opera.com> wrote:
>> >
>> > startOffsetTime seem to leave people confused, I often have to explain
>> > it, and yesterday I read the spec[5] and old emails and got confused
>> > myself. It hasn't been implemented after almost 2 years.
>>
>> We (Opera) have wanted to implement this for a long time, but it has
>> been stalled by the fact that the spec is confusing to the point that we
>> haven't been able to agree on what it's actually trying to say. Let's
>> fix that.
>
> I'm happy to make it clearer, but it seems clear to me. What are your
> interpretations, so that I can explicitly rule out in the spec the ones
> that are not intended?

Excellent, see below.

>> I agree that it would be useful to expose the constant by which
>> timestamps are adjusted
>
> Time stamps should not be adjusted.
>
>
>> to guarantee that that currentTime starts at 0 and ends at duration.
>
> That is not what the spec requires.
>
>
>> I think that both a name like startTime (or initialTime) would suggest
>> that it is the initial value of currentTime, which it is not.
>
> initialTime is the initial value of currentTime.
>
>
>> I suggest the property offsetTime, defined as the stream time in seconds
>> which currentTime and duration are relative to.
>
> I don't understand what this means. The currentTime is relative to the
> media timeline, which is UA-defined and "should" be based on the media
> timeline.

The BBC wrote a blog post [1] about how currentTime varies between Firefox  
and Chrome. Opera does the same as Firefox here. You're right, however,  
that the way "media timeline" doesn't make any guarantee that currentTime  
starts at 0 or that duration is the duration. I think that the  
implementations predate the "media timeline" concept, and I agree with the  
BBC blog post that the Opera/Firefox behavior is better. Controls written  
assuming that currentTime goes from 0 to duration won't break and duration  
will actually mean duration.

>> In practice it would often be understood as the "time since the server
>> began streaming" and would be useful to sync live streams with
>> out-of-band content simply by letting the out-of-band content be
>> relative to the start of the stream.
>
> That "should" be zero. I can change that to a "must" if you like; it's
> a "should" because in some cases (e.g. MJPEG) you don't know what the
> media timeline is or how to interpret it, so there's no way to do it.

Which "should" are you referring to here?

>> No round-trip with Date representations should be necessary in the
>> common case.
>
> The startOffsetTime attribute is intended for display, no? Why would you
> round-trip with it?
>
>
>> As hinted above, I don't think that startOffsetTime should really be the
>> first choice for trying to sync live streams.
>
> Indeed.

I really don't know what startOffsetTime is intended for. AFAICT it's a  
piece of metadata that you could just as well provide out-of-band, but for  
convenience it is exposed via the DOM API. I think it could be handy to  
have and would like to implement it, but I don't understand if it's any  
different from other metadata like producer or location of a video.

>> However, knowing the date of a video is still useful, potentially even
>> for the streaming case, so we do want to expose the DateUTC field from
>> WebM. However, startOffsetTime is a bad name for it, since it's not
>> using the same unit as currentTime. I suggest offsetDate, to go with
>> offsetTime.
>
> I don't mind renaming startOffsetTime if people think that would help. I
> don't think "offsetDate" is any clearer though.
>
> How about "mediaTimelineOriginDate"?

Simply "originDate" or "startDate", perhaps? It could also do with a good  
example. The spec says:

"If the media resource specifies an explicit start time and date, then  
that time and date should be considered the zero point in the media  
timeline; the timeline offset will be the time and date, exposed using the  
startOffsetTime attribute."

I interpret this as a date at currentTime=0 in the spec's definition of  
currentTime, and currentTime=-initialTime (unless media fragments are  
used) in the Opera/Firefox definition of currentTime. However, there's a  
weird spec example which can lead one into thinking otherwise:

"The startOffsetTime attribute would return a Date object with a time  
corresponding to 2010-03-20 23:15:00 UTC. However, if a different user  
agent connected five minutes later, it would (presumably) receive  
fragments covering timestamps 2010-03-20 23:20:00 UTC to 2010-03-21  
00:05:00 UTC and 2010-02-12 14:25:00 UTC to 2010-02-12 14:35:00 UTC, and  
would expose this with a media timeline starting at 0s and extending to  
3,300s (fifty five minutes)."

This seems like a rather atypical streaming scenario. It would be a lot  
nicer if the single example of startOffsetTime was for the common scenario  
where each client gets the same stream that thus has the same timeline and  
the same startOffsetTime.

>> Finally, what about initialTime? It can be set to a non-zero value at
>> two points in the spec:
>>
>> "Establish the media timeline for the purposes of the current playback
>> position, the earliest possible position, and the initial playback
>> position, based on the media data."
>>
>> "If either the media resource or the address of the current media
>> resource indicate a particular start time, then set the initial playback
>> position to that time and"
>>
>> Does any format expose something like this in-band? I don't know of any
>> that do and how to implement this, so the only thing that remains is
>> exposing the start time of media fragments. This seems rather useless to
>> me, so unless someone has already implemented initialTime and explain
>> what it means, I suggest dropping it from the spec.
>
> The address of the current media resource can indicate a particular start
> time if you implement media fragments.

Yes, but why do we need to expose that in the DOM API, what is the use  
case? For media fragments I think it's just as well to parse the URL to  
get the end time as well, while the initial value of currentTime can  
trivially be saved in the loadedmetadata event handler. It would certainly  
help if the spec didn't suggest that initialTime can be given in-band,  
unless there are formats that support this. Unless initialTime solves a  
problem, just dropping it would be preferable, of course.

>> We discussed the concatenation of two clips and how to represent the
>> date. At least chained WebM and chained Ogg should be able to represent
>> this.
>
> The spec requires ("must") that in the case of chained clips with
> discontinuous timelines, the first clip's timeline be extended to cover
> the others, and any data regarding the timeline in the subsequest clips  
> is
> dropped.

So the second and subsequent clips of a chain have their timelines  
normalized, but not the first?

>> To reduce the possibility for confusion about what date is represented
>> and to allow the recording date to be preserved in editing, how about
>> exposing currentDate instead?
>
> What's the use case?

The use case is "don't be confusing", so let me first try to summarize  
what I think the spec says:

* currentTime need not start at 0, for streams it will typically represent  
for how long the server has been serving a stream.

* duration is not the duration, it is the last timestamp of a resource.

* startOffsetTime is the date at time 0, it's not an offset. It has  
nothing to do with syncing live streams.

* initialTime is the first timestamp of the stream or the start time of a  
media fragment URL, if one is used.

* For chained streams, the 2nd and subsequent clips have their timelines  
normalized and appended to the first clips timeline.

Is that correct?

[1]  
http://www.bbc.co.uk/blogs/researchanddevelopment/2012/02/what-does-currenttime-mean-in.shtml

-- 
Philip Jägenstedt
Core Developer
Opera Software