[whatwg] HTML5 video: frame accuracy / SMPTE

[Philip Jägenstedt]

On Mon, 24 Jan 2011 21:10:21 +0100, Robert O'Callahan  
<robert at ocallahan.org> wrote:

> On Mon, Jan 24, 2011 at 11:32 PM, Philip Jägenstedt  
> <philipj at opera.com>wrote:
>
>> On Mon, 24 Jan 2011 02:36:15 +0100, Robert O'Callahan <
>> robert at ocallahan.org> wrote:
>>
>>  Interop problems are going to arise with approximate seeking no matter
>>> what
>>> we do, which is why it shouldn't be the default.
>>>
>>
>> OK, let's go with that, then.
>
>
> Yay! :-)
>
>
>>  You don't want seekApproximate(T) to always land on the same T'. For
>>> example, if a player wants to "seek forward approximately N seconds"  
>>> via
>>> seekApproximate(currentTime + N), but the approximation to currentTime  
>>> + N
>>> is fixed to be some T' less than or equal to currentTime, that would be
>>> broken.
>>>
>>> I would say that seekApproximate(T) should be specified to aim as  
>>> close as
>>> possible to T while being "fast", but the only guarantee is that it  
>>> seeks
>>> somewhere after currentTime if T > currentTime, or somewhere before
>>> currentTime if T < currentTime.
>>>
>>
>> 1. There's a race condition here, seekApproximate(currentTime+0.001)  
>> will
>> be quite random since currentTime changes while the script is running.
>>
>
> Interesting. It doesn't in Firefox; script always sees a snapshot of a
> consistent state until it returns to the event loop or does something  
> modal
> (although audio, and soon video, will continue to play while script  
> runs).
> I'm not sure if the spec should require that ... overall our APIs try  
> pretty
> hard not to expose races to JS.

How does that work? Do you take a copy of all properties that could  
possibly change during script execution, including ones that create a new  
object, like buffered and seekable? If you instead only make a copy on the  
first read, isn't it still possible to get an inconsistent state, e.g.  
where currentTime isn't in the buffered ranges?

How about HTMLImageElement.complete, which the spec explicitly says can  
change during script execution?

In any case, it sounds like either HTMLMediaElement is underspecified or  
one of us has interpreted in incorrectly, some interop on this point would  
be nice.

> But maybe you want seekApproximateRelative(deltaT) then...
>
> 2. No media framework I've worked with seems to provide fast seeking with
>> any guarantee of direction, so it might be hard to actually implement  
>> this
>> way on many platforms, while fast seeking in general is very easy.
>>
>
> Perhaps you could implement it as "seek fast to T + deltaT, then if that
> didn't land us on the right side of currentTime (hopefully rare), do an
> exact seek to T + deltaT"?
>
> If approximate seeking can't be used to reliably implement "go  
> forward/back
> a bit", then it'll be significantly less useful IMHO.

The biggest use case is clicking a seek bar and ending up somewhere close  
enough, but yes, being able to do fast relative seeking is a nice bonus.  
Maybe we should do what many media frameworks do and use a "reference"  
parameter, defining what the seek is relative to. Usually you can seek  
relative to the beginning, end and current position, but perhaps we could  
reduce that to just "absolute" and "relative". That's a bit less magic  
than inspecting currentTime when the method is called.

So far:

seek(t, ref, how);

ref is "absolute" (default) or "relative"

how is "accurate" (default) or "fast"

(or numeric enums, if that's what DOM interfaces usually have)

-- 
Philip Jägenstedt
Core Developer
Opera Software