[whatwg] HTML5 video: frame accuracy / SMPTE

[Philip Jägenstedt]

On Sat, 22 Jan 2011 02:19:10 +0100, Chris Pearce <chris at pearce.org.nz>  
wrote:

>
> On 22/01/2011 7:31 a.m., Gregory Maxwell wrote:
>> It's usually the decoding, not the file access that kill you.  Firefox
>> seeking on low resolution clips is very snappy index or not. But try a
>> 1080p clip encoded with a 10 second maximum keyframe interval...
>
> This is true. If you want fast frame accurate seeking, particularly over  
> the internet, it's best to not encode with such a large keyframe  
> interval. This is a problem caused by a webdev's inappropriate encoding  
> choice, not by a bad API choice.
>
> If seeking is slow when you encode with a large keyframe interval, don't  
> encode with a large keyframe interval!
>
> What if the browser's controls by default seeked to the previous  
> keyframe, if the user had enough precision in the controls and wanted to  
> seek to 9 seconds after a keyframe (1 second before the subsequent  
> keyframe), then they could not.

Simple heuristics could solve this:

HTMLMediaElement.prototype.magicSeek = function(t) {
   if (Math.abs(t - this.currentTime) < 10)
     this.seek(t, "accurate");
   else
     this.seek(t, "fast");
}

Or some refinement thereof...

> On 22/01/2011 10:04 a.m., Philip Jägenstedt wrote:
>> Since, as you say, the behavior is currently inconsistent, there is  
>> still time to agree on something that makes sense and have everyone  
>> implement that. I think the best default is keyframe seeking and  
>> haven't seen any strong arguments for accurate seeking as the default  
>> yet.
>
> I disagree. The default should be exact, with "approximate" seeking to  
> the nearest keyframe.  When you call video.seek(X.xx), you've specified  
> an exact time, and would likely expect an exact time, so the media  
> should seek to that exact time. Another reason to make exact the  
> default, is that if the media is seekable, it can always be seeked  
> exactly, whereas media without an index may not be able to be seeked  
> approximately.

Nitpick: If you can't seek approximately then simply not doing it would be  
fine, there's no requirement for a fast seek to always miss the target.

> We've already implemented frame accurate seeking in Firefox. I'd be  
> happy for us to implement approximate seeking, it would be useful for  
> seeking into areas of the media which are unbuffered. We may change our  
> controls to use approximate seeking when seeking into unbuffered areas,  
> but we wouldn't use approximate seeking when seeking into buffered  
> areas. Videos are usually short enough that you want accuracy higher  
> than keyframe granularity when seeking using the default controls (at  
> least in buffered areas).

OK, having the feature is more important than the default.

> On 22/01/2011 10:50 a.m., Roger Hågensen wrote:
>> Hmm. I think the default (nothing) should be synonymous with  
>> "best-effort" (or "best") and leave it to the  
>> browser/os/codec/format/etc. as to what "best effort" actually is.
>
> We should specify the default, otherwise by default webdevs will always  
> need to specify their seek accuracy level.

Indeed.

> On 22/01/2011 11:25 a.m., Silvia Pfeiffer wrote:
>>>> * KEYFRAME is keyframe-accurate seeking, so to the previous keyframe
>>> What does this mean when a seekable stream doesn't have interior
>>> keyframes? Should the client always seek to the beginning? Why is this
>>> valuable over a "fast" option?
>> Where no keyframes are available, this seek option simply doesn't do
>> anything, since obviously there are not keyframes. The point is that
>> where this concept exists and people want to take advantage of it,
>> this seek should be possible.
>>
>
> Some media format's index for audio as well, so where there are no  
> keyframes, you could seek to the nearest indexed point from the index.  
> Or (probably a better idea) have some way of reporting whether keyframe  
> level approximate seeking is available.
>
> Exact seeking is always available if the media is seekable (so it makes  
> a good default...), and script can determine whether faster but  
> approximate seeking is available and can choose to use it.

Do we really need to make this feature testable? Why not just let them  
request fast seeking and give them what's possible?

-- 
Philip Jägenstedt
Core Developer
Opera Software