[whatwg] HTML5 video: frame accuracy / SMPTE

[Chris Pearce]

On 22/01/2011 7:31 a.m., Gregory Maxwell wrote:
> It's usually the decoding, not the file access that kill you.  Firefox
> seeking on low resolution clips is very snappy index or not. But try a
> 1080p clip encoded with a 10 second maximum keyframe interval...

This is true. If you want fast frame accurate seeking, particularly over 
the internet, it's best to not encode with such a large keyframe 
interval. This is a problem caused by a webdev's inappropriate encoding 
choice, not by a bad API choice.

If seeking is slow when you encode with a large keyframe interval, don't 
encode with a large keyframe interval!

What if the browser's controls by default seeked to the previous 
keyframe, if the user had enough precision in the controls and wanted to 
seek to 9 seconds after a keyframe (1 second before the subsequent 
keyframe), then they could not.

On 22/01/2011 10:04 a.m., Philip Jägenstedt wrote:
> Since, as you say, the behavior is currently inconsistent, there is 
> still time to agree on something that makes sense and have everyone 
> implement that. I think the best default is keyframe seeking and 
> haven't seen any strong arguments for accurate seeking as the default 
> yet.

I disagree. The default should be exact, with "approximate" seeking to 
the nearest keyframe.  When you call video.seek(X.xx), you've specified 
an exact time, and would likely expect an exact time, so the media 
should seek to that exact time. Another reason to make exact the 
default, is that if the media is seekable, it can always be seeked 
exactly, whereas media without an index may not be able to be seeked 
approximately.

We've already implemented frame accurate seeking in Firefox. I'd be 
happy for us to implement approximate seeking, it would be useful for 
seeking into areas of the media which are unbuffered. We may change our 
controls to use approximate seeking when seeking into unbuffered areas, 
but we wouldn't use approximate seeking when seeking into buffered 
areas. Videos are usually short enough that you want accuracy higher 
than keyframe granularity when seeking using the default controls (at 
least in buffered areas).


On 22/01/2011 10:50 a.m., Roger Hågensen wrote:
> Hmm. I think the default (nothing) should be synonymous with 
> "best-effort" (or "best") and leave it to the 
> browser/os/codec/format/etc. as to what "best effort" actually is.

We should specify the default, otherwise by default webdevs will always 
need to specify their seek accuracy level.



On 22/01/2011 11:25 a.m., Silvia Pfeiffer wrote:
>>> * KEYFRAME is keyframe-accurate seeking, so to the previous keyframe
>> What does this mean when a seekable stream doesn't have interior
>> keyframes? Should the client always seek to the beginning? Why is this
>> valuable over a "fast" option?
> Where no keyframes are available, this seek option simply doesn't do
> anything, since obviously there are not keyframes. The point is that
> where this concept exists and people want to take advantage of it,
> this seek should be possible.
>

Some media format's index for audio as well, so where there are no 
keyframes, you could seek to the nearest indexed point from the index. 
Or (probably a better idea) have some way of reporting whether keyframe 
level approximate seeking is available.

Exact seeking is always available if the media is seekable (so it makes 
a good default...), and script can determine whether faster but 
approximate seeking is available and can choose to use it.


On 22/01/2011 1:47 p.m., Roger Hågensen wrote:
>
> To me only TIME makes sense right now as HH:II:SS.MS 
> (hours:minutes:seconds.milliseconds) and FRAME if <1ms for rare cases 
> where video is more than 1000fps.

I don't think there's much (or any?) demand for seeking in anything 
other than seconds?

> So maybe just let the flag be default and nothing else, but as 
> mentioned previously, leave it an enum "just in case" for the future 
> (I'm thinking possible future timing standards that might appear, 
> though it's hard to beat doubles really).

We can always add it later if demand arises. HTML is a living standard 
right? ;)


On 22/01/2011 11:05 a.m., Gregory Maxwell wrote:
> The potentially high cost of an exact seek is the primary reason why I
> wouldn't want to make the default behavior mandate exact, but exact
> still needs to be available.

Seeks would only be "slow" in the case where keyframes were infrequent. 
In many cases a "snap to keyframe" seek would be inaccurate enough to be 
annoying with infrequent keyframes. I can only imagine the bugs we'll 
get filed if we make it the default!


Chris Pearce.