[whatwg] HTML5 video: frame accuracy / SMPTE
chris at pearce.org.nz
Fri Jan 21 17:19:10 PST 2011
On 22/01/2011 7:31 a.m., Gregory Maxwell wrote:
> It's usually the decoding, not the file access that kill you. Firefox
> seeking on low resolution clips is very snappy index or not. But try a
> 1080p clip encoded with a 10 second maximum keyframe interval...
This is true. If you want fast frame accurate seeking, particularly over
the internet, it's best to not encode with such a large keyframe
interval. This is a problem caused by a webdev's inappropriate encoding
choice, not by a bad API choice.
If seeking is slow when you encode with a large keyframe interval, don't
encode with a large keyframe interval!
What if the browser's controls by default seeked to the previous
keyframe, if the user had enough precision in the controls and wanted to
seek to 9 seconds after a keyframe (1 second before the subsequent
keyframe), then they could not.
On 22/01/2011 10:04 a.m., Philip Jägenstedt wrote:
> Since, as you say, the behavior is currently inconsistent, there is
> still time to agree on something that makes sense and have everyone
> implement that. I think the best default is keyframe seeking and
> haven't seen any strong arguments for accurate seeking as the default
I disagree. The default should be exact, with "approximate" seeking to
the nearest keyframe. When you call video.seek(X.xx), you've specified
an exact time, and would likely expect an exact time, so the media
should seek to that exact time. Another reason to make exact the
default, is that if the media is seekable, it can always be seeked
exactly, whereas media without an index may not be able to be seeked
We've already implemented frame accurate seeking in Firefox. I'd be
happy for us to implement approximate seeking, it would be useful for
seeking into areas of the media which are unbuffered. We may change our
controls to use approximate seeking when seeking into unbuffered areas,
but we wouldn't use approximate seeking when seeking into buffered
areas. Videos are usually short enough that you want accuracy higher
than keyframe granularity when seeking using the default controls (at
least in buffered areas).
On 22/01/2011 10:50 a.m., Roger Hågensen wrote:
> Hmm. I think the default (nothing) should be synonymous with
> "best-effort" (or "best") and leave it to the
> browser/os/codec/format/etc. as to what "best effort" actually is.
We should specify the default, otherwise by default webdevs will always
need to specify their seek accuracy level.
On 22/01/2011 11:25 a.m., Silvia Pfeiffer wrote:
>>> * KEYFRAME is keyframe-accurate seeking, so to the previous keyframe
>> What does this mean when a seekable stream doesn't have interior
>> keyframes? Should the client always seek to the beginning? Why is this
>> valuable over a "fast" option?
> Where no keyframes are available, this seek option simply doesn't do
> anything, since obviously there are not keyframes. The point is that
> where this concept exists and people want to take advantage of it,
> this seek should be possible.
Some media format's index for audio as well, so where there are no
keyframes, you could seek to the nearest indexed point from the index.
Or (probably a better idea) have some way of reporting whether keyframe
level approximate seeking is available.
Exact seeking is always available if the media is seekable (so it makes
a good default...), and script can determine whether faster but
approximate seeking is available and can choose to use it.
On 22/01/2011 1:47 p.m., Roger Hågensen wrote:
> To me only TIME makes sense right now as HH:II:SS.MS
> (hours:minutes:seconds.milliseconds) and FRAME if <1ms for rare cases
> where video is more than 1000fps.
I don't think there's much (or any?) demand for seeking in anything
other than seconds?
> So maybe just let the flag be default and nothing else, but as
> mentioned previously, leave it an enum "just in case" for the future
> (I'm thinking possible future timing standards that might appear,
> though it's hard to beat doubles really).
We can always add it later if demand arises. HTML is a living standard
On 22/01/2011 11:05 a.m., Gregory Maxwell wrote:
> The potentially high cost of an exact seek is the primary reason why I
> wouldn't want to make the default behavior mandate exact, but exact
> still needs to be available.
Seeks would only be "slow" in the case where keyframes were infrequent.
In many cases a "snap to keyframe" seek would be inaccurate enough to be
annoying with infrequent keyframes. I can only imagine the bugs we'll
get filed if we make it the default!
More information about the whatwg