[whatwg] HTML5 video: frame accuracy / SMPTE

[Gregory Maxwell]

On Fri, Jan 21, 2011 at 8:19 PM, Chris Pearce <chris at pearce.org.nz> wrote:
> On 22/01/2011 7:31 a.m., Gregory Maxwell wrote:
>> It's usually the decoding, not the file access that kill you.  Firefox
>> seeking on low resolution clips is very snappy index or not. But try a
>> 1080p clip encoded with a 10 second maximum keyframe interval...
>
> This is true. If you want fast frame accurate seeking, particularly over the
> internet, it's best to not encode with such a large keyframe interval. This
> is a problem caused by a webdev's inappropriate encoding choice, not by a
> bad API choice.

One second of decoding time is already a fairly slow seek. Surely you
aren't suggesting that people should be using one-to-sub-second
keyframe intervals?

~Two second intervals are a norm for conservative intervals. It's not
unusual for keyframes to be 7x the size of delta frames for low motion
HD footage, and going lower than this starts to seriously inflate the
file size for a given quality (e.g. you're looking at something like a
11% file size boost from going from 64 to 32). A two second seek is
slow enough to be annoying, but thats what you'll get on a pretty
typical file with a pretty conservative keyframe interval which can
only be decoded in realtime.

I can equally turn your argument around and say that setting the
keyframe interval so long that it's unacceptable for seeking is a
problem of a webdev's inappropriate encoding choice.

In either case— an inaccurate seek or a slow exact seek you end up
wasting the error's worth of additional time. In the inaccurate case
it's irrelevant if your desired seek wasn't accurate to being with
(e.g. clicking on a small seek bar), and regardless you'll spend that
time watching output. In the exact case a faster than realtime decode
may reduce the waiting but whatever waiting you do you'll spend
watching the spinner.  (Though I full agree that when the
decode&download is much faster than realtime (e.g. 10x) that exact is
the way to go)

[snip]
> Seeks would only be "slow" in the case where keyframes were infrequent. In
> many cases a "snap to keyframe" seek would be inaccurate enough to be
> annoying with infrequent keyframes. I can only imagine the bugs we'll get
> filed if we make it the default!

Inaccurate seeking is pretty much universal in media-player
applications. ::shrugs:: I haven't conducted an exhaustive survey
across systems, but on my desktop firefox is the only tool that does
frame accurate seeking by default. Totem doesn't, VLC doesn't, Mplayer
doesn't.

I'm very happy that Firefox can do exact seeking and I've bragged
about it, but I think you're underestimating the cost. Cortado (which
doesn't do accurate seeking) makes firefox seeking look like a joke—
it plays pretty much as fast as you can click, even over a slow
network connection— and with sane keyframe intervals no one even
notices that it's not accurate.