[whatwg] HTML5 video: frame accuracy / SMPTE

[Michael Dale]

It seems the needs for a seek bar are different from the api usage of
setting video.currentTime. As people mention rules of 'least surprise'
are important. If you set currentTime to 3.453 you expect it to return
something very close to 3.453 ..

Why not have a separate api for "seek bars" that just seek to a
percentage of the video stream. Letting implementations do what's
fastest and consistent, but without guarantees for a given play time
being set. This way the different needs are addressed and we don't
conflate the currentTime api with parameters for accuracy.

In terms of real world use cases, we use accurate seeks in an html5
video editor. Its especially important when flattening with firefogg
that as we can accurately set a given time in source videos per our
output frame interval.
http://commons.wikimedia.org/wiki/Commons:Sequencer

--michael


On 01/22/2011 03:41 AM, Philip Jägenstedt wrote:
> On Sat, 22 Jan 2011 02:19:10 +0100, Chris Pearce <chris at pearce.org.nz>
> wrote:
>
>>
>> On 22/01/2011 7:31 a.m., Gregory Maxwell wrote:
>>> It's usually the decoding, not the file access that kill you.  Firefox
>>> seeking on low resolution clips is very snappy index or not. But try a
>>> 1080p clip encoded with a 10 second maximum keyframe interval...
>>
>> This is true. If you want fast frame accurate seeking, particularly
>> over the internet, it's best to not encode with such a large keyframe
>> interval. This is a problem caused by a webdev's inappropriate
>> encoding choice, not by a bad API choice.
>>
>> If seeking is slow when you encode with a large keyframe interval,
>> don't encode with a large keyframe interval!
>>
>> What if the browser's controls by default seeked to the previous
>> keyframe, if the user had enough precision in the controls and wanted
>> to seek to 9 seconds after a keyframe (1 second before the subsequent
>> keyframe), then they could not.
>
> Simple heuristics could solve this:
>
> HTMLMediaElement.prototype.magicSeek = function(t) {
>   if (Math.abs(t - this.currentTime) < 10)
>     this.seek(t, "accurate");
>   else
>     this.seek(t, "fast");
> }
>
> Or some refinement thereof...
>
>> On 22/01/2011 10:04 a.m., Philip Jägenstedt wrote:
>>> Since, as you say, the behavior is currently inconsistent, there is
>>> still time to agree on something that makes sense and have everyone
>>> implement that. I think the best default is keyframe seeking and
>>> haven't seen any strong arguments for accurate seeking as the
>>> default yet.
>>
>> I disagree. The default should be exact, with "approximate" seeking
>> to the nearest keyframe.  When you call video.seek(X.xx), you've
>> specified an exact time, and would likely expect an exact time, so
>> the media should seek to that exact time. Another reason to make
>> exact the default, is that if the media is seekable, it can always be
>> seeked exactly, whereas media without an index may not be able to be
>> seeked approximately.
>
> Nitpick: If you can't seek approximately then simply not doing it
> would be fine, there's no requirement for a fast seek to always miss
> the target.
>
>> We've already implemented frame accurate seeking in Firefox. I'd be
>> happy for us to implement approximate seeking, it would be useful for
>> seeking into areas of the media which are unbuffered. We may change
>> our controls to use approximate seeking when seeking into unbuffered
>> areas, but we wouldn't use approximate seeking when seeking into
>> buffered areas. Videos are usually short enough that you want
>> accuracy higher than keyframe granularity when seeking using the
>> default controls (at least in buffered areas).
>
> OK, having the feature is more important than the default.
>
>> On 22/01/2011 10:50 a.m., Roger Hågensen wrote:
>>> Hmm. I think the default (nothing) should be synonymous with
>>> "best-effort" (or "best") and leave it to the
>>> browser/os/codec/format/etc. as to what "best effort" actually is.
>>
>> We should specify the default, otherwise by default webdevs will
>> always need to specify their seek accuracy level.
>
> Indeed.
>
>> On 22/01/2011 11:25 a.m., Silvia Pfeiffer wrote:
>>>>> * KEYFRAME is keyframe-accurate seeking, so to the previous keyframe
>>>> What does this mean when a seekable stream doesn't have interior
>>>> keyframes? Should the client always seek to the beginning? Why is this
>>>> valuable over a "fast" option?
>>> Where no keyframes are available, this seek option simply doesn't do
>>> anything, since obviously there are not keyframes. The point is that
>>> where this concept exists and people want to take advantage of it,
>>> this seek should be possible.
>>>
>>
>> Some media format's index for audio as well, so where there are no
>> keyframes, you could seek to the nearest indexed point from the
>> index. Or (probably a better idea) have some way of reporting whether
>> keyframe level approximate seeking is available.
>>
>> Exact seeking is always available if the media is seekable (so it
>> makes a good default...), and script can determine whether faster but
>> approximate seeking is available and can choose to use it.
>
> Do we really need to make this feature testable? Why not just let them
> request fast seeking and give them what's possible?
>