[whatwg] <video> feedback

[Silvia Pfeiffer]

On Wed, Oct 3, 2012 at 6:41 AM, Jer Noble <jer.noble at apple.com> wrote:
> On Sep 17, 2012, at 12:43 PM, Ian Hickson <ian at hixie.ch> wrote:
>
>> On Mon, 9 Jul 2012, adam k wrote:
>>>
>>> i have a 25fps video, h264, with a burned in timecode.  it seems to be
>>> off by 1 frame when i compare the burned in timecode to the calculated
>>> timecode.  i'm using rob coenen's test app at
>>> http://www.massive-interactive.nl/html5_video/smpte_test_universal.html
>>> to load my own video.
>>>
>>> what's the process here to report issues?  please let me know whatever
>>> formal or informal steps are required and i'll gladly follow them.
>>
>> Depends on the browser. Which browser?
>>
>>
>>> i'm aware that crooked framerates (i.e. the notorious 29.97) were not
>>> supported when frame accuracy was implemented.  in my tests, 29.97DF
>>> timecodes were incorrect by 1 to 3 frames at any given point.
>>>
>>> will there ever be support for crooked framerate accuracy?  i would be
>>> more than happy to contribute whatever i can to help test it and make it
>>> possible.  can someone comment on this?
>>
>> This is a Quality of Implementation issue, basically. I believe there's
>> nothing inherently in the API that would make accuracy to such timecodes
>> impossible.
>
> TLDR; for precise navigation, you need to use a a rational time class, rather than a float value.
>
> The nature of floating point math makes precise frame navigation difficult, if not impossible.  Rob's test is especially hairy, given that each frame has a timing bound of [startTime, endTime), and his test attempts to navigate directly to the startTime of a given frame, a value which gives approximately zero room for error.
>
> I'm most familiar with MPEG containers, but I believe the following is also true of the WebM container: times are represented by a rational number, timeValue / timeScale, where both numerator and denominator are unsigned integers.


FYI: the Ogg container also uses rational numbers to represent time.


>  To seek to a particular media time, we must convert a floating-point time value into this rational time format (e.g. when calculating the 4th frame's start time, from "3 * 1/29.97" to "3 * 1001/30000").  If there is a floating-point error in the wrong direction (e.g., as above, a numerator of 3002 vs 3003), the end result will not be the frame's startTime, but one timeScale before it.
>
> We've fixed some frame accuracy bugs in WebKit (and Chromium) by carefully rounding the incoming floating point time value, taking into account the media's time scale, and rounding to the nearest 1/timeScale value.  This fixes Rob's precision test, but at the expense of precision. (I.e. in a 30 fps movie, "currentTime = 0.999999 / 30" will navigate to the second frame, not the first, due to rounding, which is technically incorrect.)
>
> This is a common problem, and Apple media frameworks (for example) therefore provide rational time classes which provide enough accuracy for precise navigation (e.g. QTTime, CMTime). Using a floating point number to represent time with any precision is not generally accepted as good practice when these rational time classes are available.
>
> -Jer