[whatwg] What is the purpose of timeupdate?

[Jonas Sicking]

On Fri, Nov 6, 2009 at 1:44 AM, Philip Jägenstedt <philipj at opera.com> wrote:
> On Thu, 05 Nov 2009 21:11:15 +0100, Andrew Scherkus <scherkus at chromium.org>
> wrote:
>
>> On Thu, Nov 5, 2009 at 6:10 AM, Brian Campbell <
>> brian.p.campbell at dartmouth.edu> wrote:
>>
>>> On Nov 5, 2009, at 1:17 AM, Andrew Scherkus wrote:
>>>
>>>  On Fri, Oct 30, 2009 at 10:18 PM, Brian Campbell <
>>>>
>>>> Brian.P.Campbell at dartmouth.edu> wrote:
>>>> As a multimedia developer, I am wondering about the purpose of the
>>>> timeupdate event on media elements. On first glance, it would appear
>>>> that
>>>> this event would be useful for synchronizing animations, bullets,
>>>> captions,
>>>> UI, and the like. The spec specifies a rate of 4 to 66 Hz for these
>>>> events.
>>>> The high end of this (30 or more Hz) is pretty reasonable for displaying
>>>> things in sync with the video. The low end, however, 4 Hz, is far too
>>>> slow
>>>> for most types of synchronization; everything feels laggy at this
>>>> frequency.
>>>> From my testing on a two year old MacBook Pro, Firefox is giving me
>>>> about 25
>>>> timeupdate events per second, while Safari and Chrome are giving me the
>>>> bare
>>>> minimum, of 4 timeupdate events per second.
>>>>
>>>> At 4 timeupdate events per second, it isn't all that useful. I can
>>>> replace
>>>> it with setInterval, at whatever rate I want, query the time, and get
>>>> the
>>>> synchronization I need, but that makes the timeupdate event seem to be
>>>> redundant. At 25 timeupdate events per second, it is reasonably useful,
>>>> and
>>>> can be used to synchronize various things to the video.
>>>>
>>>> So, I'm wondering if there's a purpose for the timeupdate event that I'm
>>>> missing. If it is intended for any sort of synchronization with the
>>>> video, I
>>>> think it should be improved to give better guarantees on the interval
>>>> between updates, or just dropped from the spec; it's not useful enough
>>>> in
>>>> its current form. To improve it, the maximum interval between updates
>>>> could
>>>> be reduced to about 40 ms, or perhaps the interval could be made
>>>> settable so
>>>> the author could control how often they want to get the event.
>>>>
>>>> -- Brian
>>>>
>>>> I believe it's a convenience over using setTimeout/setInterval + polling
>>>> to determine whether playback has progressed ie., for rendering your own
>>>> playback progress bar.  I've also seen it been used as a signal to copy
>>>> frames into <canvas> on Firefox, however if timeupdate frequency != fps
>>>> of
>>>> video you either miss frames or paint too much.
>>>>
>>>> I don't think timeupdate today is very useful for doing anything beyond
>>>> a
>>>> progress bar or other simple synchronized animations.
>>>>
>>>
>>> Right. That's what I figured the point is; I just wanted to check to make
>>> sure I wasn't missing something.
>>>
>>> As implemented by Safari and Chrome (which is the minimum rate allowed by
>>> the spec), it's not really useful for that purpose, as 4 updates per
>>> second
>>> makes any sort of synchronization feel jerky and laggy. If it were done
>>> at
>>> the frame rate of the video, or perhaps with a minimum of 25 frames per
>>> second, it would be much more useful. Even at a minimum of 15 frames per
>>> second, you would still be able to get some sorts of useful
>>> synchronization,
>>> though animations synchronized wouldn't feel as smooth as they could.
>>>
>>>  Would something like <video> firing events for every frame rendered help
>>>>
>>>> you out?  This would help also fix the <canvas> over/under painting
>>>> issue
>>>> and improve synchronization.
>>>>
>>>
>>> Yes, this would be considerably better than what is currently specced.
>>>
>>> -- Brian
>>>
>>
>> I'll see if we can do something for WebKit based browsers, because today
>> it
>> literally is hardcoded to 250ms for all ports.
>>
>> http://trac.webkit.org/browser/trunk/WebCore/html/HTMLMediaElement.cpp#L1254
>>
>> Maybe we'll end up firing events based on frame updates for video, and
>> something arbitrary for audio (as it is today).
>>
>> Brian, since Firefox is doing what you proposed -- can you think of any
>> other issues with its current implementation?  What about for audio files?
>>
>> Thanks,
>> Andrew
>
> We've considered firing it for each frame, but there is one problem. If
> people expect that it fires once per frame they will probably write scripts
> which do frame-based animations by moving things n pixels per frame or
> similar. Some animations are just easier to do this way, so there's no
> reason to think that people won't do it. This will break horribly if a
> browser is ever forced to drop a frame, which is going to happen on slower
> machines. In balance this may or may not be a risk worth taking.

When timeupdate was added, the stated goal was actually as a battery
saving feature for for example mobile devices. The idea was that the
implementation could scale back how often it fired the event in order
to save battery.

Now that we have implementation experience, is timeupdate fulfilling
this goal? If not, is it fulfilling any other goals making it worth
keeping?

/ Jonas