[whatwg] Video Element Events? - Use Case: Custom Progress Bar

[Ian Hickson]

Summary: I've left the 'timeupdate' event in, firing at a rate under the 
UA's control. I think, as discussed below, that it provides a far better 
solution than relying on authors using JS timers.

On Mon, 17 Nov 2008, Jonas Sicking wrote:
> > 
> > I don't think firing timeupdate on every frame is *that* bad for us, 
> > to be honest.
> 
> It seems like a pretty big waste of resources to have the following 
> script executing 50 times per second:

The idea is that the browsers can reduce the number of times they fire the 
event, so that they _don't_ fire it 50 times per second.


> Sure, we can pull it off, but why do it? As it certainly doesn't seem to 
> achieve the goal of the event which apparently is to reduce the amount 
> of CPU resources used.

I don't see how it could ever be worse than a setTimeout().


On Tue, 18 Nov 2008, Robert O'Callahan wrote:
> 
> It ensures that the UI is always perfectly up to date. We can throttle 
> timeupdate if we get too busy but otherwise, we may as well update the 
> UI as frequently as the video rendering itself.

Indeed.


On Mon, 17 Nov 2008, Jeremy Doig wrote:
>
> i would hope that repainting a progress bar that has not moved 
> 50x/second would not be a normal implementation too. 2x/second seems 
> more realistic (a 300s video with a 600 pixel-wide playbar).

Don't forget that some UIs might have sub-second timers, too.


On Mon, 17 Nov 2008, Jeremy Doig wrote:
>
> no - all I'm saying is that in 48 of the 50x/sec you get called, you can 
> trivially figure out that nothing needs to be done, so return quickly.

On Mon, 17 Nov 2008, Jonas Sicking wrote:
> 
> Are they really going to bother since things will 'work' even if they 
> don't (just suck up a lot of CPU speed)? And are they really going to do 
> something more clever than just update on every 10th event?

If what's happening is that the script is setting an <input type=range> to 
a particular value, or changing the width of a <div>, or whatnot, then the 
amount of actual work done will be mostly limited to just the script. 
Browsers already do optimisations to avoid repaints when nothing visible 
has changed.


On Mon, 17 Nov 2008, Antti Koivisto wrote:
> 
> First of all, I'm not sure I fully understand the problem you are 
> solving here. Could you summarize it?

The timeupdate event is solving the problem of updating the UI of the 
video controller without running script at a rate that the UA can't handle 
or running script when the UA could be spending CPU doing better things 
like seeking.


> On 16.11.2008, at 16:16, Ian Hickson wrote:
> > You don't have to fire the event if nobody is listening for it. 
> > (Browsers likely already have this implementation for mutation events 
> > if nothing else, so this shouldn't be that difficult to implement.)
> 
> If I want to update a time display once per second (which I think is 
> fairly typical) why should I listen to a flood of events 60 times per 
> second? On some devices those events and the Javascript that needs to 
> run to filter them might consume substantial portion of the available 
> CPU capacity.

The idea of 'timeupdate' is that the frequency can be adjusted to match 
the device's capabilities.


> > The idea of the timeupdate event is actually to lighten the load on 
> > the user agent while simultaneously making the display more accurate 
> > than a timer could be.
> 
> As far as I see what you write below does not describe anything that 
> would make the display more accurate or lighten the load. But perhaps 
> you have some additional mechanism in mind that you did not document 
> here?

The browser is free to determine the rate of the event. That is how the 
load is lightened.

The spec calls out the times where timeupdate must be fired in addition to 
the regular firings, e.g. when seeking, when ending, etc. This makes the 
accuracy higher than an author-based timer.



> > With polling, the polling will miss key points, e.g. when the playback 
> > loops, which will result in the UI appearing to lag behind the 
> > playback. It will also cause higher processing cost while there is no 
> > need to send updates, e.g. while seeking or waiting for data, times 
> > where you really don't want extra load.
> 
> The earlier iteration of the spec already fired timeupdate events on all 
> discontinuous changes in time. How would this event flood be an 
> improvement over that?

It relieves authors from having to use a setInterval()-based timer, which 
the browser wouldn't be able to throttle if necessary.


> Any reasonable application will have its polling timer running only when 
> the playback is actually in progress.

Reasonable applications are the minority. We really have to help people do 
the right thing by making it easier than the wrong thing.


> - We cant require time update events to be frame accurate during 
> continuous playback since there is no way to guarantee smooth playback 
> with that requirement. If the events are not accurate they have no 
> benefits over simple polling.

I disagree that there are no benefits.


> - Requiring events on every frame might make some playback optimizations 
> impossible (by requiring constant activation of the web engine thread 
> during playback).

Agreed; the spec doesn't require that.


> - Any media synchronization mechanism should be considered as a separate 
> issue.

Agreed.


On Tue, 18 Nov 2008, Robert O'Callahan wrote:
> 
> That makes a lot more sense than limiting the timeupdate period. Firing 
> timeupdate on discontinuous changes and requiring apps to also use a 
> regular timer to get periodic updates sounds reasonable to me.

It seems like it would be lead to even higher CPU usage.


On Tue, 18 Nov 2008, timeless wrote:
> 
> if that little code causes the entire page to reflow because the web 
> author is stupid, then it's a problem.

Indeed, but if the author does that, he's going to do it with a 
setInterval() timer instead if we don't provide timeupdate; and in that 
case, the browser will not be able to throttle it to reduce the damage.


On Mon, 17 Nov 2008, Paul Chambers wrote:
> 
> I'd like to see the option to have an timeupdate event sent whenever the 
> time/position has changed by more than a time period from the last one 
> sent, that time period threshold being settable from javascript.

I haven't added this; I'm not convinced it's really necessary (except as a 
way to reduce the load further). It would be good to get implementation 
experience on whether this is needed; if it is, we can add it in v2.


> Say I have a progress bar 120 pixels long, and a four minute long video 
> clip is playing. I'd like to be able to set that threshold to 2 seconds, 
> so the timeupdate event would be received each time the progress bar 
> actually needs to be updated (i.e. the position has moved by the 
> equivalent of at least one pixel on the progress bar).

Many controllers also have a time display, which is accurate to at least a 
second, and thus you want to update at least 2 to 4 times a second, so 
that you are never more than a few hundred milliseconds off (once a second 
would lead to very irritating UI if the user tried to seek around, since 
the timer could be as much as a second off at a time).


> Making a distinction between progress of the position through the 
> content, and 'real time' passing (as you'd get with a timer) becomes 
> important when you're playing at rates other than 1.0x forward. It makes 
> sense to me to receive more frequent update events when the content is 
> being played at greater than 1.0x speeds (e.g. fast forward, rewind) and 
> less frequently if in a slow-motion mode, or paused.

True. This also provides a reason for the UA to be in charge -- the UA can 
speed up the events if playback is faster.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'