[whatwg] Video playback quality metric

[Ian Hickson]

On Mon, 9 Feb 2009, Jeremy Doig wrote:
>
> Measuring the rate at which the playback buffer is filling/emptying 
> gives a fair indication of network goodput, but there does not appear to 
> be a way to measure just how well the client is playing the video 
> itself. If I have a wimpy machine behind a fat network connection, you 
> may flood me with HD that I just can't play very well. The cpu or video 
> card may just not be able to render the video well.Exposing a metric 
> (eg: Dropped Frame count, rendered frame rate) would allow sites to 
> dynamically adjust the video which is being sent to a client [eg: switch 
> the url to a differently encoded file] and thereby optimize the playback 
> experience.

One concern is that there are several possible reasons for playback to be 
poor; the hardware could be simply unable to handle it, but it could also 
be that the system is overloaded. For example, multiple videos could be 
playing at once.

As a user, if I see choppy video, I can try to figure out whether my 
system is loaded, and frankly I'd rather do that than have the Web page 
automatically try to downgrade me...


On Tue, 10 Feb 2009, Philip Jägenstedt wrote:
> 
> While I think this kind of thing might be useful, I would be careful 
> about requiring any kind of detailed metrics like dropped frames or 
> effective frame rate to be exposed via DOM, as getting this information 
> reliably over different devices, platforms and media frameworks would be 
> quite difficult. How about an event which the user agent can optionally 
> fire to indicate that it cannot play at the requested rate due to 
> processing/memory constraints (rather than network)? This would (I 
> think) provide the same functionality but put less burden on 
> implementors.
> 
> There is already a placeholder for non-fatal errors in the spec, perhaps 
> this could be included with that in some fashion?

On Tue, 10 Feb 2009, James Graham wrote:
> 
> It seems like, in the short term at least, the "worse is better" 
> solution to this problem is for content providers to provide links to 
> resources at different quality levels, and allow users to choose the 
> most appropriate resource based on their internet connection and their 
> computer rather than having the computer try to work it out for them. 
> Assuming that the majority of users use a relatively small number of 
> sites with the resources to provide multiple-quality versions of their 
> videos and use a small number of computing devices with roughly 
> unchanging network conditions (I imagine this scenario applies to the 
> majority of non-technical), they will quickly learn which versions of 
> the media works best for them on each site. Therefore the burden of this 
> simple approach on end users does not seem to be very high.

On Tue, 10 Feb 2009, Michael A. Puls II wrote:
> 
> Flash has low, medium and high quality that the user can change 
> (although a lot of sites/players seem to rudely disable that option in 
> the menu for some reason). This helps out a lot and can allow a video to 
> play better. I could imagine an "Auto" option too that automatically 
> switched quality as necessary to get decent playback.
> 
> As an event, a site could use it like:
> 
> video.onplaybacktooslow = function() {
>    this.quality = "low";
>    this.setToNativeSize(); // stretched videos use more cpu
> };
> 
> Or, something like that.

I'd be interested in seeing what implementors would find easiest to 
expose, once we have more implementation experience. Just an event along 
the lines of "well I can't keep up with this"? An arbitrary quality 
number where 0 is "this is the worst experience I've ever exposed the 
user to" and 1 is "I'm not even breaking a sweat playing this"? Frames per 
second? Dropped frames per second?

It should be noted that the spec already supports having the _browser_ 
automatically fall back to another stream. The author can include multiple 
streams like this:

   <video>
    <source src="hd.mov">
    <source src="sd.mov">
    <source src="postage-stamp.mov">
   </video>

...and the browser is well within its rights to decide that it can't play 
hd.mov (having downloaded it and examined it) and that it will use sd.mov 
instead. I would be interested in feedback from browser vendors regarding 
whether this is feasible to implement or not -- if it is, can we rely on 
this instead of exposing it to scripts?

I've noted the idea of having an explicit way for scripts to determine 
rendering quality for the v3 media element API. I haven't added anything 
to the spec yet because we're still waiting for the current features to 
get implemented and shipped reliably.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'