[whatwg] Scripted querying of <video> capabilities

[Ian Hickson]

On Thu, 7 Aug 2008, Tim Starling wrote:
>
> Would it be possible to add methods or properties to HTMLMediaElement to 
> support scripted determination of client codec capabilities?

The answer, based on replies quoted below from browser vendors, appears to 
be "no", sadly. I agree that it would be a really useful thing to have.


On Thu, 7 Aug 2008, Robert O'Callahan wrote:
>
> That would be nice to have. Unfortunately DirectShow and Quicktime do 
> not seem to expose the ability to enumerate supported codecs, so it 
> might be hard to implement for some browsers.
> 
> As things stand, you can use <source> elements to offer different 
> formats, and you can try to play a stream and use script to detect if it 
> fails to play.

On Thu, 7 Aug 2008, Dave Singer wrote:
> 
> In general, the source fallbacks are also a way to 'probe' this, albeit 
> in a very different way.
> 
> I'm not sure you can always get a definitive answer to the question "if 
> I gave you a file with this (extended) MIME type, could you play it?" 
> and I am fairly sure that asking the implementation to enumerate all the 
> types it could support would be hard.

It is sad that we can't provide an API such as the requested one.

This brings up another point, which is, is the "type" attribute on 
<source> actually useful? Should we remove that and just have browsers 
probe the video subsystem for each resource? We can always add the 
attribute back later if it becomes useful again, but I'd rather not have 
something that isn't used by browsers, since then it'll be used wrong by 
authors, making it useless forever.

Also, should we make .load() asynchronous as far as selecting a media file 
goes? Right now, a media file has to be instantaneously and synchronously 
selected, so the UA can't try each one in turn.


On Fri, 8 Aug 2008, Henri Sivonen wrote:
>
> Does what HTML5 says now match the framework APIs?
> 
> The MIME codecs parameter seems to confuse both WebKit and Minefield, for
> instance:
> http://hsivonen.iki.fi/test/moz/video-selection/source-mp4-ogg.html
> vs.
> http://hsivonen.iki.fi/test/moz/video-selection/source-mp4-ogg-params.html

This is a bad sign; what should we do to fix this?


On Thu, 7 Aug 2008, Tim Starling wrote:
> 
> DirectShow and QuickTime can add those interfaces at a later date. When 
> the backends develop this capability, there should be a standard way to 
> go the next step and expose it to JavaScript. Otherwise every 
> implementor will develop their own interface.

As soon as implementors want to expose this, I would be more than happy to 
specify how it should work. We shouldn't specify it too soon, though, 
because otherwise UAs are wont to implement the API and just guess or lie, 
making the API useless -- and when there's a more reliable option, we'll 
have to invent a new API anyway.


On Wed, 13 Aug 2008, Chris Double wrote:
> 
> I don't believe this to be the case. See my previous message about this. 
> There's one specific instance of it not working as far as I know:
> 
> <video src="foo.ogg">
>  <object>....fallback for Ogg playback using plugin</object>
> </video>
> 
> A browser that supports <video> but not Ogg will not use the fallback 
> <object>. Instead it will just give an error when loading the foo.ogg 
> file. If some way of having this case work is supplied then a media 
> sniffing API is possibly not needed. Tim, can you confirm that?

If you can get the user to download a binary platform-specific NPAPI 
plugin for Ogg support, why not get the user to download a binary 
platform-specific video backend plugin for Ogg support?


On Tue, 12 Aug 2008, James Ide wrote:
> 
> In this case I believe it would be possible to listen to the `error' 
> event and dynamically insert fallback content if the <video> element has 
> an error state of MEDIA_ERR_DECODE. However, this fails when using 
> <source> elements as the resource-selecting algorithm just terminates if 
> all of the media sources are of unsupported types.

It throws an exception in that case too, and the networkState stays EMPTY. 
So this case can be detected. Would you prefer an error condition?


> More complications arise when we consider cases where only a subset of 
> the codecs in the resource are supported and different events are fired 
> and error codes set.
> 
> I agree that there should be some type of fallback mechanism in place, 
> perhaps akin to how <object> deals with unsupported data. Currently 
> <source> fills in some of the gaps by providing fallback support to a 
> degree, but I feel that there should be a more comprehensive solution 
> that encourages authors to use <video>, with or without <source>, but 
> also provides a safety net by allowing currently existing fallback 
> solutions to function as expected. Intuitively, I would expect <video> 
> with an unsupported codec to behave as if the <video> element itself 
> were unsupported, with perhaps a small note on how to acquire the 
> codec(s) required to play the media, if the UA can determine that much 
> and is configured to display such messages.

The idea of <video> is to provide a video rendering surface, even when no 
video is selected, so that there's no drastic flipping back and forth as 
the video is loading or as videos are changed, etc.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'