[whatwg] Scripted querying of <video> capabilities

[Philip Jägenstedt]

(sorry, my mail was work in progress when I accidentally hit send)

I'm also a bit concerned about how to interpret the yes, no and maybe
return values. The truthful answer is going to be "maybe" for all but
the obviously unsupporter (application/x-ms-dos-executable) and the more
trivial formats (audio/wav).

When asking about application/ogg, this could mean 2 things:

1. "can I demux Ogg streams?"
2. "can I demux Ogg streams and decode unknown codecs?"

The answer to question 1 is reasonably simple to answer in most cases,
even if the underlying media frame doesn't support querying for mime
types a mapping between mime type and supported decoders isn't impossible.

The answer to question 2, on the other hand, is always "no" or "maybe".

Unless the codecs parameter is to be made mandatory I think that spec
should explicitly make it such that the question asked is 1. In either
case we will end up there because 2 is not a meaningful question and
user agents will make untruthful answers in attempts to stay compatible
with unknown and future content (which might be supported by installing
new codecs in the media framework without upgrading the browser).

Now, if the codec parameter is used then the user agent may answer yes
and no in a way that actually makes some sense.

I also think that this should be explicitly related to the type
attribute of the source element. One should be able to use canPlayType
to predict with 100% accuracy which source the user agent will use when
it encounters.

<video>
<source type="application/ogg"/>
<source type="application/ogg codecs=dirac,flac"/>
</video>

If this isn't specified then one can expect lots of web app authors
being forced to write scripts to work around the different behaviour of
canPlayType and the "pick a source" algorithm in different browsers.

Philip

On Thu, 2008-11-13 at 10:52 -0800, Jeremy Doig wrote:
> did this thread go anywhere ?
> i'm concerned about the "maybe" case - looks way too much like:
> http://en.wikipedia.org/wiki/DShow#Codec_hell
> 
> 
> also - when you probe for mime type, do you mean the entire "type"
> parameter (including the codecs string) ? for example, there are too
> many cases where just passing "video/mp4" would be insufficient.
> (fragmented index support ? base/main/high profile ? paff ? cabac ?)
> <source src="video.mp4" type="video/mp4; codecs=&quot;avc1.42E01E, mp4a.40.2&quot;">
> 
> On Wed, Oct 15, 2008 at 11:14 PM, Maciej Stachowiak <mjs at apple.com>
> wrote:
>         
>         On Oct 15, 2008, at 1:44 AM, Ian Hickson wrote:
>         
>                 
>                 On Tue, 14 Oct 2008, Robert O'Callahan wrote:
>                         On Tue, Oct 14, 2008 at 12:13 PM, Maciej
>                         Stachowiak <mjs at apple.com> wrote:
>                         
>                                 While the underlying media frameworks
>                                 can't necessarily answer, "if I
>                                 give you a file with this MIME type,
>                                 can you play it?", they can at
>                                 least give a yes/no/maybe answer,
>                                 which can still be quite helpful,
>                                 since the UA will know it does not
>                                 need to check some media streams at
>                                 all.
>                         
>                         I agree. If the API lets us answer "maybe",
>                         there is not much need or
>                         temptation to lie, and we can still return
>                         information that could be
>                         useful to scripts.
>                 
>                 I have added window.navigator.canPlayType(mimeType).
>                 It returns 1, 0, or
>                 -1 to represent positive, neutral, and negative
>                 responses.
>         
>         
>         This API would be tempting to treat as a boolean but would of
>         course do completely the wrong thing. I think it would be
>         better to either ensure that the positive and neutral
>         responses are both values that JS would treat as true (for
>         instance make the values true, "maybe" and false), or else
>         make all of the return values something self-descriptive and
>         symbolic (for instance the strings "yes", "maybe" and "no"). I
>         think 1, 0, -1 are neither clear nor likely to be in any way
>         beneficial for perforamnce.
>         
>         Regards,
>         Maciej
>         
> 
> 
-- 
Philip Jägenstedt
Opera Software