[whatwg] HTML 5 video tag questions

[Ian Hickson]

On Mon, 15 Jun 2009 jjcogliati-whatwg at yahoo.com wrote:
> 
> Maybe to make this more clear section 4.8.7.1 should add a sentence 
> somewhere like:
> 
>| Authors may provide multiple source elements to provide different 
>| codecs for different user agents.

That section no longer exists.


On Tue, 16 Jun 2009, Sam Dutton wrote:
>
> Could multiple source elements also be used to provide different 
> bit-rate sources (or even alternative versions, e.g. different 
> languages) as well as different codecs, something like the HTTP 
> streaming playlist idea?

On Tue, 16 Jun 2009, Benjamin M. Schwartz wrote:
> 
> If the browser decides that a video is too high resolution and cannot be 
> played on this system, then why can't it fall through to a 
> lower-resolution copy if one exists?

The user agent is allowed to skip past a video file it deems unsuitable, 
but as currently written, the algorithm precludes the user agent skipping 
past a resource based on the presence or absence of a further resource 
that it deems superior.


On Tue, 16 Jun 2009, David Singer wrote:
> 
> [...] there are media queries, and I have posted a couple of times that 
> we should considering allowing source selection based on accessibility 
> needs.  we should probably also make it possible to select on language, 
> it's a common need.

Localisation in general of HTML isn't supported at the moment; it's 
assumed that this will be done server-side. (I don't see why we'd support 
localisation of video and audio media when we don't support localisation 
of text or images yet).


On Fri, 10 Jul 2009, Jonas Sicking wrote:
> >>
> >> Hmm.. is that good? What if you want to use an <object> (to use flash 
> >> or java) or a <img> as fallback?
> >
> > Then you do it with script.
> >
> > The design is based around the assumption that we will eventually find 
> > a common codec so that fallback won't ever be needed in supporting 
> > UAs.
> 
> I agree that the current design makes sense once there is a common codec 
> supported across all browsers. However currently it seems like we might 
> not reach that point until after all major browsers support <video>.
> 
> What would be the downside of displaying the fallback contents if none 
> of the videos can be displayed due to unsupported codecs?

When would you fall back? For example, while parsing, would you fall back 
in between the <video> element being parsed and the first <source> element 
being parsed?

The design you describe is what <object> tried to do, and it proved to be 
extremely problematic in practice -- and that was without another built-in 
fallback mechanism to complicate matters.


On Fri, 10 Jul 2009, Jeff Walden wrote:
> >
> > The design is based around the assumption that we will eventually find 
> > a common codec so that fallback won't ever be needed in supporting 
> > UAs.
> 
> So has anyone ever actually pointed out the elephant in the room here, 
> that we might never do so?  I can't remember if so.  Maybe HTML5: Galaxy 
> Quest (cf. Captain Taggart's line) just isn't going to happen in the 
> foreseeable future.

In practice, as people keep pointing out to me, it's only Apple that 
supports <video> but doesn't support Theora. If people really want to push 
Apple into supporting Theora, the best way to do it would be to just keep 
using it as if it was the common codec, and _not_ provide another fallback 
for <video>-supporting UAs -- then things would work fine it non-<video>- 
supporting UAs like IE (through fallback flash support inside <video>), 
and would work fine in Theora-supporting UAs, but Safari would be left in 
the cold.


> On Fri, 10 Jul 2009, Jeff Walden wrote:
> >
> > (I'm also not convinced that we substantially hurt ourselves by making 
> > fallback content the final <source>.)
> 
> I also think that would make more sense. Right now if a site wants to 
> publish Ogg-only with Cortado as the fallback, they have to use 
> scripting to make it work in Safari.

Not supporting Safari would be an effective way of pressuring Apple.


> And if a site wants to publish H.264-only with Flash or QuickTime as the 
> fallback, they have to use scripting to make it work in Firefox.

We presumably don't want to make the use of uncumbered codecs easier.


> Sites might want to do this even if there were an agreed-upon common 
> codec that simply didn't meet their needs. So a common codec won't 
> completely eliminate this issue. I can't see any advantage to the 
> current design.

Sanity and simplicity in implementation is the main advantage. With 
<object>, fallback proved to be a huge source of bugs for years.


On Sat, 11 Jul 2009, Maciej Stachowiak wrote:
> 
> There is likely an upper bound set by the maximum possible expiration 
> date of any patents applying to any of the viable candidates. It's just 
> that we'd like to reach agreement well before then.

On Sat, 11 Jul 2009, Gregory Maxwell wrote:
> 
> Not really, at some point well before then the argument will shift to a 
> newer clearly superior to H.264 encumbered format. I would expect that 
> H.264 would spend a period of time as a non-consideration by almost 
> everyone since it would be inferior to something newer and yet would 
> still require fees.
> 
> You could counter that H.264 and AAC have reached some magic threshold 
> of adoption or usability that they will not fail to the great 
> hamster-wheel of encumbered codec upgrades, but since I've never seen 
> anyone state what those requirements are I'm doubtful.
> 
> Regardless— It's far from clear that simply waiting 15ish years would 
> resolve the problem, even if anyone found that to be desirable.

On Sat, 11 Jul 2009, Maciej Stachowiak wrote:
> 
> You may be right. On the other hand, Theora and it's immediate precursor 
> have been around for about a decade, and Theora is still considered to 
> be contender quality-wise.

I tend to agree that if Theora is considered a contender now, I don't see 
why we wouldn't think H.264 would be a contender when its patents expire. 
But in any case I doubt we have to wait that long, as I discussed in my 
long e-mail on this a few weeks ago.


> This was a minor side point to what I think is the main point:  <video> 
> should use the fallback content as a last resort if none of the provided 
> sources are suitable. If you are right that in a decade current codecs 
> may all be obsolete, that is supporting evidence for this position. A 
> final <source> fallback to a baseline codec may someday not be a 
> suitable solution, if the baseline is considered obsolete by then and no 
> one wants to use it.

The design of <video> from the ground up is based on the idea that in
browsers that support the element, the API will be used. If we change this 
assumption, then the entire design of the element would have to be 
reconsidered -- in particular, I think we would need to find a way to 
avoid people having to implement the script side twice, in such a 
scenario. We don't get a consistent design if we change the assumptions at 
the slightest provocation. In practice I don't think that the assumption 
is wrong on the long term, though it may be tested on the short term.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'