[whatwg] Codecs for <audio> and <video>

[Gregory Maxwell]

On Tue, Jun 30, 2009 at 5:31 AM, Mikko
Rantalainen<mikko.rantalainen at peda.net> wrote:
[snip]
> Patent licensing issues aside, H.264 would be better baseline codec than
> Theora.

I don't know that I necessarily agree there.

H.264 achieves better efficiency (quality/bitrate) than Theora, but it
does so with greater peak computational complexity and memory
requirements on the decoder.

This isn't really a fault in H.264, it's just a natural consequence of
codec development. Compression efficiency will always be strongly
correlated to computational load.

So, I think there would be an argument today for including something
else as a *baseline* even in the absence of licensing.  (Though the
growth of computational power will probably moot this in the 15-20
years it will take for H.264 to become licensing clear)

Of course there are profiles, but they create a lot of confusion:
People routinely put out files that others have a hard time playing.
Of course, were it not for the licensing Theora wouldn't exist but
there would likely be many other codec alternatives with differing
CPU/bandwidth/quality tradeoffs.

I just wanted to make the point that there are other considerations
which have been ignored simply because the licensing issue is so
overwhelmingly significant, but if it weren't we'd still have many
things to discuss.

The subject does bring me to a minor nit on Ian's decent state of
affairs message:

One of the listed problems is "lack of hardware support". I think Ian
may be unwittingly be falling to a common misconception.

This is a real issue, but it's being misdescribed— it would be more
accurately and clearly stated as "lack of software support on embedded
devices".   Although people keep using the word hardware in this
context I believe that in 999 times out of 1000 they are mistaken in
doing so.

As far as I, or anyone I've spoken to, can tell no one is actually
doing H.264 decode directly in silicon, at least no one with a web
browser. The closest thing to that I see are small microcoded DSPs
which you buy pre-packaged with codec software and ready to go.  I'm
sure someone can correct me if I'm mistaken.

There are a number of reasons for this such as the rapid pace of codec
development vs ASIC design horizons, and the mode switching heavy
nature of modern codecs (H.264 supports many mixtures of block sizes,
for example) simply requiring a lot of chip real-estate if implemented
directly in hardware.

In some cases the DSP is proprietary and not sufficiently open for
other software. But at least in the mobile device market it appears to
be the norm to use an off the shelf general purpose DSP.

This is a very important point because "the hardware doesn't support
it" sounds like an absolute deal breaker while "No one has bothered
porting Theora to the TMS320c64x DSP embedded in the OMAP3 CPU used in
this handheld device" is an obviously surmountable problem.


In the future, when someone says "no hardware support" it would be
helpful to find out if they are talking about actual hardware support
or just something they're calling "hardware" because it's some
mysterious DSP running a vendor-blob that they themselves aren't
personally responsible for programming... or if they are just
regurgitating "common wisdom".


Cheers!