[whatwg] Codecs for <audio> and <video>

[Gregory Maxwell]

On Tue, Jun 30, 2009 at 10:41 PM, Maciej Stachowiak<mjs at apple.com> wrote:
> I looked into this question with the help of some experts on video decoding
> and embedded hardware. H.264 decoders are available in the form of ASICs,
> and many high volume devices use ASICs rather than general-purpose
> programmable DSPs. In particular this is very common for mobile phones and
> similar devices - it's not common to use the baseband processor for video
> decoding, for instance, as is implied by some material I have seen on this
> topic, or to use other fully general DSPs.

Can you please name some specific mobile products? Surely if it's
common doing so shouldn't be hard.  I don't mean to argue that it
isn't true or intend to debate you on the merits of any examples…  But
this is an area which has been subject to a lot of very vague claims
which add a lot more confusion rather than insight.

Iphone (of all vintages), and Palm Pre have enough CPU power to do
Theora decode for 'mobile resolutions' on the main cpu (no comment on
battery life; but palm pre is OMAP3 and support for that DSP is in the
works as mentioned). I can state this with confidence since the
horribly slow 400mhz arm4t based SOC in the OpenMoko freerunner is
able to (just barely) do it with the completely unoptimized (for arm)
reference libraries (on x86 the assembly optimizations are worth a
30-40% performance boost).

Another example I have is the WDTV, a set top media box.  It's often
described as using a dedicated hardware H.264 decoder, but what it
actually uses is a SMP8634. Which is a hardware decode engine based on
general purpose processors which appears to be format-flexible enough
to decode other formats. (Although the programing details aren't
freely available so its difficult to make concrete claims).

[snip]
> As far as I know, there are currently no commercially available ASICs for
> Ogg Theora video decoding. (Searching Google for Theora ASIC finds some
> claims that technical aspects of the Theora codec would make it hard to
> implement in ASIC form and/or difficult to run on popular DSPs, but I do not
> have the technical expertise to evaluate the merit of these claims.)

There is, in fact, a synthetically VHDL implementation of the Theora
decoder backend available at http://svn.xiph.org/trunk/theora-fpga/

I'm not able to find the claims regarding Theora on DSPs which you are
referring to, care to provide a link?

Not especially relevant but worth mentioning for completeness: Elphel
also distributes a complete i-frame only theora encoder as
syntheziable verilog under the GPL which is used on an FPGA in their
prior generation camera products.
(http://www3.elphel.com/xilinx/publications/xcellonline/xcell_53/xc_video53.htm)

Existence trumps speculation. But I'm still not of the impression that
the hardware forms are not all that relevant.