[whatwg] Codec mess with <video> and <audio> tags
jjcogliati-whatwg at yahoo.com
jjcogliati-whatwg at yahoo.com
Sun Jun 7 07:31:14 PDT 2009
Here is my update of Dave Singer's 2007 summary of the previous discussion.
Wikia Page where this summary is (feel free to make changes there):
The HTML5 specification contains new elements to allow the embedding
of audio and video, similar to the way that images have historically
been embedded in HTML. In contrast to today's behavior, using
object, where the behavior can vary based on both the type of the
object and the browser, this allows for consistent attributes, DOM
behavior, accessibility management, and so on. It also can handle
the time-based nature of audio and video in a consistent way.
However, interoperability at the markup level does not ensure
interoperability for the user, unless there are commonly supported
formats for the video and audio encodings, and the file format
wrapper. For images there is no mandated format, but the widely
deployed solutions (PNG, JPEG/JFIF, GIF) mean that interoperability
is, in fact, achieved.
The problem is complicated by the IPR situation around audio and
video coding, combined with the W3C patent policy
<http://www.w3.org/Consortium/Patent-Policy-20040205/>. "W3C seeks to
issue Recommendations that can be implemented on a Royalty-Free (RF)
basis." Note that much of the rest of the policy may not apply (as
it concerns the specifications developed at the W3C, not those that
are normatively referenced). However, it's clear that at least
RF-decode is needed.
There are, of course, a number of codecs and formats that can be
considered. A non-exhaustive list might include a variety of
'public' codecs, as well, of course, as proprietary ones:
a) open-source projects: the ogg family (vorbis, theora), and the
BBC Dirac video codec project
b) Current ISO/IEC (MPEG) standard codecs, notably the MPEG-4 family:
AVC (14496-10, jointly published with the ITU as H.264), AAC (part of
c) Older MPEG codecs, notably MPEG-2 layer 3 (aka MP3), MPEG-2 layer
1 and 2 audio, and maybe MPEG-4 part 2 video (14496-2)
d) Current standard codecs from other bodies; SMPTE VC-1, for example
e) Older standards from other bodies: ITU recommendations H.263
(with or without its many enhancement annexes) or even H.261
f) Very old standard codecs, formats, or industry practices; notably
the common format for video from digital still cameras (Motion JPEG
with uncompressed audio in an AVI wrapper)
g) Proprietary codecs, such as Dolby AC-3 audio
There are concerns or issues with all of these:
a) a number of large companies are concerned about the possible
unintended entanglements of the open-source codecs; a 'deep pockets'
company deploying them may be subject to risk here. Google and other companies have announced plans to ship Ogg Vorbis and Theora or are shipping Ogg Vorbis and Theora, so this may not be considered a problem in the future.
b) the current MPEG codecs are currently licensed on a royalty-bearing basis.
c) this is also true of the older MPEG codecs; though their age
suggests examining the lifetime of the patents; MPEG-1 without
MPEG-1 Audio Layer 3 might be royalty free right now. Three problems were
mentioned with using a subset of MPEG-1. First, by using a subset some people might use the full MPEG-1 and not realize that this would not work on browsers only implementing a subset. Second, even clearing MPEG-1 subset as royalty free might be expensive, and third MPEG-1 has a lower quality than other codecs. MPEG-1 352 pixels x 240 lines at 30 frames a second with audio is about 1.9 Mbit/sec.
d) and also SMPTE VC-1
e) H.263 and H.261 both have patent declarations at the ITU.
However, it is probably worth examining the non-assert status of
these, which parts of the specifications they apply to (e.g. H.263
baseline or its enhancement annexes), and the age of the patents and
their potential expiry. H.261's patent declarations either are expired,
only applications, or seem to be a mistake. H.261 however only allows two video sizes, so it is unsuitable. Sun's OMS project is trying to make a video codec based on H.261, which might be worth considering when it is finished.
f) This probably doesn't have significant IPR risk, as its wide
deployment in systems should have exposed any risk by now; but it
hardly represents competitive compression. Motion JPEG 320 x 240 pixels at 30 frames a second with 8 bit PCM audio is about 5 Mbit/sec.
g) Most proprietary codecs are licensed for payment, as that is the
business of the companies who develop them.
==Other licensing concerns==
It's also possible that there are other issues around licensing:
a) variations in licensing depending on filed patents in various geographies
b) restrictions on usage, or fees on usage, other than the fees on
implementation (e.g. usage fees on content sold for remuneration).
It's not entirely clear, also, whether 'implementing' HTML means the
ability to decode and display, or whether encoding is also included.
Including encoding in the equation might significantly complicate
== Potential Action ==
Baring a consensus, the current state of just leaving the video codec unspecified will continue.
--- On Fri, 6/5/09, King InuYasha <ngompa13 at gmail.com> wrote:
> From: King InuYasha <ngompa13 at gmail.com>
> Subject: [whatwg] Codec mess with <video> and <audio> tags
> To: whatwg at lists.whatwg.org
> Date: Friday, June 5, 2009, 6:24 PM
> I have been keeping track of the developments of
> the <video> and <audio> tag for awhile now, and
> I recently noticed that there was a mailing list for
> discussing the spec.
> So, I wanted to put forth my comments about this
> mess regarding codecs I'm seeing sprawled all over the
> mailing list.
More information about the whatwg