[whatwg] Give guidance about RFC 4281 codecs parameter

[Henri Sivonen]

The <video> element is designed to accommodate codec-based fallbacks  
based on MIME types. The reasonable codec extension mechanism for  
browsers is using the facilities of the timed media framework of the  
underlying operating system (QuickTime on Mac OS X, DirectShow on  
Windows and gStreamer on GNU systems.) However, these systems, AFAIK,  
dispatch on container filename extension or magic number and codec  
FourCC rather than by MIME parameters. Therefore, special care is  
needed in order to make extensibility and MIME-based fallback work  
with codecs not shipped by the browser vendor.

For example, on Mac OS X, one would want QuickTime itself to handle  
the MP4 container, AAC, the 3 lowest profiles of H.264 and MPEG-4  
Simple Profile while giving MPEG-4 Advanced Simple Profile to 3ivx,  
Windows Media stuff to Flip4Mac and Ogg Theora to Xiph QT Components.  
These would all need to hook into the MIME-based fallback system.

Since giving guidance in the spec itself is more likely to lead to  
vendors and authors understanding codecs parameter than expecting  
them to follow normative references, I think the spec should document  
the correct <source> element type attribute values for the following  
realistic cases. (MIME without parameters and filename extension in  
parentheses.)

  * Theora video and Vorbis audio in Ogg container. (application/ 
ogg; .ogg)
  * Dirac video and Vorbis audio in Ogg container. (application/ 
ogg; .ogg)
  * Theora video and Vorbis audio in Matroska container. (video/x- 
matroska; .mkv)
  * Dirac video and Vorbis audio in Matroska container. (video/x- 
matroska; .mkv)
  * H.264 Baseline profile video and Low-Complexity AAC audio in MP4  
container. (video/mp4; .mp4)
  * H.264 Extended profile video and Low-Complexity AAC audio in MP4  
container. (video/mp4; .mp4)
  * H.264 Main profile video and Low-Complexity AAC audio in MP4  
container. (video/mp4; .mp4)
  * H.264 High profile video and Low-Complexity AAC audio in MP4  
container. (video/mp4; .mp4)
  * MPEG-4 Simple Profile profile video and Low-Complexity AAC audio  
in MP4 container. (video/mp4; .mp4)
  * MPEG-4 Advanced Simple Profile profile video and Low-Complexity  
AAC audio in MP4 container. (video/mp4; .mp4)
  * MPEG-4 Simple Profile profile video and AMR audio in 3GPP  
container. (video/3gpp; .3gp)
  * WMV9 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)
  * WMV8 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)
  * VC-1 video and WMA 10 Pro audio in ASF container. (video/x-ms- 
wmv; .wmv)
  * Real Video 10 video and High-Efficiency AAC audio in Real Media  
container. (application/vnd.rn-realmedia; rm)
  * XviD video and MP3 audio in AVI container. (video/x-msvideo; .avi)
  * Motion-JPEG video and uncompressed PCM audio in AVI container.  
(video/x-msvideo; .avi)
  * MPEG-1 video and MPEG-1 Audio Layer II audio in MPEG-1 program  
stream (video/mpeg; .mpg)

(That's a lot of cases and, yet, none are contrived.)

I tried to figure this out on my own with RFC 4281 and concluded that  
this is not something that authors or even browser implementors are  
likely to get right without an expert-created lookup table. I see  
that at least of the RFC authors reads this mailing list. :-)

-- 
Henri Sivonen
hsivonen at iki.fi
http://hsivonen.iki.fi/