[whatwg] Give guidance about RFC 4281 codecs parameter

[Ian Hickson]

On Sun, 8 Apr 2007, Henri Sivonen wrote:
> 
> Since giving guidance in the spec itself is more likely to lead to 
> vendors and authors understanding codecs parameter than expecting them 
> to follow normative references, I think the spec should document the 
> correct <source> element type attribute values for the following 
> realistic cases. (MIME without parameters and filename extension in 
> parentheses.)

I've added examples for many of these in the spec. I hope they're right.

   http://www.whatwg.org/specs/web-apps/current-work/#type8

I couldn't add examples that I didn't know, though. If anyone from 
Microsoft is listening, it would be useful to hear your input on WMV, ASF, 
AVI, etc, codecs="" values (see below for more details).


On Mon, 9 Apr 2007, Dave Singer wrote:
> >
> >  * H.264 Baseline profile video and Low-Complexity AAC audio in MP4
> > container. (video/mp4; .mp4)
> 
> The AAC is covered by the RFC;  the example is there - mp4a.40.2.
> 
> H.264 was recently covered by 3GPP.  See 26.244 V7.1.0 section A.2.2,
> available from www.3gpp.org.
> 
> When the first element of a value is 'avc1', indicating H.264 (AVC) video
> [29], the second element is the hexadecimal representation of the following
> three bytes in the sequence parameter set NAL unit specified in [29]: 1)
> profile_idc, 2) a byte composed of the values of constraint_set0_flag,
> constraint_set1_flag, constraint_set2_flag, constraint_set3_flag, and
> reserved_zero_4bits in bit-significance order, starting from the most
> significant bit, and 3) level_idc. Note that reserved_zero_4bits is required
> to be equal to 0 in [29], but other values for it may be specified in the
> future by ITU-T or ISO/IEC.
> 
> You don't give me a level number, so I put xx for those hex digits below.
> 
> Assuming we're simple baseline, and also main and extended compatible
> avc1.42E0xx

Assuming level 3, I assume that the correct example here would be:

  H.264 Simple baseline profile video (main and extended video compatible) 
  level 3 and Low-Complexity AAC audio in MP4 container

    type="video/mp4; codecs="avc1.42E030, mp4a.40.2"


> >  * H.264 Extended profile video and Low-Complexity AAC audio in MP4 
> > container. (video/mp4; .mp4
> 
> Extended profile has the value (decimal) 88, and typically Extended 
> profile streams would be marked as Baseline compatible, at least. Here 
> is an example for this AVC
> 
> avc1.58A0xx

Again assuming level 3:

   H.264 Extended profile video (baseline-compatible) level 3 and 
   Low-Complexity AAC audio in MP4 container

     type="video/mp4; codecs="avc1.58A030, mp4a.40.2""


> >  * H.264 Main profile video and Low-Complexity AAC audio in MP4 container.
> > (video/mp4; .mp4)
> 
> Main profile is decimal 77, so something like
> avc1.4D40xx

"something like" seems a bit vague... what do I tell authors to put? Is 
this correct?:

   H.264 Main profile video level 3 and Low-Complexity AAC audio in MP4 
   container

     type="video/mp4; codecs="avc1.4D4030, mp4a.40.2""


> >  * H.264 High profile video and Low-Complexity AAC audio in MP4 container.
> > (video/mp4; .mp4)
> 
> There are a number of high profiles, confusingly, though there is one called
> 'high', with value decimal 100.
>
> avc1.6400xx
> 
> if it's not compatible with main, baseline, or extended profiles.

So is this the right thing to put in the spec?

   H.264 "High" profile video (incompatible with main, baseline, or 
   extended profiles) level 3 and Low-Complexity AAC audio in MP4  
   container

     type="video/mp4; codecs="avc1.640030, mp4a.40.2""



> >  * MPEG-4 Simple Profile profile video and Low-Complexity AAC audio in MP4
> > container. (video/mp4; .mp4)
> 
> Covered by the RFC:
> 
> For example, MPEG-4 Visual Simple Profile Level 0 has the value 9,
>    so a complete string for MPEG-4 Visual Simple Profile Level 0 would
>    be "mp4v.20.9".
> 
> Though when checking the next answer, I read in the spec. that we may have a
> typo here, it might be 8.  Ooops, if so.
> 
> Simple Profile/Level 0	00001000
> Reserved			00001001 - 00001111

So:

   MPEG-4 Visual Simple Profile Level 0 video and Low-Complexity AAC 
   audio in MP4 container

    type="video/mp4; codecs="mp4v.20.8, mp4a.40.2""

...?


> >  * MPEG-4 Advanced Simple Profile profile video and Low-Complexity AAC audio
> > in MP4 container. (video/mp4; .mp4)
> 
> Advanced Simple  Profile/Level 0	11110000
> 
> so, mp4v.20.240

Advanced Simple? Really? How do you guys come up with these names!

Is this right?:

   MPEG-4 Advanced Simple Profile Level 0 video and Low-Complexity AAC 
   audio in MP4 container

     type="video/mp4; codecs="mp4v.20.240, mp4a.40.2""


> >  * MPEG-4 Simple Profile profile video and AMR audio in 3GPP container.
> > (video/3gpp; .3gp)
> 
> Video we've covered.  AMR is in 26.244,
> 
> samr

So to confirm:

   MPEG-4 Visual Simple Profile Level 0 video and AMR audio in 3GPP 
   container

     type="video/3gpp; codecs="mp4v.20.8, samr""


> >  * WMV9 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)
> >  * WMV8 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)
> 
> These would be up to Microsoft to define, I assume.  I am not aware of a
> definition.
>
> >  * VC-1 video and WMA 10 Pro audio in ASF container. (video/x-ms-wmv; .wmv)
> 
> VC-1 is standardized by SMPTE, but again this container format is Microsoft's.
> 
> >  * Real Video 10 video and High-Efficiency AAC audio in Real Media
> > container. (application/vnd.rn-realmedia; rm)
> 
> We'd have to ask Real, but I don't think it is defined.
> 
> >  * XviD video and MP3 audio in AVI container. (video/x-msvideo; .avi)
> >  * Motion-JPEG video and uncompressed PCM audio in AVI container.
> > (video/x-msvideo; .avi)
> 
> AVI I *think* is owned by Microsoft, and I *think* they deprecate it; it's up
> to the owner to define.  Again, I am not aware of a definition, but the 4CC
> style from MP4 might be appropriate.

Do we think these companies come up with appropriate definitions any time 
soon? It might be in our interests to do this for them.


> >  * MPEG-1 video and MPEG-1 Audio Layer II audio in MPEG-1 program stream
> > (video/mpeg; .mpg)
> 
> MPEG has not defined the codecs parameter for these 'file' (stream) formats,
> as far as I know.

So what should someone use?



It seems RFC4281 is actually not as useful as I'd hoped, if it doesn't 
define the majority of cases -- it seems to only handle recent MPEG 
standards.


On Tue, 10 Apr 2007, Silvia Pfeiffer wrote:
> 
> Xiph is using multiple MIME types to deal with this situation.
> 
> * application/ogg (http://www.rfc-archive.org/getrfc.php?rfc=3534)
> provides for the container format, basically with any codecs included:
> this could e.g. be Theora/Vorbis inside Ogg or Dirac/Vorbis inside
> Ogg, but is not restricted to this.
> 
> Xiph has defined a Skeleton track for Ogg files through which the
> MIME type of the contained codecs is specified at a known location in
> the header of the Ogg file. Thus, a Ogg Theora/Vorbis File with a
> skeleton track will have a MIME type of application/ogg and contain
> two tracks with video/x-theora and audio/x-vorbis as the MIME types.
> Xiph encourages everyone to use the Skeleton track in all future
> created Ogg files. Skeleton is built into a standard Ogg container
> format (as defined in RFC 3533
> http://www.rfc-archive.org/getrfc.php?rfc=3533) and thus Ogg files
> with Skeleton are compatible with Ogg files without Skeleton.
> 
> * audio/x-vorbis is used to specify the vorbis codec. This is of
> particular importance in rtp/rtsp streaming applications.
> 
> * video/x-theora is particularly used to specify the theora codec.
> 
> 
> The community has also been seen to use these specifications:
> # audio/x-vorbis+ogg  for Ogg Vorbis files
> # video/x-theora+ogg for Ogg Theora/Vorbis files
> though Xiph is not supportive of these.
> 
> 
> Recent discussion at Xiph around http://www.ietf.org/rfc/rfc4281
> suggests the use of the following parameters:
> 
> # application/ogg; codecs="theora, vorbis"   for Ogg Theora/Vorbis files
> # application/ogg; codecs="theora, speex"   for Ogg Theora/Speex files
> # application/ogg; codecs="vorbis"                 for Ogg Vorbis files
> 
> and also use the disposition parameter:
> 
> # application/ogg; disposition=moving-image; codecs="theora, vorbis"
> # application/ogg; disposition=sound; codecs="speex"
> 
> Skeleton and the use of these MIME parameters should make things clear
> for the application developers.

I've put the above codecs="" suggestions in the spec.

What about Dirac video with Vorbis audio?


On Wed, 11 Apr 2007, Thomas Davies wrote:
> 
> We haven't finalised profiles and levels yet, but there will probably be 
> two profiles, a Main profile covering everything, and a Professional 
> profile covering a restricted set for professional applications, such as 
> very low delay coding or intra-only coding. I can't see this profile 
> being deployed over the internet - it will be for very high bit rate 
> hardware apps, but if it was, a Main-capable decoder could decode it.

Do you have any information on what the RFC4281 values for Dirac will be?

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'