[whatwg] On implementing videos with multiple tracks in HTML5

[Eric Carlson]

On Aug 20, 2010, at 5:53 PM, Silvia Pfeiffer wrote:

> On Sat, Aug 21, 2010 at 10:03 AM, Eric Carlson <eric.carlson at apple.com> wrote:
> 
> On Aug 19, 2010, at 5:23 PM, Silvia Pfeiffer wrote:
> 
> >
> > * Whether to include a multiplexed download functionality in browsers for media resources, where the browser would do the multiplexing of the active media resource with all the active text, audio and video tracks? This could be a context menu functionality, so is probably not so much a need to include in the HTML5 spec, but it's something that browsers can consider to provide. And since muxing isn't quite as difficult a functionality as e.g. decoding video, it could actually be fairly cheap to implement.
> >
> 
>  I don't understand what you mean here, can you explain?
> 
>  
> 
> Sure. What I mean is: you get a video resource through the <video> element and a list of text resources through the <track> element. If I as a user want to take away (i.e. download and share with friends) the video file with the text tracks that I have activated and am currently watching, then I'd want a download feature that allows me to download a single multiplexed video file with all the text tracks inside. Something like a MPEG-4 file with the <track> resources encoded into, say, 3GPP-TT. Or a WebM with WebSRT encoded (if there will be such a mapping). Or a Ogg file with WebSRT - maybe encoded in Kate or natively.
> 
> The simplest implementation of such a functionality is of course where the external text track totally matches the format used in the media resource for encoding text. Assuming WebM will have such a thing as a WebSRT track, the "download" functionality would then consist of multiplexing a new WebM resource by re-using the original WebM resource and including the WebSRT tracks into that. It wouldn't require new video and audio encoding, since it's just a matter of a different multiplexed container. If transcoding to the text format in the native container is required, then it's a bit more complex, but no less so than what we need to do for extracting such data into a Web page for the JavaScript API (it's in fact the inverse of that operation).
> 
> So, I wouldn't think it's a very complex functionality, but it certainly seems to be outside the HTML spec and a browser feature, possibly at first even a browser plugin. Sorry if this is now off topic. :-)
> 
  Even in the hypothetical case where the external text track is already in a format supported by the media container file, saving will require the UA to regenerate the movie's "table of contents" (eg. the 'moov' atom in MPEG-4 or QuickTime files, Meta Seek Information in a WebM file) as well as muxing the text track with the other media data. 

  As you note transcoding is "a bit more complex", especially in the case where a feature in the text track format is not supported by the text format of the native container.

  Further, what should a UA do in the case where the native container format doesn't support any form of text track - eg. mp3, WAVE, etc?

  I disagree that it is not a complex feature, but I do agree that it is outside of the scope of the HTML spec.

eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20100823/6951abc4/attachment-0002.htm>