[whatwg] On implementing videos with multiple tracks in HTML5

[Carlos Andrés Solís]

Hello, I've been writing lately in the WHATWG and WebM mail-lists and would
like to hear your opinion on the following idea.

Imagine a hypothetical website that delivers videos in multiple languages.
Like on a DVD, where you can choose your audio and subtitles language. And
also imagine there is the possibility of downloading a file with the video,
along with either the chosen audio/sub tracks, or all of them at once. Right
now, though, there's no way to deliver multiple audio and subtitle streams
on HTML5 and WebM. Since the latter supports only one audio and one video
track, with no embedded subtitles, creating a file with multiple tracks is
impossible, unless using full Matroska instead of WebM - save for the fact
that the standard proposed is WebM and not Matroska.
A solution could be to stream the full Matroska with all tracks embedded.
This, though, would be inefficient, since the user often will select only
one language to view the video, and there's no way yet to stream only the
selected tracks to the user. I have thought of two solutions for this:
* Solution 1: Server-side demuxing. The video with all tracks is stored as a
Matroska file. The server demuxes the file, generates a new one with the
chosen tracks, and streams only the tracks chosen by the user. When the user
chooses to download the full video, the full Matroska file is downloaded
with no overhead. The downside is the server-side demuxing and remuxing;
fortunately most users only need to choose once. Also, there's the problem
of having to download the full file instead of a file with only the tracks
wanted; this could be solved by even more muxing.
* Solution 2: User-side muxing. Each track (video, audio, subtitles) is
stored in standalone files. The server streams the tracks chosen by the
user, and the web browser muxes them back. When the user chooses to download
the video, the generation of the file can be done either server-side or
client-side. This can be very dynamic but will force content providers to
use extra coding inside of the pages.

Any ideas or suggestions?
- Carlos Solís
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20100522/bc794004/attachment.htm>