[whatwg] Proposal for a MediaSource API that allows sending media data to a HTMLMediaElement

[Mark Watson]

Hi Aaron,

I think it would be good if the API recognized the fact that the media data may becoming from several different original files/streams (e.g. different bitrates) as the player adapts to network or other conditions.

The different files may have different initialization information (Info and Tracks in WebM, Movie Box in mp4 etc.), which could be provided either in the first append call for each stream or with a separate API call. But subsequently you need to know which initialization information is relevant for each appended block. An integer streamId in the append call would be sufficient - the absolute value has no meaning - it would just associate data from the same stream across calls.

The alternatives are:
(a) to require that all streams have the same or compatible initialization information or
(b) to pass the initialization information every time you change streams

(a) has the disadvantage of constraining encoding, and making adding new streams more dependent on the details of how the existing streams were encoded/packaged
(b) is ok, except that it is nice for the player to know "this data is from the same stream you were playing a while ago" - it can re-use some previously established state - rather than every stream change being 'out of the blue'.

A separate comment is that practically we have found it very useful for the media player to know the maximum resolution, frame rate and codec level/profile that will be used, which may be different from the resolution and codec/level/profile of the first stream.

...Mark

On Jul 11, 2011, at 11:42 AM, Aaron Colwell wrote:

> Hi,
> 
> Based on comments in the File API Streaming
> Blobs<http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-January/029973.html>
> thread and
> my Extending HTML 5 video for adaptive
> streaming<http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-June/032277.html>
> thread,
> I decided on taking a stab at writing a MediaSource API
> spec<http://html5-mediasource-api.googlecode.com/svn/trunk/draft-spec/mediasource-draft-spec.html>
> for
> streaming data to a media tag.
> 
> Please take a look at the
> spec<http://html5-mediasource-api.googlecode.com/svn/trunk/draft-spec/mediasource-draft-spec.html>and
> provide some feedback.
> 
> I've tried to start with the simplest thing that would work and hope to
> expand from there if need be. For now, I'm intentionally not trying to solve
> the generic streaming file case because I believe there might be media
> specific requirements around handling seeking especially if we intend to
> support non-packetized media streams like WAV.
> 
> If the feedback is generally positive on this approach, I'll start working
> on patches for WebKit & Chrome so people can experiment with an actual
> implementation.
> 
> Thanks,
> Aaron
>