[whatwg] A standard for adaptive HTTP streaming for media resources

[Silvia Pfeiffer]

On Fri, Aug 20, 2010 at 11:08 AM, Ian Hickson <ian at hixie.ch> wrote:

> On Tue, 25 May 2010, Silvia Pfeiffer wrote:
> >
> > We've in the past talked about how there is a need to adapt the bitrate
> > version of a audio or video resource that is being delivered to a user
> > agent based on the available bandwidth on the network, the available CPU
> > cycles, and possibly other conditions.
> >
> > It has been discussed to do this using @media queries and providing
> > links to alternative versions of a media resources through the <source>
> > element inside it. But this is a very inflexible solution, since the
> > side conditions for choosing a bitrate version may change over time and
> > what is good at the beginning of video playback may not be good 2
> > minutes later (in particular if you're on a mobile device driving
> > through town).
> >
> > Further, we have discussed the need for supporting a live streaming
> > approach such as RTP/RTSP - but RTP/RTSP has its own "non-Web" issues
> > that will make it difficult to make it part of a Web application
> > framework - in particular it request a custom server and won't just work
> > with a HTTP server.
> >
> > In recent times, vendors have indeed started moving away from custom
> > protocols and custom servers and have moved towards more intelligence in
> > the UA and special approaches to streaming over HTTP.
> >
> > Microsoft developed "Smooth Streaming", Apple developed "HTTP Live
> > Streaming" and Adobe recently launched "HTTP Dynamic Streaming". (Also
> > see a comparison at). As these vendors are working on it for MPEG files,
> > so are some people for Ogg. I'm not aware anyone is looking at it for
> > WebM yet.
> >
> > Standards bodies haven't held back either. The 3GPP organisation have
> > defined 3GPP adaptive HTTP Streaming (AHS) in their March 2010 release 9
> > of 3GPP. Now, MPEG has started consolidating approaches for adaptive
> > bitrate streaming over HTTP for MPEG file formats.
> >
> > Adaptive bitrate streaming over HTTP is the correct approach towards
> > solving the double issues of adapting to dynamic bandwidth availability,
> > and of providing a live streaming approach that is reliable.
> >
> > Right now, no standard exists that has been proven to work in a
> > format-independent way. This is particularly an issue for HTML5, where
> > we want at least support for MPEG4, Ogg Theora/Vorbis, and WebM.
> >
> > I know that it is not difficult to solve this issue in a
> > format-independent way, which is why solutions are jumping up
> > everywhere. They are, however, not compatible and create a messy
> > environment where people have to install solutions for multiple
> > different approaches to make sure they are covered for different
> > platforms, different devices, and different formats. It's a clear
> > situation where a new standard is necessary.
> >
> > The standard basically needs to provide three different things:
> > * authoring of content in a specific way
> > * description of the alternative files on the server and their
> > features for the UA to download and use for switching
> > * a means to easily switch mid-way between these alternative files
>
> On Mon, 24 May 2010, Chris Holland wrote:
> >
> > I don't have something decent to offer for the first and last bullets
> > but I'd like to throw-in something for the middle bullet:
> >
> > The http protocol is vastly under-utilized today when it comes to URIs
> > and the various Accept* headers.
> >
> > Today developers might embed an image in a document as chris.png. Web
> > daemons know to find that resource and serve it, in this sense,
> > chris.png is a resource locator.
> >
> > Technically one might reference the image as a resource identifier named
> > "chris". The user's browser may send "image/gif" as the only value of an
> > accept header, signaling the following to the server: "I'm supposed to
> > download an image of chris here, but I only support gif, so don't bother
> > sending me a .png". In a perhaps more useful scenario the user agent may
> > tell the server "don't bother sending me an image, I'm a screen reader,
> > do you have anything my user could listen to?". In this sense, the
> > document's author didn't have to code against or account for every
> > possible "context" out there, the author merely puts a reference to a
> > higher-level representation that should remain forward-compatible with
> > evolving servers and user-agents.
> >
> > By passing a list of accepted mimetypes, the accept http header provides
> > this ability to serve context-aware resources, which starts to feel like
> > a contender for catering to your middle bullet.
> >
> > To that end, new mime-types could be defined to encapsulate media
> > type/bit rate combinations.
> >
> > Or the accept header might remain confined to media types and acceptable
> > bit rate information might get encapsulated into a new header, such as:
> > X-Accept-Bitrate .
> >
> > If you combined the above approach with existing standards for http byte
> > range requests, there may be a mechanism there to cater to your 3rd
> > bullet as well: when network conditions deteriorate, the client could
> > interrupt the current stream and issue a new request "where it left off"
> > to the server. Although this likel wouldn't work because a byte range
> > request would mean nothing on files of two different sizes. For
> > playbacked media, time codes would be needed to define range.
>
> On Tue, 25 May 2010, Silvia Pfeiffer wrote:
> >
> > That's not quite sufficient, actually. You need to know which byte range
> > to retrieve or which file segment. Apple solved it by introducing a m3u8
> > file format, Microsoft by introducing a SMIL-based server manifest file,
> > Adobe by introducing a XML-based Flash Media Manifest file F4M. That
> > kind of complexity canot easily be transferred through HTTP headers.
> >
> > The idea of the manifest file is to provide matching transition points
> > between the different files of different bitrate to segments or byte
> > ranges. This information has to somehow come to the UA (amongst other
> > information as available in typical manifest files). I don't think that
> > can be achieved without a manifest file.
>
> On Fri, 28 May 2010, Jeroen Wijering wrote:
> >
> > Indeed, one such key condition is the current dimensions of the video
> > window. Tracking this condition allows user-agents to:
> >
> > *) Not waste bandwidth, e.g. by pushing a 720p video in a 320x180 video
> > tag.
> >
> > *) Respond to changes in the video display, e.g. when the video is
> > switched to fullscreen playback.
> >
> > Providing the different media options using <source> elements might
> > still work out fine, if there's a clearly defined API that covers all
> > scenarios. A rough example:
> >
> > <video>
> >   <source bitrate="100" height="120" src="video_100.mp4" type="video/mp4;
> codecs='avc1.42E01E, mp4a.40.2'; keyframe-interval='00:02'" width="160">
> >   <source bitrate="500" height="240" src="video_500.mp4" type="video/mp4;
> codecs='avc1.42E01E, mp4a.40.2'; keyframe-interval ='00:02'" width="320">
> >   <source bitrate="900" height="540" src="video_900.mp4" type="video/mp4;
> codecs='avc1.42E01E, mp4a.40.2'; keyframe-interval ='00:02'" width="720">
> > </video>
> >
> > This example would tell the user-agent that the three MP4 files have a
> > keyframe-interval of 2 seconds - which of course raises the issue that
> > fixed keyframe-intervals would be required.
> >
> > The user-agent can subsequently use e.g. the Media Fragments API to
> > request chunks, switching between sources as the conditions change.
>
> It seems to me that we are not lacking in solutions in this space -- it
> would behoove us to try to leverage the existing solutions rather than
> making up new ones. Have the above solutions been tried in browsers?
>
>

Apple's m3u files work in Safari. There is no implementation for WebM or Ogg
yet.

Cheers,
Silvia.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20100820/b688dc32/attachment-0002.htm>