[whatwg] Extending HTML 5 video for adaptive streaming

[Robert O'Callahan]

On Sat, Jul 2, 2011 at 2:51 AM, Aaron Colwell <acolwell at google.com> wrote:

> On Thu, Jun 30, 2011 at 4:13 PM, Robert O'Callahan <robert at ocallahan.org>wrote:
>
>> On Fri, Jul 1, 2011 at 4:59 AM, Aaron Colwell <acolwell at google.com>wrote:
>>
>>> I've also been looking at the WebRTC MediaStream API and was wondering if
>>> it
>>> makes more sense to create an object similar to the LocalMediaStream
>>> object.
>>> This has the benefits of unifying how media streams are handled
>>> independent
>>> of whether they come from a camera or a JavaScript based streaming
>>> algorithm. This could also enable sending the media stream through a
>>> Peer-to-peer connection instead of only allowing a camera as a source.
>>> Here
>>> is an example of the type of object I'm talking about.
>>>
>>
>> I think MediaStreams should not be dealing with compressed data except as
>> an optimization when access to decoded data is not required anywhere in the
>> stream pipeline. If you want to do processing of decoded stream data (which
>> I do --- see
>> http://hg.mozilla.org/users/rocallahan_mozilla.com/specs/raw-file/tip/StreamProcessing/StreamProcessing.html),
>> then introducing a decoder inside the stream processing graph creates all
>> sorts of complications.
>>
>> Nice spec. If I understand correctly, your position is that MediaStreams
> should only represent uncompressed media?
>

Sort of. I want the data format (compressed vs uncompressed, etc) to be
hidden from Web authors unless they use APIs like Worker-based processing
that require access to decoded data. What I don't want to have to deal with
is compressed data being injected at arbitrary points in the graph. Right
now the only the place compressed data is injected is at stream sources ---
media elements and getUserMedia.

Rob
-- 
"If we claim to be without sin, we deceive ourselves and the truth is not in
us. If we confess our sins, he is faithful and just and will forgive us our
sins and purify us from all unrighteousness. If we claim we have not sinned,
we make him out to be a liar and his word is not in us." [1 John 1:8-10]