[whatwg] Video feedback

[Silvia Pfeiffer]

On Wed, Jun 8, 2011 at 6:14 PM, Philip Jägenstedt <philipj at opera.com> wrote:
> On Wed, 08 Jun 2011 02:46:15 +0200, Silvia Pfeiffer
> <silviapfeiffer1 at gmail.com> wrote:
>
>> On Tue, Jun 7, 2011 at 7:04 PM, Philip Jägenstedt <philipj at opera.com>
>> wrote:
>>>
>>> On Sat, 04 Jun 2011 03:39:58 +0200, Silvia Pfeiffer
>>> <silviapfeiffer1 at gmail.com> wrote:
>>>
>>>
>>>> On Fri, Jun 3, 2011 at 9:28 AM, Ian Hickson <ian at hixie.ch> wrote:
>>>>>
>>>>> On Thu, 16 Dec 2010, Silvia Pfeiffer wrote:
>>>>>>
>>>>>> I do not know how technically the change of stream composition works
>>>>>> in
>>>>>> MPEG, but in Ogg we have to end a current stream and start a new one
>>>>>> to
>>>>>> switch compositions. This has been called "sequential multiplexing" or
>>>>>> "chaining". In this case, stream setup information is repeated, which
>>>>>> would probably lead to creating a new steam handler and possibly a new
>>>>>> firing of "loadedmetadata". I am not sure how chaining is implemented
>>>>>> in
>>>>>> browsers.
>>>>>
>>>>> Per spec, chaining isn't currently supported. The closest thing I can
>>>>> find
>>>>> in the spec to this situation is handling a non-fatal error, which
>>>>> causes
>>>>> the unexpected content to be ignored.
>>>>>
>>>>>
>>>>> On Fri, 17 Dec 2010, Eric Winkelman wrote:
>>>>>>
>>>>>> The short answer for changing stream composition is that there is a
>>>>>> Program Map Table (PMT) that is repeated every 100 milliseconds and
>>>>>> describes the content of the stream.  Depending on the programming,
>>>>>> the
>>>>>> stream's composition could change entering/exiting every
>>>>>> advertisement.
>>>>>
>>>>> If this is something that browser vendors want to support, I can
>>>>> specify
>>>>> how to handle it. Anyone?
>>>>
>>>> Icecast streams have chained files, so streaming Ogg to an audio
>>>> element would hit this problem. There is a bug in FF for this:
>>>> https://bugzilla.mozilla.org/show_bug.cgi?id=455165 (and a duplicate
>>>> bug at https://bugzilla.mozilla.org/show_bug.cgi?id=611519). There's
>>>> also a webkit bug for icecast streaming, which is probably related
>>>> https://bugs.webkit.org/show_bug.cgi?id=42750 . I'm not sure how Opera
>>>> is able to deal with icecast streams, but it seems to deal with it.
>>>>
>>>> The thing is: you can implement playback and seeking without any
>>>> further changes to the spec. But then the browser-internal metadata
>>>> states will change depending on the chunk you're on. Should that also
>>>> update the exposed metadata in the API then? Probably yes, because
>>>> otherwise the JS developer may deal with contradictory information.
>>>> Maybe we need a "metadatachange" event for this?
>>>
>>> An Icecast stream is conceptually just one infinite audio stream, even
>>> though at the container level it is several chained Ogg streams. duration
>>> will be Infinity and currentTime will be constantly increasing. This
>>> doesn't
>>> seem to be a case where any spec change is needed. Am I missing
>>> something?
>>
>>
>> That is all correct. However, because it is a sequence of Ogg streams,
>> there are new Ogg headers in the middle. These new Ogg headers will
>> lead to new metadata loaded in the media framework - e.g. because the
>> new Ogg stream is encoded with a different audio sampling rate and a
>> different video width/height etc. So, therefore, the metadata in the
>> media framework changes. However, what the browser reports to the JS
>> developer doesn't change. Or if it does change, the JS developer is
>> not informed of it because it is a single infinite audio (or video)
>> stream. Thus the question whether we need a new "metadatachange" event
>> to expose this to the JS developer. It would then also signify that
>> potentially the number of tracks that are available may have changed
>> and other such information.
>
> Nothing exposed via the current API would change, AFAICT.

Thus, after a change mid-stream to, say,  a smaller video width and
height, would the video.videoWidth and video.videoHeight attributes
represent the width and height of the previous stream or the current
one?


> I agree that if we
> start exposing things like sampling rate or want to support arbitrary
> chained Ogg, then there is a problem.

I think we already have a problem with width and height for chained
Ogg and we cannot stop people from putting chained Ogg into the @src.

I actually took this discussion away from MPEG PTM, which is where
Eric's question came from, because I don't understand how it works
with MPEG. But I can see that it's not just a problem of MPEG, but
also of Ogg (and possibly of WebM which can have multiple Segments).
So, I think we need a generic solution for it.

Cheers,
Silvia.