[whatwg] Video feedback

[Silvia Pfeiffer]

On Mon, Jun 20, 2011 at 6:29 PM, Mark Watson <watsonm at netflix.com> wrote:
>
> On Jun 9, 2011, at 4:32 PM, Eric Carlson wrote:
>
>>
>> On Jun 9, 2011, at 12:02 AM, Silvia Pfeiffer wrote:
>>
>>> On Thu, Jun 9, 2011 at 4:34 PM, Simon Pieters <simonp at opera.com> wrote:
>>>> On Thu, 09 Jun 2011 03:47:49 +0200, Silvia Pfeiffer
>>>> <silviapfeiffer1 at gmail.com> wrote:
>>>>
>>>>>> For commercial video providers, the tracks in a live stream change all
>>>>>> the time; this is not limited to audio and video tracks but would include
>>>>>> text tracks as well.
>>>>>
>>>>> OK, all this indicates to me that we probably want a "metadatachanged"
>>>>> event to indicate there has been a change and that JS may need to
>>>>> check some of its assumptions.
>>>>
>>>> We already have durationchange. Duration is metadata. If we want to support
>>>> changes to width/height, and the script is interested in when that happens,
>>>> maybe there should be a dimensionchange event (but what's the use case for
>>>> changing width/height mid-stream?). Does the spec support changes to text
>>>> tracks mid-stream?
>>>
>>> It's not about what the spec supports, but what real-world streams provide.
>>>
>>> I don't think it makes sense to put an event on every single type of
>>> metadata that can change. Most of the time, when you have a stream
>>> change, many variables will change together, so a single event is a
>>> lot less events to raise. It's an event that signifies that the media
>>> framework has reset the video/audio decoding pipeline and loaded a
>>> whole bunch of new stuff. You should imagine it as a concatenation of
>>> different media resources. And yes, they can have different track
>>> constitution and different audio sampling rate (which the audio API
>>> will care about) etc etc.
>>>
>>  In addition, it is possible for a stream to lose or gain an audio track. In this case the dimensions won't change but a script may want to react to the change in audioTracks.
>
> The TrackList object has an onchanged event, which I assumed would fire when any of the information in the TrackList changes (e.g. tracks added or removed). But actually the spec doesn't state when this event fires (as far as I could tell - unless it is implied by some general definition of events called onchanged).
>
> Should there be some clarification here ?

I understood that to relate to a change of cues only, since it is on
the tracklist. I.e. it's an aggregate event from the oncuechange event
of a cue inside the track. I didn't think it would relate to a change
of existence of that track.

Note that the even is attached to the TrackList, not the TrackList[],
so it cannot be raised when a track is added or removed, only when
something inside the TrackList changes.


>>  I agree with Silvia, a more generic "metadata changed" event makes more sense.
>
> Yes, and it should support the case in which text tracks are added/removed too.

Yes, it needs to be an event on the MediaElement.


> Also, as Eric (C) pointed out, one of the things which can change is which of several available versions of the content is being rendered (for adaptive bitrate cases). This doesn't necessarily change any of the metadata currently exposed on the video element, but nevertheless it's information that the application may need. It would be nice to expose some kind of identifier for the currently rendered stream and have an event when this changes. I think that a stream-format-supplied identifier would be sufficient.


I don't know about the adaptive streaming situation. I think that is
more about statistics/metrics rather than about change of resource.
All the alternatives in an adaptive streaming "resource" should
provide the same number of tracks and the same video dimensions, just
at different bitrate/quality, no? Different video dimensions should be
provided through the <source> element and @media attribute, but within
an adaptive stream, the alternatives should be consistent because the
target device won't change. I guess this is a discussion for another
thread... :-)

Cheers,
Silvia.