[whatwg] TimedTracks and MPEG transport streams - tracks can change dynamically

[Eric Winkelman]

On Wednesday, December 15, 2010 6:33 AM Silvia Pfeiffer <silviapfeiffer1 at gmail.com> wrote:

> On Wed, Dec 15, 2010 at 3:31 AM, Philip Jägenstedt <philipj at opera.com>
> wrote:
> > On Wed, 24 Nov 2010 17:11:02 +0100, Eric Winkelman
> > <E.Winkelman at cablelabs.com> wrote:
> >
> >> I'm investigating how TimedTracks can be used for in-band-data-tracks
> >> within MPEG transport streams (used for cable television).
> >>
> >> In this format, the number and types of in-band-data-tracks can change
> >> over time.  So, for example, when the programming switches from a
> football
> >> game to a movie, an alternate language track may appear that wasn't
> there
> >> before.  Later, when the programming changes again, that language track
> may
> >> be removed.
> >>
> >> It's not clear to me how these changes are exposed by the proposed
> Media
> >> Element events.
> >
> > The thinking is that you switch between different streams by setting the
> > src="" attribute to point to another stream, in which case you'll get an
> > emptied event along with another bunch of events. If you have a single
> > source where audio/video/text streams appear and disappear, there's not
> > really any way to handle it.
> 
> I believe "in-band" was the keyword in the request.
 
That is correct.  I'm working on synchronizing JavaScript applications with video content using in-band EISS signals.

> I do not know how technically the change of stream composition works
> in MPEG, but in Ogg we have to end a current stream and start a new
> one to switch compositions. This has been called "sequential
> multiplexing" or "chaining". In this case, stream setup information is
> repeated, which would probably lead to creating a new steam handler
> and possibly a new firing of "loadedmetadata". I am not sure how
> chaining is implemented in browsers.

The short answer for changing stream composition is that there is a Program Map Table (PMT) that is repeated every 100 milliseconds and describes the content of the stream.  Depending on the programming, the stream's composition could change entering/exiting every advertisement. 

> In a live situation you basically don't have a choice and you have to
> use chaining when your input streams change. For a canned situation
> you can set up all the required tracks from the start and just have
> them send data later or stop sending earlier. In the canned document
> situation therefore the number of tracks does not change. It's just
> that data may not be available for all the time.
> 
> >> The "loadedmetadata" event is used to indicate that the TimedTracks are
> >> ready, but it appears that it is only fired before playback begins.  Is
> >> this event fired again whenever a new track is discovered?  Is there 
> >> another event that is intended for this situation?
>
> There is an "onload" event on the TimedTrack to indicate that a track
> is finished loading. But there is no event to indicate that a new
> track is available.

Does the "onload" event fire on in-band tracks?  My current implementation only uses the "oncuechange" event, as you can't be sure the in-band track is ever completely loaded.
 
> >> Similarly, is there an event that indicates when a track has been 
> >> removed?
> >>  Or is this also handled by the "loadedmetadata" event somehow?
> 
> No, there is no "onunload" event on the TimedTracks.

It's possible that you wouldn't need to unload a TimedTrack, if empty tracks were "recycled" for new tracks as needed.  I'm looking at this approach now.

> > No, the loadedmetadata event is only fired once per resource, it's not
> > the event you're looking for.
> >
> > As for actual solutions, I think that having loadedmetadata fire again
> > if the number or type of streams change would make some sense.

This is the way I'm currently doing this, but we're evaluating if this will cause other problems.

> > --
> > Philip Jägenstedt
> > Core Developer
> > Opera Software
> >
> 
> Cheers,
> Silvia.

Thanks!

Eric Winkelman
---
e.winkelman  cablelabs.com
Cable Television Laboratories, Inc.