[whatwg] Comments about the track element
cyril.concolato at telecom-paristech.fr
Wed Jul 25 06:45:58 PDT 2012
Le 7/25/2012 4:24 AM, Silvia Pfeiffer a écrit :
> Expanding a bit on what Anne said...
> On Tue, Jul 24, 2012 at 11:18 PM, Cyril Concolato
> <cyril.concolato at telecom-paristech.fr> wrote:
>> Dear WhatWG,
>> During the ongoing SVG F2F meeting, the SVG WG discussed the use case of
>> displaying SVG graphics on top of a video, in a synchronous manner.
>> The SVG WG believes that for such use case, it is necessary to indicate to
>> the browser that the SVG and video content should stay synchronized (no
>> matter what happens to the video playback), and to let the browser handle
>> the synchronization internally. The SVG WG resolved to include such
>> indication as part of the Web Animation specification, for instance using
>> the HTML mediagroup attribute or the MediaController API.
>> However, the SVG WG thinks it would also be interesting to leverage the
>> native UI controls of the video element to select (or deactivate) the
>> display of the SVG content on top of the video, in a similar manner to a
>> subtitle track. Obviously, the HTML 5 track element would be a suitable
>> option for that. However, currently it only allows text tracks. So, the SVG
>> WG would like HTML to allow the track element's URL to identify an SVG
>> resource, and in that case the track kind would be 'graphics'. There would
>> be a need to define how the graphics are displayed on top of the video,
> Right now it is fully defined how data in a TextTrack (of the defined
> kinds) is displayed on top of the video. As this is as yet unclear for
> SVG resources,
I wouldn't say it's unclear, I'd say it needs to be specified ;) meaning
that it probably doesn't require much specification. I was thinking that
we could use the CSS box of the video element to position the SVG, as if
the SVG was put in a div.
> I would suggest using the @metadata tack kind for now
> and providing the SVG as markup in a TextTrackCue (either from WebVTT
I've tried this option but I'm facing several problems (Tested with
Chrome Version 22.0.1216.0 canary).
The first problem is how to embed SVG in a cue? Should the '<', '>' and
other characters be escaped or not? According to Anne's validator, they
should be. But if I use them, then the parsing of the escaped string
returns 'empty document'
However, if I don't escape them, the parsing doesn't fail and returns an
In any case, I think embedding the SVG in WEBVTT does not really make
An other problem is in terms of design. SVG has a timing model (similar
to TTML), WebVTT another. For instance, SVG can express things like
repetitions of animations that WebVTT cannot. Are you saying that TTML
should be carried in a WebVTT file?
Similarly, in terms of design, embedding SVG in cues requires repeating
a lot of SVG content at each cue (see
this approach requires parsing an entire document at each cue. You could
probably envisage overlapping cues but that would require a lot of
overhead. Leveraging the progressive loading of SVG cannot be done this
In general, I think it would make sense to leverage the browsers'
support for SVG and not stack different technologies.
Another problem is that I don't know if it's possible to display the SVG
content in a layer between the video and the UI controls. Currently, I
display the SVG on top of the video element, therefore the UI controls
are not accessible for clicks. Having to embed my own UI controls for
that is a bit of a pain. And, semantically, when reading the spec,
'metadata' tracks say " Not displayed by the user agent. " so I think
this might be a bit confusing for users/authors.
The third problem is performance-wise. In my example, the blue line (in
SVG), when synchronized with the video, should be aligned with the
moving (white-gray) edge of the pie. As you can see, this is not the
case. Only 4-5 cuechange events seems to be processed properly. I
noticed the same problem with 'timeupdate' events. Also, I've noticed
that even though my WebVTT file is designed to have only one active cue
at a time, for some cuechange events, there are 2. This might be an
implementation issue but this might be a problem of reentrant code (the
cuechange callback being called while it's not finished), but in
engine to do that, for the processing overhead.
Can you elaborate on this one? However, I suspect it'll have the same
> timing/synchronization is provided by the TextTrackCue, which seems to
> be all you are asking for right now. The rendering is then done by
> getCueAsHTML() to simply hand the SVG to HTML for rendering.
> @metadata tracks can be part of the native video controls and the
> @label on the text track would provide a description of what it is,
> e.g. "Graphic overlays".
>> instance reusing the viewport/viewbox negotiation phase. There would also be
>> a need to make a more generic Track API or to replace the TextTrack API by
>> the SVG API when the track is of kind 'graphics'.
> I don't understand this requirement. What API needs are there aside
> from the synchronization? Trying to replicate SVG APIs through the
> TextTrack API seems like a repetition of the API and thus fragile.
Sorry for the confusion here. I didn't mean to replicate the SVG APIs
here but I just meant that the TextTrack API is very specific to 'pure'
text tracks (and even to WebVTT text tracks). You might want to expose
the SVG API when SVG content is used for the overlay to control it.
Maître de Conférences/Associate Professor
Groupe Multimedia/Multimedia Group
46 rue Barrault
75 013 Paris, France
More information about the whatwg