[whatwg] Comments about the track element

[Silvia Pfeiffer]

Expanding a bit on what Anne said...

On Tue, Jul 24, 2012 at 11:18 PM, Cyril Concolato
<cyril.concolato at telecom-paristech.fr> wrote:
> Dear WhatWG,
>
> During the ongoing SVG F2F meeting, the SVG WG discussed the use case of
> displaying SVG graphics on top of a video, in a synchronous manner.
>
> The SVG WG believes that for such use case, it is necessary to indicate to
> the browser that the SVG and video content should stay synchronized (no
> matter what happens to the video playback), and to let the browser handle
> the synchronization internally. The SVG WG resolved to include such
> indication as part of the Web Animation specification, for instance using
> the HTML mediagroup attribute or the MediaController API.
>
> However, the SVG WG thinks it would also be interesting to leverage the
> native UI controls of the video element to select (or deactivate) the
> display of the SVG content on top of the video, in a similar manner to a
> subtitle track. Obviously, the HTML 5 track element would be a suitable
> option for that. However, currently it only allows text tracks. So, the SVG
> WG would like HTML to allow the track element's URL to identify an SVG
> resource, and in that case the track kind would be 'graphics'. There would
> be a need to define how the graphics are displayed on top of the video,

Right now it is fully defined how data in a TextTrack (of the defined
kinds) is displayed on top of the video. As this is as yet unclear for
SVG resources, I would suggest using the @metadata tack kind for now
and providing the SVG as markup in a TextTrackCue (either from WebVTT
cues or from JavaScript calls to addTextTrack()). The
timing/synchronization is provided by the TextTrackCue, which seems to
be all you are asking for right now. The rendering is then done by
JavaScript, which seems the most flexible approach. You can even use
getCueAsHTML() to simply hand the SVG to HTML for rendering.

@metadata tracks can be part of the native video controls and the
@label on the text track would provide a description of what it is,
e.g. "Graphic overlays".

> for
> instance reusing the viewport/viewbox negotiation phase. There would also be
> a need to make a more generic Track API or to replace the TextTrack API by
> the SVG API when the track is of kind 'graphics'.

I don't understand this requirement. What API needs are there aside
from the synchronization? Trying to replicate SVG APIs through the
TextTrack API seems like a repetition of the API and thus fragile.


Regards,
Silvia.