[whatwg] Subtitles, captions, and other tracks augmenting video or audio

[Silvia Pfeiffer]

HI Ian,

On Wed, Apr 21, 2010 at 6:25 AM, Ian Hickson <ian at hixie.ch> wrote:
> On Tue, 20 Apr 2010, Silvia Pfeiffer wrote:
>>
>> I spent some time today filling that page and when I came back to it
>> just now it seems you have moved most of the use cases elsewhere, namely
>> to
>> http://wiki.whatwg.org/wiki/Use_cases_for_API-level_access_to_timed_tracks
>> .
>>
>> IIUC the idea is to focus on the core problem at hand which right now
>> are captions and subtitles. That is fair enough, but I think you might
>> want to reconsider this for lyrics and chapter markers. I'm ok with
>> moving the others to a later stage.
>
> The one page was starting to cover more topics than I could fit in my head
> at once. :-)
>
> The two pages are both things I think we need to support in the first
> version. The first page (timed_tracks) is stuff that needs in-video
> rendering, and that therefore affects whatever formatting model we end up
> using. The stuff on the second page (use_cases_for_API-level_access_to_
> timed_tracks) is stuff that only affects the UI or APIs but that I
> wouldn't expect the browser itself to render natively without script.

Ok, that was obviously a misunderstanding.


>> Firstly about the Lyrics. I think they are just the same as captions and
>> should go back into the first document. In particular since we are
>> talking about captions and subtitles for both the <video> and the
>> <audio> element and this shows some good examples of how lyrics are
>> being displayed as time-aligned text with audio resources. Most of these
>> examples are widgets used on the Web, so I think they are totally
>> relevant.
>
> All the examples are off-video lyrics

Those other examples are not off-video, they have no video and are
audio-related. I wanted some examples for the <audio> element, too.


> except one which I presume everyone
> would agree should require script or at least SMIL+SVG. This suggests to
> me that they are all things we should handle as part of an API, and not as
> a native built-in feature.

Yes, the API would be important there.



>> I'm also confused about the removal of the chapter tracks. These are
>> also time-aligned text files and again look very similar to SRT. Here
>> is an extract of a QTtext chapter track example:
>>
>> {QTtext} {size:16} {font:Lucida Grande} {width:320} {height:42}
>> {language:0} {textColor:65535,65535,65535} {backColor:0,0,0}
>> {doNotAutoScale:off} {timeScale:100} {timeStamps:absolute}
>> {justify:center}
>> [00:00:09.30]
>> Chocolate Rain
>> [00:00:12.00]
>> Some stay dry and others feel the pain
>> [00:00:16.00]
>> Chocolate Rain
>> [00:00:18.00]
>> A baby born will die before the sin
>
> Indeed, chapter markers are clearly something that fits into the same
> model and should be supported.
>
> (Those are some damn short chapters. Are you sure that's not a lyric
> track?)

It was an example that I had available. I am not concerned about the
actual content of it but rather about the general format of chapter
tracks. But yes, the example was indeed a transcoding from song
lyrics.


>> So, while I can understand that you currently want to focus on just
>> solving captions and subtitles, I think it is important to keep other
>> time-aligned text applications that can be solved in the exact same way
>> part of the design to keep an open mind about general time-aligned text
>> use cases.
>
> The split was not intended to indicate a lack of desire to support these
> use cases in whatever format we eventually pick or adapt (or invent, if
> necessary, though I'd really like to avoid that). Sorry for any confusion.


Thanks for clarifying. I was a bit shocked when I got back to the page
and all my work seemed to have been removed. This all makes sense now.
Thanks.

Cheers,
Silvia.