[whatwg] Ressurecting <video> a11y thread [was Re: Video, Closed Captions, and Audio Description Tracks]

[Silvia Pfeiffer]

On Fri, Aug 22, 2008 at 9:09 PM, Dave Singer <singer at apple.com> wrote:
> At 12:59  +0200 22/08/08, Aaron Leventhal wrote:
>>
>> Has anyone put any further thought on what to do about captions for Ogg?
>>
>> We've started to throw some thoughts together here:
>> https://wiki.mozilla.org/Accessibility/Captioning_Work_Plan
>>
>> We could use some help from individuals who understand the area of video
>> and captions. The problem of deciding what to do for captions in <video> or
>> specifically for Ogg does not appear to be simple. I'd appreciate it if
>> someone could prove us wrong.
>>
>> - Aaron
>
>
> I don't have ogg-specific opinions, but here are some ideas.
>
> a) burn in the captions, and allow the HTML level to select between sources
> with and without burned-in captions.  This is not ideal, but burned-in
> captions alas, do occur.
>
> b) have a file format with streams that can be enabled/disabled. Streams
> might be something like 3G timed text, or a more direct translation of e.g.
> 508, or a more direct stream of e.g. w3C DFXP (Timed text authoring format).
>
> c) use a more heavy-weight generic layer such as SVG, MPEG LASeR, or 3G DIMS
>
> d) use a SMIL file pointing at the timed text in some text format, in a
> 'par' with the video

And not to forget

e) using CMML or Kate which are timed text/karaoke codecs developed at Xiph. :-)


Aaron,

I agree that captions are a complex topic. And I can see that you have
put lots of thoughts into the workplan, explaining well the current
situation of codecs, encapsulation formats, and multimedia frameworks.
I'd be keen to help you solve the issues.

BTW: here is some more information on non-Ogg formats that you may
want to use to complete your discussion in the Background section of
that wiki page:
http://www.jeroenwijering.com/?item=Making_Video_Accessible .


When I read your wiki page, I can see that you have done a good
analysis of existing formats and their support in encapsulation
formats. There is less analysis on what is supported in which
applications. I think we have to take a larger view on the whole
process of caption creation, embedding, distribution and comsumption
before we are able to make the best decision on what to choose for
Ogg.

What software would be used to create the captions? e.g. for CMML we
have keystroke http://freshmeat.net/projects/keystroke/ to create the
CMML file

What software would be used to associate the caption file with a video
or audio file (server-side or client-side)? e.g. for CMML we have
anxenc to encapsulate CMML into Ogg, mod_annodex to dynamically create
Ogg with encapsulated CMML on an Apache server, or no association at
all and a reliance on the client to synchronise the captions with the
video file, which may be the easiest but not always the desired
solution.

What playback software exists to display the captions with the video?
e.g. for CMML we have an implementation of a firefox browser plugin
that allows the extraction of CMML annotations for a Ogg Theora video
through a javascript video API
(http://svn.annodex.net/browser_plugin/trunk/), or we have
applications that deal with the annotations/captions separately (e.g.
in the browser http://metavid.ucsc.edu/wiki/index.php/Stream:Screen_cast_march_08;
click on the caterpillar at the top right to see the CMML
representation; on the desktop xine or mplayer)


Further, taking a step back from just looking at how to provide
caption support for Ogg, we have a large challenge here to ask about
what subtitle standard should HTML5 video/audio use? considering there
is demand for applications such as this:
http://www.alp.org.au/labortv/8c6RxZzG6O or this
http://metavid.ucsc.edu/wiki/index.php/Stream:Screen_cast_march_08

But maybe this discussion is too early for WHATWG and we should first
strive to just solve it for Ogg. So, I think your approach is probably
right, though WHATWG may not be the right forum to bring it up in at
this stage. I'd be happy to be proven wrong though.

Regards,
Silvia.