[whatwg] Video, Closed Captions, and Audio Description Tracks

Mon Oct 8 12:05:33 PDT 2007

At 9:45  +1200 8/10/07, Chris Double wrote:
>The video element  description states that Theora, Voribis and Ogg
>container should be supported. How should closed captions and audio
>description tracks for accessibility be supported using video and
>these formats?
>
>I was pointed to a page outlining some previous discussion on the issue:
>
>http://wiki.whatwg.org/wiki/Video_accessibility
>
>Is there a way of identifying which track is the closed caption track,
>which is the alternate audio track, etc? How are other implementors of
>the video element handling this issue?
>
>Is CMML for the closed captions viable? Or a speex track for the
>alternate audio? Or using Ogg Skeleton in some way to get information
>about the other tracks?

There was also a thread I started in June, which I can't find on the 
archives;  my initial email is below.

We suggested two ways to achieve captioning (a) by selection of 
element, at the HTML level ('if you need captions, use this 
resource') and (b) styling of elements at the HTML level ('this video 
can be asked to display captions').

Choice (a) means that it is possible, for example, to prepare 
alternative versions with 'burned in' accessibility (e.g. captions), 
and then explicit support for them is not needed in the format.

Choice (b) is more economical in media resources, and recognizes that 
'true captioning' is sometimes better (e.g. it might be delivered out 
on analog video as line 21 data).

The previous thread faded away, but with the W3C meeting approaching, 
I'd like to get a sense of how we make progress in this area.  Should 
we (Apple) edit this into the Wiki, should we (Apple or WhatWG) carry 
the proposal to the W3C, and if so, which group?  And so on.

Thanks for re-raising this!

* * * * *

Date: Fri, 08 Jun 2007 16:22:00 -0700
From: Dave Singer <singer at apple.com>
Subject: [whatwg] accessibility management for timed media elements, proposal
Sender: whatwg-bounces at lists.whatwg.org
To: WHATWG <whatwg at whatwg.org>
X-Original-To: whatwg at lists.whatwg.org
List-Post: <mailto:whatwg at lists.whatwg.org>
List-Subscribe: <http://lists.whatwg.org/listinfo.cgi/whatwg-whatwg.org>,
	<mailto:whatwg-request at lists.whatwg.org?subject=subscribe>
List-Unsubscribe: <http://lists.whatwg.org/listinfo.cgi/whatwg-whatwg.org>,
	<mailto:whatwg-request at lists.whatwg.org?subject=unsubscribe>
List-Archive: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org>
List-Help: <mailto:whatwg-request at lists.whatwg.org?subject=help>
List-Id: Public mailing list for the WHAT working group <whatwg-whatwg.org>

Hi

we promised to get back to the whatwg with a proposal for a way to 
handle accessibility for timed media, and here it is.  sorry it took 
a while...

* * * * *

To allow the UA to select among alternative sources for media 
elements based on users' accessibility preferences, we propose to:

1) Expose accessibility preferences to users
2) Allow the UA to evaluate the suitability of content for specific 
accessibility needs via CSS media queries

Details:

1) Expose accessibility preferences to users

Proposal: user settings that correspond to a accessibility needs. For 
each need, the user can choose among the following three dispositions:

   * favor (want): I prefer media that is adapted for this kind of 
accessibility.
   * disfavor (don't want): I prefer media that is not adapted for 
this kind of accessibility.
   * disinterest (don't care): I have no preference regarding this 
kind of accessibility.

The initial set of user preferences for consideration in the 
selection of alternative media resources correspond to the following 
accessibility options:

   captions (corresponds to SMIL systemCaptions)
   descriptive audio (corresponds to SMIL systemAudioDesc)
   high contrast video
   high contrast audio (audio with minimal background noise, music 
etc., so speech is maximally intelligible)

This list is not intended to be exhaustive; additional accessibility 
options and corresponding preferences may be considered for inclusion 
in the future.

Herein we describe only those user preferences that are useful in the 
process of evaluating multiple alternative media resources for 
suitability. Note that these proposed preferences are not intended to 
exclude or supplant user preferences that may be offered by the UA to 
provide accessibility options according to the W3C accessibility 
guidelines, such as a global volume control 
<http://www.w3.org/TR/WAI-USERAGENT/uaag10-chktable.html>.

2) Allow the UA to evaluate the suitability of content for specific 
accessibility needs via CSS media queries

Note that the current specification of <video> and <audio> includes a 
mechanism for selection among multiple alternate resources 
<http://www.whatwg.org/specs/web-apps/current-work/#location>. The 
scope of our proposal here is to extend that mechanism to cover 
accessibility options.

Proposal: the media attribute of the <source> element as described in 
the current working draft of Web Applications 1.0 takes a CSS media 
query as its value <http://www.w3.org/TR/css3-mediaqueries/>, which 
the UA will evaluate in the process of selecting an appropriate media 
resource for presentation. To extend the set of media features that 
can be queried to include accessibility preferences, we define a new 
media feature for each supported accessibility preference:

   captions
   descriptive-audio
   high-contrast-video
   high-contrast-audio

For each of these media features the following values are defined:

   * The user prefers media adapted for this kind of accessibility (": want").
   * The user prefers media that is not adapted for this kind of 
accessibility (": dont-want").
   * The user has expressed no preference regarding this kind of 
accessibility (": either").

For each media feature that corresponds to accessibility preferences, 
an expression evaluates to FALSE if and only if the user has an 
explicit preference (want or don't want), and the media feature has a 
value of want or dont-want that doesn't correspond.  For all other 
combinations (user disinterest or a value of "either"), then the 
expression evaluates to TRUE.

Example. If the user has asked for
   captions:  want
   high contrast video:  don't want

and the video element has
<video ... >
   <source media="all and (captions: dont-want)" ... />
   <source media="all and (captions: either)" ... />
</video>

The second source will be selected for presentation; the second would 
also be selected if the media attribute were completely omitted.

Once a candidate source has been selected, the UA must attempt to 
apply the user's accessibility preferences to its presentation, so 
that adaptable content is presented appropriately.

--
David Singer
Apple/QuickTime

-- 
David Singer
Apple/QuickTime