[whatwg] Alt attribute for <video> and <audio>

[Philip Jägenstedt]

On Wed, 12 Aug 2009 14:45:36 +0200, Remco <remco47 at gmail.com> wrote:

> On Wed, Aug 12, 2009 at 1:26 PM, Philip Jägenstedt<philipj at opera.com>  
> wrote:
>> On Wed, 12 Aug 2009 12:52:38 +0200, Remco <remco47 at gmail.com> wrote:
>>
>>> On Wed, Aug 12, 2009 at 10:57 AM, Philip Jägenstedt<philipj at opera.com>
>>> wrote:
>>>>
>>>> Before suggesting any changes to the <source> element, make sure you  
>>>> have
>>>> read
>>>>
>>>> http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#concept-media-load-algorithm
>>>>
>>>> Put simply, the handling of <source> is already quite complex,
>>>> overloading
>>>> it with completely different meanings is not a good idea. <video>  
>>>> won't
>>>> handle "text/html" as a source, but if you want different media files  
>>>> for
>>>> different audiences I suggest experimenting with <source media>.
>>>
>>> <source media> doesn't do anything useful for my case. It can't load
>>> textual data. Also, if the resources are unavailable, there will be
>>> nothing to see, since all resources are off-page. It also doesn't work
>>> for iframe, object, embed or img.
>>>
>>> Is it really the idea that the only way you're going to have
>>> alternative textual content, is to Build It Yourself? You have to
>>> abuse <details> or a hidden <div> with some Javascript to build a
>>> construction that has alternative content in case the
>>> video/audio/iframe/object/embed is not available or desirable. If you
>>> want it to be semantically accessible, you even have to build another
>>> layer on top of that, in the form of ARIA attributes.
>>
>> No, in the long term we want native captions/subtitle support in the
>> browsers. See
>> http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-July/021658.html
>> and maybe  
>> http://lists.w3.org/Archives/Public/public-html/2009Aug/0439.html
>
> OK, that's awesome. I mean, really! That is on par with DVD
> functionality. But that's not exactly my case. It involves subtitling
> the video. The format has to be learned, and time has to be spent on a
> video. If a web author has no intention of doing so, but wants to give
> a short textual alternative and be done with it, he is not able to do
> that.
>
>>> Nobody will do that. Even the <source> solution is harder, maybe too
>>> hard, to use than the alt="" solution. It requires authors to create
>>> additional elements or pages to house the alternative content. Since
>>> accessibility is often an afterthought, about the most an author will
>>> be willing to do, is filling in an alt attribute.
>>
>> What do you suggest a browser do with the alt attribute? The resource
>> selection algorithm never ends until a suitable source is found, so when
>> should the alt text be displayed? By requiring anything at all, browsers
>> can't do things like display a box with a direct download link,  
>> suggestion
>> to install a specific codec, etc. If nothing at all is required of user
>> agents for the alt attribute, then I have no opinion (but then I expect  
>> no
>> one would use it either).
>
> Well, I would suggest that the browser displays the text when no
> desirable resource is available. In the case of network problems, no
> resource at all is available. In the case of a textual browser (or a
> "Disable Media" button), all videos are undesirable. You can still
> show a download link or a codec suggestion, but you can display the
> alt text below it, for the people who don't really care about a video,
> or people who know the network connection won't be back for some time,
> or people who can't or won't install the codec.
>
> I must admit that I don't understand the media selection algorithm.
> You say that it never ends. How does that work? The browser keeps
> looping through the source elements until one becomes desirable and
> available? How does that give browsers the opportunity to display a
> download link or a codec suggestion? How should textual browsers
> handle that?

The resource selection algorithm loops through the source elements and  
when it reaches the last one just waits for another source element to be  
inserted. It doesn't make any distinction between static markup and  
elements inserted via DOM, so even if you have <video><source></video> in  
your markup, it will still wait for another source element to be inserted  
via DOM. This is for spec simplicity basically, I'm not saying that it's a  
brilliant use case in itself.

The spec says:

In addition to the above, the user agent may provide messages to the user  
(such as "buffering", "no video loaded", "error", or more detailed  
information) by overlaying text or icons on the video or other areas of  
the element's playback area, or in another appropriate manner. [end quote]

Trying to specify exactly when such extra overlays are appropriate is  
difficult, because it's really just a guess. Something like when parsing  
has finished, there are no more source elements and no scripts running  
that might insert more of them. But that would be wrong sometimes, you  
have no way of predicting what future scripts might do.

> If videos are desirable and available, but text is also desirable,
> then the alt text could be displayed/spoken/etc when you tab onto it,
> as Silvia Pfeiffer proposed in a previous email.

I believe that was accomplished with some kind of ARIA attributes, correct  
me if I'm wrong.

-- 
Philip Jägenstedt
Core Developer
Opera Software