[whatwg] Thoughts on video accessibility

[Silvia Pfeiffer]

On Thu, Jul 16, 2009 at 6:28 PM, Philip Jägenstedt<philipj at opera.com> wrote:
> On Thu, 16 Jul 2009 07:58:30 +0200, Silvia Pfeiffer
> <silviapfeiffer1 at gmail.com> wrote:
>>
>>>> > 3. Timed text stored in a separate file, which is then parsed by the
>>>> > user agent and rendered as part of the video automatically by the
>>>> > browser.
>>>> >
>>>> Maybe we should consider solving this differently. Either we could
>>>> encapsulate into the video container upon download. Or we could create a
>>>> zip-file or tarball upon download. I'd just find it a big mistake to
>>>> ignore the majority use case in the standard, which is why I proposed
>>>> the <text> elements inside the <video> tag.
>>>
>>> If browser vendors are willing to merge subtitles and video files when
>>> saving them, that would be great. Is this easy to do?
>>
>> My suggestion was really about doing this server-side, which we have
>> already implemented years ago in the Annodex project for Ogg
>> Theora/Vorbis.
>>
>> However, it is also possible to do this in the browser: in the case of
>> Ogg, the browser just needs to have a multiplexing library installed
>> as well as a means to encode the subtitle file (which I like to call a
>> "text codec"). Since it's text, it's nowhere near as complex as
>> encoding audio or video and just consists of light-weight packaging
>> code. So, yes, it is totally possible to have the browsers create a
>> binary video file that has the subtitles encapsulated that were
>> previously only accessible as referenced text files behind a separate
>> URL.
>>
>> The only issue I see is the baseline codec issue: every browser that
>> wants to support multiple media formats has to implement this
>> multiplexing and text encoding for every media encapsulation format
>> differently, which is annoying and increases complexity. It's however
>> generally a small amount of complexity compared to the complexity
>> created by having to support multiple codecs.
>
> I disagree, remuxing files would be much more of an implementation burden
> than supporting multiple codecs, at least if a format-agnostic media
> framework is used (be that internal or external to the browser). Remuxing
> would require you to support/create parts of the media framework that you
> otherwise aren't using, i.e. parsers, muxers, file writers and plugging of
> these together (which unlike decoding isn't automatic in any framework I've
> seen).

The point that I was trying to make is that if one had to only
implement it for one encapsulation format, it would be simple and a
small piece of dedicated code. However, if one has to be
format-agnostic, it indeed requires supporting parts of a media
framework that is not needed for demuxing and decoding. So, yes, I
agree with you: in the general case it might create extraneous
complexity in a browser.


> Anything is doable of course, but I think this is really something that is
> best done server-side using specialized tools.

I agree with this. This can be a special service that some servers
would offer who want to allow their users to share single video files
that contain their timed text within.

>>> It would be interesting to hear back from the browser vendors about how
>>> easily the subtitles could be kept with the video in a way that survives
>>> reuse in other contexts.
>
> I think that in the case of external subtitles the browser could simply save
> it alongside with the video. It is my experience that is media players have
> much more robust support for external subtitles (like SRT) than for internal
> subtitles, so this is my preferred option (plus it's easier).

Agreed: this would be the fallback for content downloaded from servers
that do no offer the special muxing capability.

In fact, such a separate handling of composed content through multiple
files is nothing new to HTML: all Web pages that I download from the
Internet require me to download each component of the Web page
separately: the images, the text, the css, the javascript. (Worse even
if the text is created e.g. through a database query.) I agree with
Philip that the separate handling of subtitle files and media files is
not as much of an issue as it may seem.

Regards,
Silvia.