[whatwg] Codecs for <audio> and <video>
Charles Pritchard
chuck at jumis.com
Mon Jul 6 18:03:47 PDT 2009
Ian Hickson wrote:
> On Thu, 2 Jul 2009, Charles Pritchard wrote:
>
>> I'd like to see <canvas> support added to the <video> tag (it's as
>> natural as <img>).
>>
>
> <video> elements can be painted onto <canvas> elements already; did you
> have something more in mind?
>
This is sufficient. <video> can be used with the drawImage tag,
and original video element can be hidden.
>> and enable the <audio> tag to accept raw data lpcm),
>> just as the <canvas> tag accepts raw data (bitmap).
>>
>
> This is on the list of things to consider in a future version. At this
> point I don't really want to add new features yet because otherwise we'll
> never get the browser vendors caught up to implementing the same spec. :-)
>
Consider a programmable <audio> element as a priority.
Apple has said they will not carry the Vorbis codec in their distributions.
I don't see any objection in them carrying a programmable <audio> element.
With a raw data api, Vorbis enthusiasts may share their codec.
Otherwise, HTML 5 is codec naive.
Programmable audio has support in Flash and Java, and their wide audience.
WebIDL is described in ECMAScript and Java.
I believe this is an achievable compromise.
It's already available in ActionScript and Java,
and their associated VMs installed on well over 90% of web browsers,
with open source development tools and compilers.
>> And add an event handler when subtitles are enabled / disabled.
>>
>
> This is on the list for the next version. (Generally speaking, control
> over subtitles is one of the very next features that will be added.)
>
>
Meanwhile, using a <canvas> tag, drawImage(HTMLVideoElement) and fillText
are sufficient.
>> I'd think that FLAC would make more sense than PCM-in-Wave, as a PNG
>> analog.
>>
>
> I encourage you to suggest this straight to the relevant vendors. As it
> stands, HTML5 is codec-neutral.
>
Perhaps it's not necessary.
There's no need to standardize on an audio codec.
It can be handled in a device independent manner,
provided the player for that codec can be expressed in
ECMAScript+Java (see: WebIDL).
>> I'd like to see a font analog in audio as well. Canvas supports the font
>> attribute, audio could certainly support sound fonts. Use a generated
>> pitch if your platform can't or doesn't store sound fonts.
>>
>
> This seems like an issue for the CSS or SVG working groups, who are
> working on font-related technologies.
>
Should it gather any momentum, this would be an attribute which
HTMLMediaElement tags should
be aware of. For example: <audio style="-ext-audio-font:
tone|url(arbitrary.url)" />.
Should the audio source be a container format, such as midi, it would
interpret the css property.
>> "User agents should provide controls to enable the manual selection of
>> fallback content."
>>
>
> There's no reason the UA couldn't provide an override mechanism to select
> an alternative source if the UA vendor so desires.
>
I was considering the current dead-lock about the situation of h.264 and
theora codecs.
An explicit statement encourages UA-developers to adopt good practices.
>> Many non-technical users will want to know why there is a black screen
>> (or still image), even though they can hear the audio.
>>
This section works well:
"User agents that cannot render the video may instead make the element
represent a link to an external video playback utility or to the video
data itself."
More information about the whatwg
mailing list