[whatwg] Codecs for <audio> and <video>

Charles Pritchard chuck at jumis.com
Mon Jul 6 18:44:25 PDT 2009

Ian Hickson wrote:
> On Mon, 6 Jul 2009, Charles Pritchard wrote:
>> Ian Hickson wrote:
>>> On Mon, 6 Jul 2009, Charles Pritchard wrote:
>>>>> This is on the list of things to consider in a future version. At 
>>>>> this point I don't really want to add new features yet because 
>>>>> otherwise we'll never get the browser vendors caught up to 
>>>>> implementing the same spec. :-)
>>>> Consider a programmable <audio> element as a priority.
>>> Could you elaborate on what your use cases are? Is it just the ability 
>>> to manually decode audio tracks?
>> Some users could manually decode a Vorbis audio stream.
>> I'm interested in altering pitch and pre-mixing channels. I believe some 
>> of these things are explored in CSS already.
>> There are accessibility cases, for the visually impaired, and I think 
>> that they will be better explored.
> If you could elaborate on these use cases that would be really useful. How 
> do you envisage using these features on Web pages?
Use a sound of varying pitch to hint to a user the location of their mouse
(is it hovering over a button, is it x/y pixels away from the edge of 
the screen, how close is it to the center).

Alter the pitch of a sound to make a very cheap midi instrument.

Pre-mix a few generated sounds, because the client processor is slow.

Alter the pitch of an actual audio recording, and pre-mix it,
to give different sounding voices to pre-recorded readings of a single text.
As has been tried for "male" "female" sound fonts.

Support very simple audio codecs, and programmable synthesizers.

The API must support a playback buffer.

putAudioBuffer(in AudioData) [ Error if audiodata properties are not 
supported ]
createAudioData( in sample hz, bits per sample, length ) [ Error if 
properties are not supported. ]
AudioData ( sampleHz, bitsPerSample, length, AudioDataArray )
AudioDataArray ( length, IndexGetter, IndexSetter ) 8 bits per property.

I think that's about it.

> (There has been some discussion of suppoting an "audio canvas" before, but 
> a lack of compelling use cases has really been the main blocker. Without a 
> good understanding of the use cases, it's hard to design an API.)

More information about the whatwg mailing list