[whatwg] Synchronized play/seek of multiple <audio> elements?

[Silvia Pfeiffer]

Hi Emil,

On Tue, Mar 24, 2009 at 1:39 AM, Emil Tin <emil at koblo.com> wrote:
>
> i understand that SVG is meant for advanced timing etc.

Maybe rather SMIL - that's where SVG got it from.

> but it would be very useful to have a simple mechanism in html/javascript
> for playing sounds together. conceptually, sounds would be placed on a
> timeline at a certain time. the sounds on the timeline can then be played
> back together and will start at the right times.

As soon as you start going down this track, you run into complexity.

By specifying something like your example

<...>
> <audio src="pads.mp3" timeline="ambient" time="0" id="my_song" >
> <audio src="cello.mp3" timeline="ambient" time="0">
> <audio src="voice.mp3"  timeline="ambient" time="2000">
> <audio src="bells.mp3"  timeline="ambient" time="5000">

you have to tackle a lot of issues:
* is this a group that is meant to go along the same timeline and
therefore displays a single control element?
* what if you only really want to play one element and not the others
- can you pick one out and thus break the group?
* can you define subgroups and e.g. mute them together?
* what if  you jump to a certain offset on the controls - how is the
browser supposed to know which things are still playing in parallel
and which aren't playing any longer? (getting the duration of files
tends to be a difficult undertaking)
* what if the source files all come from different servers with
different download speeds - what's the buffering strategy?


> another idea would be to group the audio tags inside a timeline tag, but
> that might be a more complicated change?
>
> <timeline id="ambient">
>        <video .... time="0">
>        <audio .... time="400">
>        <audio .... time=" 1200">
> </timeline

This option is basically the option that SMIL has gone down. It has
enclosing tags that specify what goes in sequence and what in parallel
and has many richer constructs. For creating a audio-visual experience
out of several separate audio / video files, SMIL is probably the best
way to go right now with its <seq>, <par> and <switch> elements and
having thought through all the issues that arise around them.

I must say however that I do sympathise with a simpler approach than
SMIL if it is possible and can deal with all the issues. I don't think
the current version of HTML5 will take it but it would be a good thing
to develop a "composition" approach for a second or third version of
HTML5. The experience from SMIL should however be fully taken into
account.

Best Regards,
Silvia.

> On 18/02/2009, at 11.20, Ian Hickson wrote:
>
>> On Wed, 18 Feb 2009, Emil Tin wrote:
>>>
>>> However, I've been unable to find any information on how to trigger
>>> several audio files in a synchronized manner.
>>>
>>> Does the html 5 specification provide any way to to this?
>>
>> Not currently.
>>
>> --
>> Ian Hickson               U+1047E                )\._.,--....,'``.    fL
>> http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
>> Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'
>
>