[whatwg] Codecs for <audio> and <video>

[Chris McCormick]

Hi Boris,

On Sat, Aug 08, 2009 at 11:18:36PM -0400, Boris Zbarsky wrote:
> Chris McCormick wrote:
>> It's a bit of an open ended how-long-is-a-piece-of-string sort of a question in
>> that you will generally make synthesizers which require less CPU than what your
>> computer is able to provide, for the obvious reason that they won't work
>> otherwise. So the real answer is that you want the DSP system to go absolutely
>> as fast as possible I guess, so that you can squeeze as much synthesis out as
>> possible.
>
> OK, sure.  But you indicated that ECMAScript would be unacceptable for  
> the use cases, period....

Oh did I? Sorry about that, I didn't mean to.

>>  So you could do some
>> simple synthesis with pure Javascript if it was able to loop through say 100
>> arrays of 4410 samples each, doing vector operations on those arrays, in under
>> 100ms.
>
> See attached HTML file.  It creates 103 arrays of length 4410, then goes  
> through them starting with array #4 and sets array n to the product of  
> arrays n-1 and n-2 plus array n-3.
>
> If I initialize the arrays with random 16-bit integers, the numbers I  
> see are like so (on a year-old laptop):
>
> Gecko (build that is pretty close to Firefox 3.6a1):
> Setup: 96ms
> Vector ops: 43ms
>
> Webkit nightly (in Safari, so using SFX):
> Setup: 170ms
> Vector ops: 41ms
>
> I don't have a Chrome build around to test how well V8 would do on this  
> testcase, and Opera 10 beta is giving me numbers about 20x slower than  
> the above.  Firefox 3 gives numbers about 5x slower than the above.  I  
> can't test IE over here easily (it's a Mac laptop), but I would expect  
> it to be somewhat slower than the above numbers too.
>
> So I can certainly see how you would feel that ECMAScript is not fast  
> enough for this sort of thing: until recently it wasn't, for the most  
> part.  But at this point it seems to be ok, if not great (I have about  
> 2x headroom for your target numbers for simple sound synthesis over  
> here; someone with an older computer would have less, and more  
> complicated operations might take more time).  And I can tell that at  
> least in Gecko's case there's ongoing work to make array access faster...
>
> So it seems fairly likely to me that all UAs will end up with ECMAScript  
> implementations fast enough to do what you want here sooner than all UAs  
> would implement a brand-new set of functionality.  Certainly that's the  
> case for Safari and Gecko-based browsers.  :)

For sure, I have no doubt that they'll eventually be fast enough. The thing
with DSP though, is every last vector operation that you can squeeze out of
your CPU is important. I still think it would be worth implementing vector
operators, even if raw ECMAScript is "fast enough". In your example above, if
there was an operation which took 57ms it would cause stutterring or dropouts
in the sound.

Best,

Chris.

-------------------
http://mccormick.cx