[whatwg] Codecs for <audio> and <video>

[Boris Zbarsky]

Chris McCormick wrote:
> Of course, the ECMA script is probably going to be too slow in the short term,
> so moving forward it would be great if there was a library/API which can do the
> following vector operations in the background at a speed faster than doing them
> directly, element by element inside ECMAscript (a bit like Python's Numeric
> module). All inputs and outputs are signal vectors/audio tag buffers:
> 
> * + - add two signal vectors (2 input, 1 output)
> * * - multiply two signal vectors (2 input, 1 output)
> * z - delay a signal vector with customisable sample length (2 input, 1 output)
> * read - do a table lookup (1 input, 1 output)
> * write - do a table write (2 input, 1 output)
> * copy - memcpy a signal vector (1 input, 1 output)
> * fft do a fast fourier transform - (1 input, 2 output)
> * rfft do a reverse fast fourier transform - (2 inputs, 1 output)

I'm sort of wondering what the performance of these would actually be if 
implemented directly in ECMAScript, before we decide that's too slow and 
start looking for alternate solutions.  Do you happen to have any sample 
implementations?  What size arrays are we talking about here?

I just did a quick test in SpiderMonkey, and adding two arrays of 
integers with 441000 elements each (so 10s of 44.1kHz audio; the time 
includes allocating the sum array and all that) element-by-element like so:

   var a3 = new Array(size);
   for (var j = 0; j < size; ++j) {
     a3[j] = a1[j] + a2[j];
   }

takes about 25ms on my computer.  Multiplication takes about 35ms. 
Duplicating an array takes about 20ms.  This is without any of the 
in-progress optimizations for type-specializing arrays, etc.

What sort of performance are we looking for here?

-Boris