[whatwg] <device> element, streams and peer-to-peer connections

[James Salsman]

On Sat, May 29, 2010 at 4:57 PM, Mark Frohnmayer
<mark.frohnmayer at gmail.com> wrote:
> On Thu, May 27, 2010 at 9:26 PM, James Salsman <jsalsman at gmail.com> wrote:
>>
>> Would it be appropriate to allow selection between reliable delivery
>> involving delay and unreliable delivery with the shorter delay
>> characteristics of UDP by allowing the user to select between the
>> TCP-based asynchronous HTTP/HTTPS multipart/form-encoded POST of input
>> type=file accept=audio as per http://www.w3.org/TR/device-upload and
>> use UDP for synchronous or asynchronous device element I/O?
>
> I can see use cases for both methods -- a voice mail, server based
> application could use a simple form submit upload, but a live voice
> conferencing app would need real-time access more like in the Capture
> API that the W3C DAP group has published:
> http://www.w3.org/TR/capture-api/

It's hard for me to take http://www.w3.org/TR/capture-api/#formatdata
seriously.  There are no references to open codecs or codec
parameters; the only audio codec specified is audio/x-wav, which is a
Microsoft-defined union type (RIFF) with a huge number of different
possible instance types, including only a few poor quality open
vocoders and audio codecs by contemporary performance/bandwidth
standards.  Where is speex or ogg vorbis?  Where are their quality and
bit rate parameters?  Why is http://www.w3.org/TR/capture-api/#future
empty when most of the normative sections say, "No exceptions"?  Where
is the compatibility with existing file transfer standards?  The
security section doesn't contemplate permissions revocation.

If audio were segmented into separate files as per
http://www.w3.org/TR/capture-api/#captureaudiooptions how would that
affect real-time performance on mobile devices?  Are these files
required to have sequence numbers?  With phase vocoder time shifting,
UDP delivery as per http://dev.w3.org/html5/html-device/#stream-api
would be far superior in quality and intelligibility under packet loss
or delay, assuming they went with an open audio codec (or, even
better, allowed a choice of speex or ogg vorbis.)

Regards,
James Salsman