[whatwg] <device> element, streams and peer-to-peer connections

[Robin Berjon]

On May 31, 2010, at 03:01 , James Salsman wrote:
> On Sat, May 29, 2010 at 4:57 PM, Mark Frohnmayer
> <mark.frohnmayer at gmail.com> wrote:
>> On Thu, May 27, 2010 at 9:26 PM, James Salsman <jsalsman at gmail.com> wrote:
>>> Would it be appropriate to allow selection between reliable delivery
>>> involving delay and unreliable delivery with the shorter delay
>>> characteristics of UDP by allowing the user to select between the
>>> TCP-based asynchronous HTTP/HTTPS multipart/form-encoded POST of input
>>> type=file accept=audio as per http://www.w3.org/TR/device-upload and
>>> use UDP for synchronous or asynchronous device element I/O?
>> 
>> I can see use cases for both methods -- a voice mail, server based
>> application could use a simple form submit upload, but a live voice
>> conferencing app would need real-time access more like in the Capture
>> API that the W3C DAP group has published:
>> http://www.w3.org/TR/capture-api/
> 
> It's hard for me to take http://www.w3.org/TR/capture-api/#formatdata
> seriously.  There are no references to open codecs or codec
> parameters; the only audio codec specified is audio/x-wav, which is a
> Microsoft-defined union type (RIFF) with a huge number of different
> possible instance types, including only a few poor quality open
> vocoders and audio codecs by contemporary performance/bandwidth
> standards.  Where is speex or ogg vorbis?  Where are their quality and
> bit rate parameters?  Why is http://www.w3.org/TR/capture-api/#future
> empty when most of the normative sections say, "No exceptions"?  Where
> is the compatibility with existing file transfer standards?  The
> security section doesn't contemplate permissions revocation.

When a specification is fully complete, mature, and stable, we tend to release it. A rather shocking consequence of this policy is that drafts tend to be incomplete. What's even more outrageous is that sections explicitly marked as "under development" would be lacking in the contemplation of certain aspects department. The next thing you know it might turn out that Céline Dion karaoke sounds bad.

Two things you might not know though: 1) the current draft does not look at streamed transmission, merely integration with file upload form controls and <audio>/<video> through reuse of the File API (this can be discussed of course, but see below) and 2) we're waiting on input we expect from our friends at Mozilla who have some ideas (but alas little time to write).

One interesting suggestion they made was that capture should *not* include a direct way of streaming to a server. Rather, it should solely define how to wire a device to <audio>, <video>, <img>, etc. and there ought to be a separate document describe how to capture the output of an element and AV-stream it to a remote endpoint (or save it to disk). The reason for this is that there's no good reason why you wouldn't want to stream (or save) as AV things that originally aren't. There are plenty of use cases from capturing tutorials to the craziest collaborative film-making happening you can think of and including making a video effects editor inside the browser. Naturally, that's where protocol and format noodling would go. Personally, I think that that distinction makes sense.

Note that going forward you'll probably want to look at http://dev.w3.org/2009/dap/camera/ rather than the TR snapshots.

-- 
Robin Berjon - http://berjon.com/