[whatwg] Feedback on "Video conferencing and peer-to-peer communication"

[Stefan Håkansson LK]

All,

we've done a quick read through of the updated APIs and tools for "Video conferencing and peer-to-peer communication" from the latest whatwg html draft. Generally we think it is much improved over the last version, and especially a lot of details are clarified.

There are detailed comments below. We know that some stuff has been deliberately left out, and some of the things have already been addressed (after we read) - but this is a compilation after a few persons read the draft, so please bear with us.

Permission and UI (related to section 9.2)
------------------------------------------
It is not totally clear how the UI would work for granting access to use mics and cams, and furthermore how it would be possible to select several cameras (many terminals have both a front and a rear view camera) and then "tell" the web app which camera is which.

COMMENT: A mail that also addresses the issues regarding multiple cameras: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-March/030873.html

The spec says that the user-agent-specific prompt may allow user to select pre-recorded media. In that case, shouldn't it be possible to also create a Stream from a File/Blob object, containing media data?

Stream API (9.3)
----------------
Shouldn't the "ended" event be call simply "end" to match the present tempus of the other events ("pause", "play")?

The green box describes an attribute called paused which is not present in the Stream idls.

The asynchronous StreamRecorder.getRecordedData should be void. Further, the StreamRecorder API doesn't seem to support stopping a recording without stopping the entire Stream. It feels unnecessary to continue recording a long-lived Stream to support future calls to getRecordedData if the intention was to simply record a few seconds. The idl below suggests a StreamRecorder where the recorded Blob can be accessed synchronously once available.

   interface StreamRecorder {
     void stop(); // stop the recorder (not the Stream being recorded)

     readonly attribute boolean recording;
     readonly Blob data;

              attribute Function ondata; // or some other event name
   };
   StreamRecorder implements EventTarget;

Streams in PeerConnection (9.4)
-------------------------------
a) We interpret the spec as "addStream" triggers a new ICE procedure that sets up a new "channel" (5-tuple) for the stream. Correct?

b) Also related to addStream: it is not clear if the SDP (to be transmitted to the other end at getting the callback) contains all descriptions for all streams set up (minus the removed ones) so far or just the new one. The former would simplify SIP interop (re-invite).

c) addStream is uni-directional, so in our interpretation the sdp-data transmitted from sender to receiver would indicate "send-only". We guess (as mentioned above) from the description that a new ICE procedure would be deployed to set up a "channel" that is used for RTP (send direction) and RTCP (feedback). In many cases the service calls for symmetric flows, the two web apps would do "addStream" more or less simultaneously. Ideally, the "channel" (5-tuple) should be re-used. I am not sure how this can be accomplished.

d). As you already mention, it is not defined how the application could influence the media format selected. It could be discussed to what level this should be possible. But the very least should be some kind of connection between the rendering (e.g. large area at screen, small area, mono, 5.1) and the selected format.

Protection in PeerConnection(9.4)
---------------------------------
Unclear how to protect the "PeerConnection data UDP media stream" to be used by "send()" messages (sent with "send") and streams. dTLS? SRTP? How to set up and exchange keys?
Unclear how to protect the new "channel" set up by an ICE procedure at "addStream". dTLS? SRTP? How to set up and exchange keys?

Data in PeerConnection(send method 9.4)
--------------------------------------
It is stated that the data size can be up to 65467 bytes in "send()". Our network guys tell us that this is unrealistic to get over such big chunks using UDP. You would need "ack" and "re-transmission" handling in JS for such large chunks. So maybe it is better to limit the size to one MTU, and if you like to send larger chunks you would have to split them up in the application.

Event definition (9.6)
----------------------
The StreamEvent has a function called initCloseEvent.


Br,
Stefan