[whatwg] Peer-to-peer communication, video conferencing, <device>, and related topics

[Shwetank Dixit]

On Fri, 18 Mar 2011 19:32:49 +0530, Lachlan Hunt  
<lachlan.hunt at lachy.id.au> wrote:

> On 2011-03-18 05:45, Ian Hickson wrote:
>> On Thu, 16 Sep 2010, Jonathan Dixon wrote:
>>> Further, it could be useful to provide a way to query the video source
>>> as to whether the camera is oriented relative to the screen (if the
>>> underlying system knows; consider a phone device with both a main  
>>> camera
>>> and self-view camera). This is needed to drive the decision on whether
>>> to do this horizontal flip or not. In fact, such an application may  
>>> want
>>> to somehow indicate a preference for the self-view camera when multiple
>>> cameras are present in the selection list. c.f. a movie-making app  
>>> which
>>> would prefer the outward facing camera.
>>
>> Interesting.
>>
>> In getUserMedia() the input is extensible; we could definitely add
>> "prefer-user-view" or "prefer-environment-view" flags to the method  
>> (with
>> better names, hopefully, but consider that 'rear' and 'front' are
>> misleading terms -- the front camera on a DSLR faces outward from the
>> user, the front camera on a mobile phone faces toward the user). The  
>> user
>> still has to OK the use of the device, though, so maybe it should just  
>> be
>> left up to the user to pick the camera? They'll need to be able to  
>> switch
>> it on the fly, too, which again argues to make this a UA feature.
>
> We could just add flags to the options string like this:
>
> "video;view=user, audio" or "video;view=environment, audio"
>
> It's worth pointing out that The HTML Media Capture draft from the DAP  
> WG uses the terms "camera" and "camcorder" for this purpose, but I find  
> these terms to be very ambiguous and inappropriate, and so we should not  
> use them here.
Just wanted to know whether there is any consensus on this or not? Mobile  
phones are coming out with dual cameras (front and back facing) and  
depending on the use case, the developer might want access to either the  
front or back one. (For example, for a simple camera app, a back facing  
will do, but for a web conferencing app, the front facing will be  
required). At least, the developer should be able to specify which one to  
enable by default, which then can be changed the user if needed.

Another question is flash. As far as I have seen, there seems to be no  
option to specify whether the camera needs to use flash or not. Is this  
decision left up to the device? (If someone is making an app which is just  
clicking a picture of the person, then it would be nice to have the camera  
use flash in low light conditions).
>
> http://dev.w3.org/2009/dap/camera/
>
>> Similarly for exposing the kind of stream: we could add to  
>> GeneratedStream
>> an attribute that reports this kind of thing. What is the most useful  
>> way
>> of exposing this information?
>
> I'm not entirely clear about what the use cases are for knowing if the  
> camera is either user-view or environment-view.  It seems the more  
> useful information to know is the orientation of the camera.  If the  
> user switches cameras, that could also be handled by firing orientation  
> events.
>
>> Lachlan Hunt wrote:
>>> There are some use cases for which it would be useful to know the
>>> precise orientation of the camera, such as augmented reality
>>> applications.  The camera orientation may be independent of the  
>>> device's
>>> orientation, and so the existing device orientation API may not be
>>> sufficient.
>>
>> It seems like the best way to extend this would be to have the Device
>> Orientation API apply to GeneratedStream objects, either by just having
>> the events also fire on GeneratedStream objects, or by having the API be
>> based on a pull model rather than a push model and exposing an object on
>> GeneratedStream objects as well as Window objects.
>
> This could work.  But it would make more sense if there were an object  
> representing the device itself, as in Rich's proposal, and for the  
> events to be fired on that object instead of the stream.
>
>> On Mon, 24 Jan 2011, Anne van Kesteren wrote:
>>>
>>> There is a plan of allowing direct assigning to IDL attributes besides
>>> creating URLs.
>>>
>>> I.e. being able to do:
>>>
>>>   audio.src = blob
>>>
>>> (The src content attribute would then be something like  
>>> "about:objecturl".)
>>>
>>> I am not sure if that API should work differently from creating URLs  
>>> and
>>> assigning those, but we could consider it.
>>
>> Could you elaborate on this plan?
>
> This is basically what Philip and I were discussing in the other thread  
> yesterday, where we avoid the unnecessary overhead of creating a magic  
> URL, and instead just assign the object directly to the src property.  
> This lets the implementation handle all the magic transparently in the  
> background, without bothering to expose a URLs string to the author.
>
> This is what we had implemented in our experimental implementation of  
> the <device> element, and now getUserMedia.
>
> i.e.
>
> <video></video>
> <script>
> var v = document.querySelector("video");
> navigator.getUserMedia("video", function(stream) {
>    v.src = stream;
>    v.play();
> });
> </script>
>
> The getter for v.src then returns "about:streamurl".
>
> My understanding is that we don't really want to have to implement the  
> create/revokeObjectURL() methods for this.
>
>> On Wed, 16 Feb 2011, Anne van Kesteren wrote:
>>> This is just a thought. Instead of acquiring a Stream object
>>> asynchronously there always is one available showing transparent black
>>> or some such. E.g. navigator.cameraStream. It also inherits from
>>> EventTarget. Then on the Stream object you have methods to request
>>> camera access which triggers some asynchronous UI. Once granted an
>>> appropriately named event is dispatched on Stream indicating you now
>>> have access to an actual stream. When the user decides it is enough and
>>> turns of the camera (or something else happens) some other  
>>> appropriately
>>> named event is dispatched on Stream again turning it transparent black
>> again.
>>
>> This is a very interesting idea.
>
> This suggests that there would be a separate property available for the  
> microphone, and any other input device.  This differs from the existing  
> spec, which allowed a single stream to represent both audio and video.
>
>> On Mon, 14 Mar 2011, Lachlan Hunt wrote:
>>> The API includes both readystatechange event, as well as independent
>>> events for play, paused and ended.  This redundancy is unnecessary.  
>>> This
>>> is also inconsistent with the design of the HTMLMediaElement API, which
>>> does not include a readystatechange event in favour on separate events
>>> only.
>>
>> I've dropped readystatechange.
>>
>> I expect to drop play and pause events if we move to the model described
>> above that pauses and resumes audio and video separately.
>
> It may still be useful to have events for this, if the event object had  
> a property that indicated which type of stream it applied to, or if  
> there were separate objects for both the audio and video streams.
>


-- 
Shwetank Dixit
Web Evangelist,
Site Compatibility / Developer Relations / Core Engineering Group
Member - W3C Mobile Web for Social Development (MW4D) Group
Member - Web Standards Project (WaSP) - International Liaison Group
Opera Software - www.opera.com

Using Opera's revolutionary email client: http://www.opera.com/mail/