[whatwg] Stream API Feedback

[Robert O'Callahan]

On Thu, Mar 17, 2011 at 4:36 AM, Lachlan Hunt <lachlan.hunt at lachy.id.au>wrote:

> I'm not entirely sure I understand your proposal, but are you suggesting
> that the input streams from the camera/microphone would first go to <video>
> and <audio> elements, allowing the existing HTMLMediaElement API on those
> elements to be used to control those streams, the output of which can then
> be encoded and recorded to a file or streamed remotely?
>

Yes.

In fact, of all the properties that are on HTMLMediaElement, the only ones
> that seem to have any real use for streaming media are volume, muted, paused
> and ended.  So I'm not convinced that it's a good idea to try and reuse
> existing APIs simple for the sake of reusing them, when they aren't really a
> good fit.


As Olli said, <video> and <audio> are designed to support streaming media;
streaming over a low-latency network is very much like streaming from a
local device.

In Gecko, we allow seeking within cached segements of streamed video, and we
could easily allow that for local devices too --- user-controlled "instant
replay".

So for an HTML video element, the following attributes could all make sense
for streaming from local devices, IMHO:
-- videoWidth/videoHeight
-- width/height (reflected to CSS)
-- poster (to show a placeholder before camera input becomes available)
-- controls (in-page controls for mute, start/stop)
-- src
-- readyState
-- currentTime (read and write)
-- paused
-- ended (the user turned off the camera)
-- duration
-- volume
-- seeking
-- seekable
-- buffered
-- played

But that's not particularly useful for the audio element. It's rare that the
> user would want their microphone input to be echoed back to them via an
> audio element. In most cases, when a microphone stream is input into an
> audio element, the audio element itself would need to be muted to prevent
> unwanted and annoying echo or, worse, feedback loops.


Yes, direct audio output would have to be muted. This could be done
automatically when input is coming directly from a local device. (Assuming
that using your Web browser as a megaphone is not a valid use-case :-).)

That would only be useful if the audio data were being analysed and output,
> for example, to an audio spectrum visualisation (like with Mozilla's
> experimental audio data API).


Not just analysis, but also processing. For example, XBox Live lets you
distort your audio input in voice chat.

Rob
-- 
"Now the Bereans were of more noble character than the Thessalonians, for
they received the message with great eagerness and examined the Scriptures
every day to see if what Paul said was true." [Acts 17:11]