[whatwg] VIDEO Timeupdate event frequency.

[Silvia Pfeiffer]

On Sat, Sep 11, 2010 at 2:20 PM, Robert O'Callahan <robert at ocallahan.org>wrote:

> On Sat, Sep 11, 2010 at 11:03 AM, Tab Atkins Jr. <jackalmage at gmail.com>wrote:
>
>> On Fri, Sep 10, 2010 at 4:01 PM, Robert O'Callahan <robert at ocallahan.org>
>> wrote:
>> > I think an ideal API for video frame processing would involve handing
>> video
>> > frames to a Worker for processing.
>>
>> Mm, yeah, probably.  But then you'd need to be able to do canvas on
>> workers, and hand the data back...  This is a complex problem.
>>
>
> Most of the usecases I've seen just do get/putImageData, so it might make
> sense to just provide raw frame data to the Worker and not introduce a
> canvas dependency.
>

Yes, I think that makes sense, though I would not restrict it to image data,
but include audio data (when we have the API for it). Dragging image data
through a canvas just to get to the pixels is actually really annoying. If
we could set a newFrame event on a video for a worker and the event data
contains the video image with associated audio information, that would be
the best.



> So...  no newframe event for now, leave timeupdate as it is, and fix
>> this in the future?
>>
>
> I think so. Another factor is that a lot of the video effects people have
> been using canvas for can actually be done with SVG filters, which can be
> GPU-accelerated and are compatible with asynchronous compositing. So it
> might be wise to focus on use-cases for video processing that aren't
> amenable to SVG filters (or extensions thereof), and understand what their
> requirements are.
>

Things like object segmentation, face recognition, object tracking in video,
or anything with frequency analysis in audio come to mind. Workers seem like
heaven-made for these anyway, though right now with the canvas indirection
it isn't really optimal.

Cheers,
Silvia.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20100911/1cdb4aa7/attachment-0002.htm>