[whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread

[Kenneth Russell]

On Mon, Oct 21, 2013 at 3:39 PM, Glenn Maynard <glenn at zewt.org> wrote:
> On Sun, Oct 20, 2013 at 11:16 PM, Robert O'Callahan <robert at ocallahan.org>
> wrote:
>>
>> With all these proposals I think it's OK to allow the main thread to do
>> (e.g.) a toDataURL and read what the current contents of the canvas is,
>
>
> We can defer this discussion, since it's not something new to this proposal
> (or any other proposal we're discussing).
>
>
> On Sun, Oct 20, 2013 at 11:33 PM, Robert O'Callahan <robert at ocallahan.org>
> wrote:
>>
>> To me, passing the image data explicitly in an ImageBuffer along with the
>> "present" message seems like a better fit to the workers message-passing
>> model than this proposal, where the data is stored as hidden state in the
>> canvas element with (effectively) a setter in the worker and a getter in the
>> main thread, and that setting and getting has to be coordinated with
>> postMessage for synchronization. The relationship between a "commit" and its
>> "present" has to be deduced by reasoning about the timing of messages,
>> rather than by just reasoning about JS data flow through postMessage.
>
>
> Using ImageBitmap for this has a lot of issues.  It requires synchronizing
> with scripts in the UI thread.

This isn't difficult, and amounts to a few additional lines of code in
the main thread's onmessage handler.

The ImageBitmap style proposal has another significant advantage in
that it allows a single canvas context to present results in multiple
output regions on the page.


>  It requires manualling resize your canvas
> repeatedly to fit different destinations.  It also may potentially create
> lots of backbuffers. Here's an example of code using ImageBitmap
> incorrectly, leading to excess memory allocation:
>
> function render()
> {
>     var canvas = myWorkerCanvas;
>     renderTo(canvas);
>     var buffer = canvas.transferToImageBitmap();
>     postMessage(buffer);
> }
> setTimeout(render, 1);
>
> We start with one backbuffer available, render to it (renderTo), peel it off
> the canvas to be displayed somewhere, and toss it off to the main thread.
> (For the sake of the example, the main thread is busy and doesn't process it
> immediately.)  The worker enters render() again, and when it gets to
> renderTo, a new backbuffer has to be allocated, since the one buffer we have
> is still used by the ImageBuffer and can't be changed.  This happens
> repeatedly, creating new backbuffers each time, since none of them can be
> reused.
>
> This is an extreme example, but if this ever happens even once, it means
> potentially allocating an extra backbuffer.

This sort of resource exhaustion is certainly possible, but I view
this downside as smaller than the upside of addressing both of the
above use cases.

-Ken


>>
>> This proposal also requires that whenever a worker is going to return
>> image data to the main thread, the main thread must start things off by
>> creating a canvas element. It's also not possible for a worker to spawn off
>> sub-workers to do drawing (at least, not without some really ugly
>> coordination with the main thread.)
>
>
> Sure it is.  If you want an offscreen buffer, you just "new WorkerCanvas()".
> This is unrelated to offscreen drawing.
>
> --
> Glenn Maynard
>