[whatwg] Canvas in workers

[Kenneth Russell]

Would you mind looking at the proposal
http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it? This
was arrived at after extensive discussions with the Google Maps team,
and addresses their key use cases. Compared to the one below, it
solves the following problems:

  1) Rendering from a worker and displaying on the main thread with no
extra blits of the rendering results
  2) Allows one context to render to multiple canvases
  3) Supports resizing of the drawing buffer

The main difference in my mind is that in the DrawingBuffer proposal,
the back buffer of the canvas can be "detached", transferred via
postMessage to the other side, and attached to a Canvas, replacing its
previous back buffer. The semantics are simple, clear, avoid extra
blits of the rendered content, and support rendering into multiple
canvases from one context.

There's been some recent discussion in the WebGL WG on this topic and
concerns were raised from other parties at Mozilla about the
DrawingBuffer proposal above, including that it isn't possible to
render from a worker without synchronizing with the main thread.
Still, it seems to me it's worth considering all aspects of the
proposal, because it was motivated by a major existing web app which
is both using Canvas 2D and WebGL extensively, and desires to use
workers more heavily in its rendering pipeline.

-Ken



On Sat, Oct 12, 2013 at 9:12 PM, Kyle Huey <me at kylehuey.com> wrote:
> I talked at length with Robert O'Callahan about what the DOM API for
> supporting <canvas> in web workers should look like and we came up with the
> following modifications to the spec.
>
>    1. Rename CanvasProxy to WorkerCanvas and only allow it to be
>    transferred to workers.  I don't think we're interested in supporting
>    cross-origin <canvas> via CanvasProxy (I would be curious to hear more
>    about what the use cases are).
>    2. Add a worker-only WorkerCanvas constructor that takes the desired
>    width/height of the drawing surface.
>    3. Remove the rendering context constructors and the setContext method
>    on WorkerCanvas (née CanvasProxy).
>    4. Copy all of the sensible non-node related things from
>    HTMLCanvasElement to WorkerCanvas.  This would include
>    - width and height as readonly attributes
>       - getContext (to replace what we removed in step 3).  roc prefers to
>       have getContext2D and getContextWebGL, and dispense with the string
>       argument version entirely, but I don't have strong feelings.
>       - toBlob.  We do not intend to implement toDataURL here.
>    5. Add a "commit" method to WorkerCanvas.  For a WorkerCanvas obtained
>    from a main thread <canvas> element, this would cause the buffer displayed
>    on screen to swap.  For a WorkerCanvas created *de novo* on a worker
>    thread, it would do nothing.  This commit method would also commit a minor
>    violation of run-to-completion semantics, described below.
>    6. We would rely on extracting ImageBitmaps from the WorkerCanvas and
>    shipping them to the main thread via postMessage to allow synchronizing
>    canvas updates with DOM updates.  We explored a couple other options but we
>    didn't come up with anything else that allows synchronizing updates to
>    multiple canvases from a worker.  This isn't really sketched out here.
>
> So the IDL would look something like:
>
>> [Constructor(unsigned long width, unsigned long height)]
>>
>> interface WorkerCanvas {
>>
>>   readonly attribute unsigned long width;
>>
>>   readonly attribute unsigned long height;
>>
>>
>>   CanvasRenderingContext2D? getContext2D(any... args);
>>
>>   WebGLRenderingContext? getContextWebGL(any... args);
>>
>>
>>   void toBlob(FileCallback? _callback, optional DOMString type, any...
>> arguments);
>>
>>
>>   bool commit();
>>
>> };
>>
>> WorkerCanvas implements Transferable;
>>
> Everything would be behave pretty much as one would expect, except perhaps
> for the commit method.  The width and height of the canvas can be modified
> on the main thread while the worker is drawing.  This would fire an event
> off to the worker to update the WorkerCanvas's dimensions that would be
> scheduled as if the main thread had postMessage()d something to the
> worker.  But it's possible that the worker would attempt to draw to the
> <canvas> before that update runs.  It's also possible that the worker would
> simply draw in a loop without yielding.  To solve this, if commit is called
> and the current dimensions on the main thread don't match the dimensions of
> the WorkerCanvas it would fail (return false) and update the dimensions of
> the WorkerCanvas before returning.  This is technically a violation of
> run-to-completion semantics, but is needed to support workers that do not
> yield.
>
> Thoughts?
>
> - Kyle