[whatwg] Canvas in workers

[David Bruant]

Le 13/10/2013 06:12, Kyle Huey a écrit :
> I talked at length with Robert O'Callahan about what the DOM API for
> supporting <canvas> in web workers should look like and we came up with the
> following modifications to the spec.
>
>     1. Rename CanvasProxy to WorkerCanvas and only allow it to be
>     transferred to workers.  I don't think we're interested in supporting
>     cross-origin <canvas> via CanvasProxy (I would be curious to hear more
>     about what the use cases are).
>     2. Add a worker-only WorkerCanvas constructor that takes the desired
>     width/height of the drawing surface.
What is the use case for this constructor? Draw something in worker, 
output an image to be sent to main thread?

>     3. Remove the rendering context constructors and the setContext method
>     on WorkerCanvas (née CanvasProxy).
>     4. Copy all of the sensible non-node related things from
>     HTMLCanvasElement to WorkerCanvas.  This would include
>     - width and height as readonly attributes
>        - getContext (to replace what we removed in step 3).  roc prefers to
>        have getContext2D and getContextWebGL, and dispense with the string
>        argument version entirely, but I don't have strong feelings.
For the sake of writing interoperable code in main thread and worker, I 
tend to be against this sort of change. I'm aware of the ugliness of 
some APIs, but consistent ugly APIs beats a mix of beautiful and ugly API.

>        - toBlob.  We do not intend to implement toDataURL here.
>     5. Add a "commit" method to WorkerCanvas.  For a WorkerCanvas obtained
>     from a main thread <canvas> element, this would cause the buffer displayed
>     on screen to swap.  For a WorkerCanvas created *de novo* on a worker
>     thread, it would do nothing.
Let's have this method optional, then? Or create 2 interfaces? I'm not 
sure of what can be expressed in WebIDL to solve this, but useless 
methods aren't a good idea. If I can't do anything, don't give me the 
method. It's like in UI. If I can't click a button, just don't show me 
the button or at least grey it out.


>     This commit method would also commit a minor
>     violation of run-to-completion semantics, described below.
>     6. We would rely on extracting ImageBitmaps from the WorkerCanvas and
>     shipping them to the main thread via postMessage to allow synchronizing
>     canvas updates with DOM updates.  We explored a couple other options but we
>     didn't come up with anything else that allows synchronizing updates to
>     multiple canvases from a worker.  This isn't really sketched out here.
>
> So the IDL would look something like:
>
>> [Constructor(unsigned long width, unsigned long height)]
>>
>> interface WorkerCanvas {
>>
>>    readonly attribute unsigned long width;
>>
>>    readonly attribute unsigned long height;
>>
>>
>>    CanvasRenderingContext2D? getContext2D(any... args);
>>
>>    WebGLRenderingContext? getContextWebGL(any... args);
>>
>>
>>    void toBlob(FileCallback? _callback, optional DOMString type, any...
>> arguments);
>>
>>
>>    bool commit();
Boolean as return value for success? :-s
A promise instead maybe? throw instead of false at least?
In any case, it looks like commit could be a long operation (tell me if 
I'm wrong here. Do you have numbers on how long it takes/would take?), 
having it async sounds reasonable.

>>
>> };
>>
>> WorkerCanvas implements Transferable;
>>
> Everything would be behave pretty much as one would expect, except perhaps
> for the commit method.  The width and height of the canvas can be modified
> on the main thread while the worker is drawing.  This would fire an event
> off to the worker to update the WorkerCanvas's dimensions that would be
> scheduled as if the main thread had postMessage()d something to the
> worker.  But it's possible that the worker would attempt to draw to the
> <canvas> before that update runs.  It's also possible that the worker would
> simply draw in a loop without yielding.  To solve this, if commit is called
> and the current dimensions on the main thread don't match the dimensions of
> the WorkerCanvas it would fail (return false) and update the dimensions of
> the WorkerCanvas before returning.  This is technically a violation of
> run-to-completion semantics, but is needed to support workers that do not
> yield.
I feel fairly strongly against the run-to-completion violation as it's a 
foundation of how JavaScript works, how people reason about programs. 
Also, TC39 is working hard to close the gap between what can be 
expressed in pure ECMAScript and what the web platform does express, 
it'd be inappropriate to add things that widen this gap in my opinion.

I would be much more in favor of a solution like how IndexedDB handles 
transactions. Here is how it could work:
For a canvas which WorkerCanvas has been transfered, when trying to 
change width/height:
*1 Store the values, but don't do it yet
*2 Send a message to the worker to re-dimension,
#1 in the worker: wait for the current message to be taken care of
#2 dispatch a "dimension changed" event (or equivalent)
#3 send a confirmation to main thread
#4 run the listeners
*3 back to main thread: process all committed changes before the 
confirmation
*4 Actually change the dimensions

This guarantees that the worker can always works on a canvas which it 
knows the dimensions of makes life a whole lot easier for authors.

Potentially *3 could be changed to "cancel all unprocessed committed 
changes". If commit returns a promise, the worker could be warned about 
the cancellation.

The downside of this solution is that the worker may take time to yield 
to the event loop, but that results in crappy UX and it's the authors 
responsibility to make sure it doesn't happen.
Also, in my experience, <canvas> most often keep their dimensions, so I 
don't know if it's worth polishing too much this use case.

David