[whatwg] Counterproposal for canvas in workers

[Justin Novosad]

Rik, I don't think the nested tasks in your example are a good use case.
 That workflow adds a frame of latency to the sub tasks. This is a problem
because the drawImage call would be drawing from a source canvas that it
out of phase with the mainscene context.  To synchronize the content, I
think the drawImage calls would have to be placed in a Promise resolution
handler that gets invoked once all the executeTask Promises for the
subtasks are resolved.  That means that the drawImage calls necessary for
drawing "mainscene" would end-up executing asynchronously, therefore
outside the scope of the "mainscene" task function, which is a problem.  So
I don't think the executeTask proposal is well suited for sharding
rendering jobs, at least not the way you illustrated it with that example.


On Fri, Oct 18, 2013 at 12:50 AM, Rik Cabanier <cabanier at gmail.com> wrote:

> Extra methods on the canvas API:
>
> Promise setTaskScript(DOMString script); // can error be in promise?
> Promise executeTask(DOMString id, dictionary json, boolean synchronized =
> true); // Transferable elements allowed in dictionary
>
> Object that is active in the task:
>
> interface CanvasTask {
>
> HTMLCanvasElement createCanvas(unsigned long width, unsigned long height);
> attribute Function onTask;
>
> }
>
> CanvasTask implements HTMLCanvasElement;
>
> Example code:
>
> var c = document.getElementById("gameCanvas");
>
> var gameState = {};
>
> window.addEventListener("load", function(){
>
> c.setTaskScript("gameLogic.js").then(function(){
>
> c.executeTask("mainscene", gameState);
>
> });
>
> });
>
>
> window.requestAnimationFrame(function(){
>
> c.executeTask("mainscene", gameState);
>
> }
>
>
> Example code for gameLogic.js:
>
> var ctx = getContext("2d");
>
> onTask = function(DOMString id, dictionary json) {
>
> if(id == "mainscene") {
>
> if(typeof(MinionCanvas)=="Undefined") {
>
> MinionCanvas = createCanvas(200, 300);
>
>  MinionCanvas.executeTask("drawMinion", {}) // creates promise under the
> hood
>
> }
>
> if(typeof(SpaceShipCanvas)=="Undefined")
>
> SpaceShipCanvas = createCanvas(300, 300);
>
>
> SpaceShipCanvas.executeTask("drawSpaceShip", gameState); // redraw
> spaceship
>
>
> executeTask("drawBackDrop",  gameState); // in other task
>
> executeTask("drawBoss", gameState); // lots of js to draw the boss so
> better done in task
>
>
> for(...) //for each minion {
>
>  ... // set the matrix
> ctx.drawImage(MinionCanvas, ...); // draw the minion <- note that the
> minion might still be drawing in the other thread
>
>   }
> for(...) //for each spaceship {
>
> ..// set the matrix
>
> ctx.drawImage(SpaceShipCanvas); // draw the spaceship <- it might still be
> drawing in the other task
>
> }
>
> .. // other drawing commands for score, controls, etc
>
> } else if(id == "drawMinion") {
>
> ...
>
> }  else if(id == "drawSpaceShip") {
>
> ... // set up tasks to draw parts of the ship?
>
> } ...
>
> }
>
>
>
> On Thu, Oct 17, 2013 at 8:10 PM, Rik Cabanier <cabanier at gmail.com> wrote:
>
>>
>>
>>
>> On Thu, Oct 17, 2013 at 4:01 PM, Robert O'Callahan <robert at ocallahan.org>wrote:
>>
>>> On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad <junov at google.com>wrote:
>>>
>>>> On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier <cabanier at gmail.com>wrote:
>>>>
>>>>> Creating temporary canvases is still possible. I'm unsure how it would
>>>>> be different from a worker.
>>>>> An advantage would be that you can draw to the temporary canvases in
>>>>> parallel to using them. Only PIXEL access is disallowed, you can still call
>>>>> drawImage using a canvas that has outstanding tasks.
>>>>>
>>>>
>>>> Right. The write-only restriction would only apply to canvas contexts
>>>> that commit (push their tasks) directly down to the compositor. You could
>>>> still create a canvas that is local to the worker, rasterize it in the
>>>> worker and do readbacks in the worker, create ImageBitmaps from it, etc.
>>>>
>>>
>>> I'm not sure that you and Rik are talking about the same thing, since
>>> he's still talking about "outstanding tasks". If you are talking about the
>>> same thing, I don't know what it is. I'd like to see some concrete details
>>> for what you'd change in the current WorkerCanvas proposal. For the sake of
>>> clarity I've put (my understand of) it here:
>>> https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal
>>>
>>
>> I'll work on drawing up an example of my proposal.
>>
>> With WorkerCanvas and transferToImageBitmap, you can draw multiple layers
>>> in parallel (and actually draw, not just queue drawing commands) by
>>> creating multiple workers, having them each produce an ImageBitmap, and
>>> compositing those ImageBitmaps together by stacking <img> elements or
>>> drawing them all to a single canvas. It uses more memory but you get more
>>> parallelism.
>>>
>>
>> They would still have to wait for each other so the images are composited
>> in-order. If you don't care about that, the 'synchronized' option would let
>> you draw as soon as you exit the task (which is how Chrome always draws
>> since it's faster)
>>
>> In fact, an implementation could choose to take the deferred-drawing
>>> approach instead. You would queue up drawing commands in the WorkerCanvas
>>> (or the drawing context), and then transferToImageBitmap would not
>>> immediately render but produce an ImageBitmap implementation encapsulating
>>> the list of drawing commands to be drawn later, wherever/whenever that
>>> ImageBitmap ended up being used. I think for commit() the implementation
>>> would always want to force rasterization on the worker (or possibly some
>>> dedicated canvas-rendering thread); you could forward a list of drawing
>>> commands to the compositor thread for rasterization but I don't think
>>> there's any reason to do that (and some good reasons not to).
>>>
>>
>> Can you tell me how you can ensure that you don't do too much work?
>> Drawing in a continuous loop using 'Commit' would waste a lot of resources.
>>
>
>