[whatwg] Counterproposal for canvas in workers

[Rik Cabanier]

Hi Justin,

no, everything is running synchronized and there is no added latency.

API calls on canvas will be recorded if there are outstanding tasks.
So, for this API call:

ctx.drawImage(MinionCanvas, ...)

Since it is happening in a task, drawImage will be recorded. It will only
execute once the task and its subtasks (ie
MinionCanvas.executeTask("drawMinion",
{}) ) are done.

It sounds complicated but I think it's much easier for an author than
having to send bitmaps and message back and forth.

On Fri, Oct 18, 2013 at 6:48 AM, Justin Novosad <junov at google.com> wrote:

> Rik, I don't think the nested tasks in your example are a good use case.
>  That workflow adds a frame of latency to the sub tasks. This is a problem
> because the drawImage call would be drawing from a source canvas that it
> out of phase with the mainscene context.  To synchronize the content, I
> think the drawImage calls would have to be placed in a Promise resolution
> handler that gets invoked once all the executeTask Promises for the
> subtasks are resolved.  That means that the drawImage calls necessary for
> drawing "mainscene" would end-up executing asynchronously, therefore
> outside the scope of the "mainscene" task function, which is a problem.  So
> I don't think the executeTask proposal is well suited for sharding
> rendering jobs, at least not the way you illustrated it with that example.
>
>
>
> On Fri, Oct 18, 2013 at 12:50 AM, Rik Cabanier <cabanier at gmail.com> wrote:
>
>> Extra methods on the canvas API:
>>
>> Promise setTaskScript(DOMString script); // can error be in promise?
>> Promise executeTask(DOMString id, dictionary json, boolean synchronized =
>> true); // Transferable elements allowed in dictionary
>>
>> Object that is active in the task:
>>
>> interface CanvasTask {
>>
>> HTMLCanvasElement createCanvas(unsigned long width, unsigned long height);
>> attribute Function onTask;
>>
>> }
>>
>> CanvasTask implements HTMLCanvasElement;
>>
>> Example code:
>>
>> var c = document.getElementById("gameCanvas");
>>
>> var gameState = {};
>>
>> window.addEventListener("load", function(){
>>
>> c.setTaskScript("gameLogic.js").then(function(){
>>
>> c.executeTask("mainscene", gameState);
>>
>> });
>>
>> });
>>
>>
>> window.requestAnimationFrame(function(){
>>
>> c.executeTask("mainscene", gameState);
>>
>> }
>>
>>
>> Example code for gameLogic.js:
>>
>> var ctx = getContext("2d");
>>
>> onTask = function(DOMString id, dictionary json) {
>>
>> if(id == "mainscene") {
>>
>> if(typeof(MinionCanvas)=="Undefined") {
>>
>> MinionCanvas = createCanvas(200, 300);
>>
>>  MinionCanvas.executeTask("drawMinion", {}) // creates promise under the
>> hood
>>
>> }
>>
>> if(typeof(SpaceShipCanvas)=="Undefined")
>>
>> SpaceShipCanvas = createCanvas(300, 300);
>>
>>
>> SpaceShipCanvas.executeTask("drawSpaceShip", gameState); // redraw
>> spaceship
>>
>>
>> executeTask("drawBackDrop",  gameState); // in other task
>>
>> executeTask("drawBoss", gameState); // lots of js to draw the boss so
>> better done in task
>>
>>
>> for(...) //for each minion {
>>
>>  ... // set the matrix
>> ctx.drawImage(MinionCanvas, ...); // draw the minion <- note that the
>> minion might still be drawing in the other thread
>>
>>   }
>> for(...) //for each spaceship {
>>
>> ..// set the matrix
>>
>> ctx.drawImage(SpaceShipCanvas); // draw the spaceship <- it might still
>> be drawing in the other task
>>
>> }
>>
>> .. // other drawing commands for score, controls, etc
>>
>> } else if(id == "drawMinion") {
>>
>> ...
>>
>> }  else if(id == "drawSpaceShip") {
>>
>> ... // set up tasks to draw parts of the ship?
>>
>> } ...
>>
>> }
>>
>>
>>
>> On Thu, Oct 17, 2013 at 8:10 PM, Rik Cabanier <cabanier at gmail.com> wrote:
>>
>>>
>>>
>>>
>>> On Thu, Oct 17, 2013 at 4:01 PM, Robert O'Callahan <robert at ocallahan.org
>>> > wrote:
>>>
>>>> On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad <junov at google.com>wrote:
>>>>
>>>>> On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier <cabanier at gmail.com>wrote:
>>>>>
>>>>>> Creating temporary canvases is still possible. I'm unsure how it
>>>>>> would be different from a worker.
>>>>>> An advantage would be that you can draw to the temporary canvases in
>>>>>> parallel to using them. Only PIXEL access is disallowed, you can still call
>>>>>> drawImage using a canvas that has outstanding tasks.
>>>>>>
>>>>>
>>>>> Right. The write-only restriction would only apply to canvas contexts
>>>>> that commit (push their tasks) directly down to the compositor. You could
>>>>> still create a canvas that is local to the worker, rasterize it in the
>>>>> worker and do readbacks in the worker, create ImageBitmaps from it, etc.
>>>>>
>>>>
>>>> I'm not sure that you and Rik are talking about the same thing, since
>>>> he's still talking about "outstanding tasks". If you are talking about the
>>>> same thing, I don't know what it is. I'd like to see some concrete details
>>>> for what you'd change in the current WorkerCanvas proposal. For the sake of
>>>> clarity I've put (my understand of) it here:
>>>> https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal
>>>>
>>>
>>> I'll work on drawing up an example of my proposal.
>>>
>>> With WorkerCanvas and transferToImageBitmap, you can draw multiple
>>>> layers in parallel (and actually draw, not just queue drawing commands) by
>>>> creating multiple workers, having them each produce an ImageBitmap, and
>>>> compositing those ImageBitmaps together by stacking <img> elements or
>>>> drawing them all to a single canvas. It uses more memory but you get more
>>>> parallelism.
>>>>
>>>
>>> They would still have to wait for each other so the images are
>>> composited in-order. If you don't care about that, the 'synchronized'
>>> option would let you draw as soon as you exit the task (which is how Chrome
>>> always draws since it's faster)
>>>
>>> In fact, an implementation could choose to take the deferred-drawing
>>>> approach instead. You would queue up drawing commands in the WorkerCanvas
>>>> (or the drawing context), and then transferToImageBitmap would not
>>>> immediately render but produce an ImageBitmap implementation encapsulating
>>>> the list of drawing commands to be drawn later, wherever/whenever that
>>>> ImageBitmap ended up being used. I think for commit() the implementation
>>>> would always want to force rasterization on the worker (or possibly some
>>>> dedicated canvas-rendering thread); you could forward a list of drawing
>>>> commands to the compositor thread for rasterization but I don't think
>>>> there's any reason to do that (and some good reasons not to).
>>>>
>>>
>>> Can you tell me how you can ensure that you don't do too much work?
>>> Drawing in a continuous loop using 'Commit' would waste a lot of resources.
>>>
>>
>>
>