[whatwg] Canvas in workers

[Kenneth Russell]

On Mon, Oct 14, 2013 at 1:34 PM, Robert O'Callahan <robert at ocallahan.org> wrote:
> On Mon, Oct 14, 2013 at 2:20 PM, Kenneth Russell <kbr at google.com> wrote:
>>
>> Would you mind looking at the proposal
>> http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it?
>
>
> Sure. Kyle and I looked at it while we were working on our proposal. The
> main issues I have with it are that rearchitecting <canvas> to introduce the
> DrawingBuffer layer of abstraction seems unnecessarily complex, and it
> doesn't handle direct presentation of frames from the worker, bypassing the
> main thread.

Note that the CanvasInWorkers draft solves some other longstanding
issues not addressed by the WorkerCanvas proposal. It provides the
ability to render to multiple canvases from a single context, whether
workers are involved or not. It achieves ideal memory utilization by
being very explicit in the API, without the need for extensive and
subtle optimizations behind the scenes.

It's worth considering whether a change to the CanvasInWorkers
proposal could support presenting frames directly from the worker.


>> There's been some recent discussion in the WebGL WG on this topic and
>> concerns were raised from other parties at Mozilla about the
>> DrawingBuffer proposal above, including that it isn't possible to
>> render from a worker without synchronizing with the main thread.
>
>
> This is a huge limitation. For games and other animated content, achieving a
> stable frame rate is super important and a key motivation for adding canvas
> support to workers.
>
> My vision for handling the Maps use-cases based on our proposal is this: the
> worker renders to one or more WorkerCanvases, then does
> createImageBitmap(canvasContext), then ships the ImageBitmap(s) to the main
> thread using postMessage, and then renders those ImageBitmaps either by
> drawing them to a canvas or in some more direct way.
>
> This can all be implemented in a zero-copy way with the APIs currently in
> the spec, though it's not trivial. You'd need to do a few optimizations:
> -- createImageBitmap(canvasContext) would take a lazy snapshot of the canvas
> contents; i.e., further modifications to the canvas would trigger a copy (on
> the worker), but if the canvas is untouched no copy is required.
> -- structured clone of the ImageBitmap would not copy. This may actually
> require a small spec change.
> -- drawing an ImageBitmap to a 2D <canvas>, if it covers the whole canvas,
> would simply replace the canvas buffer with the ImageBitmap contents (and
> perform copy-on-write if the script later makes changes to that canvas).
> These optimizations would all be useful in other contexts too. Whatever
> happens with canvas-in-a-worker, I bet we'll end up doing these
> optimizations for other reasons.

I agree that it's probably possible to make this work. It is still
worth considering whether these optimizations are going to be fragile,
and whether developers will fall off a performance cliff if something
subtle changes in their code or in the browser's behavior in the
future.


> It might make sense to create an API that renders an ImageBitmap more
> directly than drawing it to a canvas. For example we could create an API
> that allows an <img> element to render an ImageBitmap. It would be a bit
> simpler for authors and perhaps a bit simpler to implement than the final
> optimization in my list above.

That's an interesting possibility. It would probably be quite a bit
simpler to both specify and implement rather than using a canvas on
the receiving end.

-Ken


> Rob
> --
> Jtehsauts  tshaei dS,o n" Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni le
> atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o  Whhei csha iids  teoa
> stiheer :p atroa lsyazye,d  'mYaonu,r  "sGients  uapr,e  tfaokreg iyvoeunr,
> 'm aotr  atnod  sgaoy ,h o'mGee.t"  uTph eann dt hwea lmka'n?  gBoutt  uIp
> waanndt  wyeonut  thoo mken.o w