[whatwg] Offscreen canvas (or canvas for web workers).

[Jonas Sicking]

On Fri, Mar 12, 2010 at 10:07 PM, Maciej Stachowiak <mjs at apple.com> wrote:
> > On Mar 12, 2010, at 6:20 PM, Jonas Sicking wrote:
> > Oh, another thing to keep in mind is that if/when we add fromBlob to
> > the main-thread canvas, it has to be asynchronous in order to avoid
> > main thread synchronous IO. This isn't a big deal, but I figured I
> > should mention it while we're on the subject.
>
> This is part of why I think Blob is the wrong tool for the job - we really
> want to use a data type here that can promise synchronous access to the
> data. When you copy the canvas backing store to a new in-memory
> representation, it seems silly to put that behind an interface that's the
> same as data to which you can only promise async access, such as part of a
> file on disk. There's nothing about copying bits from one canvas to another
> that needs to be async.
> (Also it's not clear to me why a Blob would be faster to copy from, copy to,
> or copy cross-thread than ImageData; I thought the motivation for adding it
> was to have a binary container that can be uploaded to a server via XHR.)

There is a use case, which I suspect is quite common, for using
<canvas> to manipulate files on the users file system. For example
when creating a photo uploader which does client side scaling before
uploading the images, or for creating a web based GIMP like
application.

In this case we'll start out with a File object that needs to be read
in to a <canvas>. One solution could be to read the File into memory
in a ByteArray (or similar) and add a synchronous
canvas2dcontext.fromByteArray function. This has the advantage of
being more generic, but the downside of forcing both the encoded and
decoded image to be read into memory.

This is why I suggested adding a asynchronous fromBlob function.

For extracting image data from a <canvas> I agree that a toBlob
function has little advantage over a toByteArray function (with the
possible exception that ByteArray so far is still vapor ware).

> In general I wonder if we should add API to convert directly between
> Blob and ImageData. Or at least Blob->ImageData and
> ImageData->ByteArray. That could avoid overhead of going through a
> canvas context. That is probably a win no matter which thread we are
> on.
>
> We could even add APIs to rotate and scale ImageData objects directly.
> If those are asynchronous the implementation could easily implement
> them using a background thread. I'm less sure that this is worth it
> though given that you can implement this yourself using workers if we
> add the other stuff we've talked about.
>
> Scaling and rotation can be done with just pixels if you code it by hand,
> but you can get native code to do it for you if you can manipulate actually
> offscreen buffers - you just establish the appropriate transform before
> painting the ImageData. REally the question is, how much slower is a scaling
> or rotating image paint than an image paint with the identity transform? Is
> it more than twice as expensive? That's the only way copying image data to a
> background thread will give you a responsiveness win. I'd like to see some
> data to establish that this is the case, if scales and rotates are the only
> concrete use cases we have in mind.

I agree that data would be great. Though for scaling I suspect that
it's complicated enough that it's worth exposing *some* built in API
for doing it. Especially considering that you want to use anti
aliasing and ideally things like gamma correction. Be that through
what we already have on 2d context, or on ImageData directly.

/ Jonas