[whatwg] Offscreen canvas (or canvas for web workers).

[Maciej Stachowiak]

On Mar 12, 2010, at 6:20 PM, Jonas Sicking wrote:

> On Fri, Mar 12, 2010 at 4:19 PM, Jonas Sicking <jonas at sicking.cc>  
> wrote:
>> On Fri, Mar 12, 2010 at 3:38 PM, David Levin <levin at google.com>  
>> wrote:
>>>
>>>
>>> On Fri, Mar 12, 2010 at 2:35 PM, Jonas Sicking <jonas at sicking.cc>  
>>> wrote:
>>>>
>>>> On Fri, Mar 12, 2010 at 12:46 PM, Oliver Hunt <oliver at apple.com>  
>>>> wrote:
>>>>>
>>>>> On Mar 12, 2010, at 12:16 PM, Jonas Sicking wrote:
>>>>>> I'm not saying that the proposed API is bad. It just doesn't  
>>>>>> seem to
>>>>>> solve the (seemingly most commonly requested) use case of
>>>>>> rotating/scaling images. So if we want to solve those use cases  
>>>>>> we
>>>>>> need to either come up with a separate API for that, or extend  
>>>>>> this
>>>>>> proposal to solve that use case somehow.
>>>>>
>>>>> Just for reference I think one thing that people are forgetting  
>>>>> that
>>>>> there is a difference between
>>>>> being computationally faster, and being more responsive.
>>>>
>>>> As I mentioned in my email, if you look at the steps listed,  
>>>> enough of
>>>> them happen *on the main thread* that you're spending far more of  
>>>> the
>>>> main threads CPU cycles than you'd like. Possibly even more than  
>>>> doing
>>>> all the resizing on the main thread.
>>>>
>>>> With the other improvements suggested by David things do definitely
>>>> look different, but those are not in a proposal yet.
>>>
>>> There is the other scenario I mentioned, but I'll see what I can  
>>> do about
>>> separately working up a proposal for adding those methods because  
>>> they were
>>> next on my list to deal with. (fromBlob/load may be enough for  
>>> this.)
>>
>> Note that the other proposals that have been made has put toBlob on
>> HTMLCanvasElement, not on the context. That makes the most sense for
>> the main-thread canvas as that way its available on all contexts.
>
> Oh, another thing to keep in mind is that if/when we add fromBlob to
> the main-thread canvas, it has to be asynchronous in order to avoid
> main thread synchronous IO. This isn't a big deal, but I figured I
> should mention it while we're on the subject.

This is part of why I think Blob is the wrong tool for the job - we  
really want to use a data type here that can promise synchronous  
access to the data. When you copy the canvas backing store to a new in- 
memory representation, it seems silly to put that behind an interface  
that's the same as data to which you can only promise async access,  
such as part of a file on disk. There's nothing about copying bits  
from one canvas to another that needs to be async.

(Also it's not clear to me why a Blob would be faster to copy from,  
copy to, or copy cross-thread than ImageData; I thought the motivation  
for adding it was to have a binary container that can be uploaded to a  
server via XHR.)


> In general I wonder if we should add API to convert directly between
> Blob and ImageData. Or at least Blob->ImageData and
> ImageData->ByteArray. That could avoid overhead of going through a
> canvas context. That is probably a win no matter which thread we are
> on.
>
> We could even add APIs to rotate and scale ImageData objects directly.
> If those are asynchronous the implementation could easily implement
> them using a background thread. I'm less sure that this is worth it
> though given that you can implement this yourself using workers if we
> add the other stuff we've talked about.

Scaling and rotation can be done with just pixels if you code it by  
hand, but you can get native code to do it for you if you can  
manipulate actually offscreen buffers - you just establish the  
appropriate transform before painting the ImageData. REally the  
question is, how much slower is a scaling or rotating image paint than  
an image paint with the identity transform? Is it more than twice as  
expensive? That's the only way copying image data to a background  
thread will give you a responsiveness win. I'd like to see some data  
to establish that this is the case, if scales and rotates are the only  
concrete use cases we have in mind.

Regards,
Maciej







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20100312/b83ef1c8/attachment-0002.htm>