[whatwg] More random comments on the putImageData definition

[Oliver Hunt]

So I came across this wonderful piece of javascript:
http://jsmsxdemo.googlepages.com/jsmsx.html

If present it uses putImageData (and getImageData to get the ImageData  
object, which isn't required by html5 now -- and i think hixie was  
look at doing some more work with ImageData).

With the current model for putImageData there is no way for them to  
specify a dirty rect, which means their only option is to update the  
entire display -- they have to copy the entire buffer.  If  
putImageData were to take an optional dirty rect parameter we could  
reduce the amount of work necessary, and still maintain backwards  
compatibility.  My attention was drawn to this as a brief bit of  
hackery in webkit showed that the above webapp only updates the  
necessary dirty parts of a frame and yet Opera and Firefox both *have*  
to copy, blit and repaint the entire canvas, ironically because they  
both provide the faster get/putImageData API's (this is not to suggest  
either implementation is in any way slow, just that it seems we are  
missing a perfectly reasonable API to improve performance beyond what  
has already been achieved).

Anyway, i was thinking we would just need to putDataImage methods:
void putImageData(in ImageData imagedata, in float dx, in float dy);
void putImageData(in ImageData imagedata, in float dx, in float dy, in  
float dirtyX, in float dirtyY, in float dirtyWidth, in float  
dirtyHeight);

Where the dirtyX and dirtyY are relative to the ImageData's origin.

The repaint region would be (dx+dirtyX, dy+dirtyY),  
dirtyWidth*dirtyHeight in the canvas domain, and (more importantly)  
only that subsection of the ImageData would need to be copied, and in  
the case of those UAs that need it, unpremultiplied.  This could  
result in significant gains for more complex uses of canvas, like the  
one above.

Compatibility:
If the dirty rect is provided to a UA that doesn't support it (eg. new  
content in old/current Opera, Firefox) the entire ImageData will be  
blitted as would currently be expected (opera and firefox both ignore  
extraneous arguments on putImageData, i'm not sure about konqueror/ 
khtml).

If a dirty rect is not provided (eg. old content in a newer UA) the  
entire block would be considered dirty, and so repaint as expected.

Any thoughts?

--Oliver

On Jan 23, 2008, at 11:28 AM, Oliver Hunt wrote:

>
> On 23/01/2008, at 5:44 AM, Philip Taylor wrote:
>
>> On 23/01/2008, Oliver Hunt <oliver at apple.com> wrote:
>>> It would be great if putImageData
>>> could take a source region, in addition to the destination.  One of
>>> the primary reasons for using get/putImageData is to allow JS to
>>> rapidly blit data to the screen, however without an ability to blit
>>> only a subregion of the image data the only available options are to
>>> either re-blit the entire imagedata region (which can be expensive  
>>> due
>>> to the need for [un]premultiplying in some (all?) implementations),
>>
>> ((Opera does non-premultiplied colour internally.))
> Righto.  There's still the necessary type/range checking involved at  
> some point.
>
>>
>>> or create and populate a new ImageData object which still requires  
>>> more
>>> work than would ideally be necessary.
>>
>> You can also create a temporary canvas and putImageData once onto
>> that, and then drawImage sections onto the screen as they are needed.
>> That lets you draw lots of sections lots of times quickly (since
>> you're mostly drawing from the optimised canvas surface format, not
>> from a JS array), which perhaps helps in some (most?) of the cases.
>> (You still have to do a single putImageData of the whole data to get
>> it onto the temporary canvas, but if there are parts of the data you
>> aren't ever using then you just should make the ImageData smaller and
>> cut out the unused bits.)
>
> Yes, there are many ways you can resolve this if you're willing to  
> copy data
> around in JS, which is far less efficient than letting you use your  
> single backing
> buffer, but only (effectively) repainting part of it.
>
> Using a separate canvas also works, but still requires additional  
> copying, much more memory, and the use of drawImage which doesn't  
> have the same semantics as putImageData.
>
> --Oliver
>
>>
>> -- 
>> Philip Taylor
>> excors at gmail.com
>