[whatwg] Canvas and imageData

[Ian Hickson]

On Sat, 12 May 2007, Anne van Kesteren wrote:
>
> These features are nice but I don't think authors will understand that 
> imagedata.height != canvas.height (likewise for width). Authors will 
> just make something that works in their browser and then assume it will 
> work everywhere else. Which would horribly break. Even the more 
> experienced <canvas> developers playing with these features have already 
> made these mistakes.

On Sat, 12 May 2007, Philip Taylor wrote:
> 
> Looking for people using getImageData already, I find:
> 
> http://canvaspaint.org/paint.js - incorrect (though commented out)
> 
> http://svn.sourceforge.net/viewvc/jsmsx/trunk/msx.js?view=markup - incorrect
> 
> http://www.thescripts.com/forum/thread521112.html - incorrect
> 
> http://tech.groups.yahoo.com/group/canvas-developers/files/buttons.html
> (or maybe
> <http://f1.grp.yahoofs.com/v1/gORFRoOLlxuXtJddXwdSyravD-aFfgNuYoSzjI8vUevuBxus3V1sXs5xckiHKd1osiUpDE_bku-vtGMFPV_M-2JZkLKXTqc/buttons.html>)
> - correct
> 
> I couldn't find any others doing anything with the 
> ImageData.width/height, but three of those four will break when somebody 
> implements getImageData with imagedata.width != canvas.width.

That's sad. I'm not sure what to do about it though.


On Sat, 12 May 2007, Philip Taylor wrote:
>
> I think it would make much more sense (and be much less error-prone) if 
> <canvas> acted like a dynamic bitmap <img> and the browser just scaled 
> up the bitmap before rendering to the screen, rather than optionally 
> using a higher-resolution buffer for all the intermediate drawing. That 
> matches the behaviour when you change the <canvas>'s size with CSS - 
> "... the element can be sized arbitrarily by a style sheet. During 
> rendering, the image is scaled to fit this layout size."
> 
> It would reduce the quality of the output to some extent

It would in fact reduce the quality to be essentially worthless on 
high-res displays when people want to do decent quality graphics.


> For people who really do want high quality output and know what they're 
> doing, there could be some way to query the current canvas->display 
> scaling factor (taking account of high-res displays, zoomed pages, CSS 
> scaling, etc) and they can manually change the canvas's width/height to 
> get a 1:1 match between device pixels and canvas pixels. Then they can 
> even update the size of the bitmap every frame, so it continues looking 
> good if the user has zoomed in on the page. Or they could use SVG.

I guess that's one way.


On Sat, 12 May 2007, Anne van Kesteren wrote:
> 
> The best way forward would probably be to get rid of "<canvas> device 
> pixels" and just make such a pixel map a <canvas> pixel. This means that 
> getImageData() and putImageData() work similarly to toDataURL() and 
> drawImage(). If people really want high resolution <canvas> they could 
> just make it high resolution and scale it down for display using CSS. 
> Incidentally such an approach might get us rid of all the floats 
> currently in the API...

That approach would make existing canvases shrink to a tiny dot when 
printing.


On Sun, 13 May 2007, Anne van Kesteren wrote:
>
> If this is indeed to be allowed (and it seems to work fine in Firefox) 
> this is an additional argument for not having a distinction between the 
> <canvas> "back end" grid and the actual <canvas> grid as people will 
> just assume they map one to one once it works that way in an 
> implementation they test in.

Hm.


On Sun, 13 May 2007, Anne van Kesteren wrote:
> 
> Oops. I was looking at the May 10 version of the specification. My 
> apologies. It would probably still be good to point out that you can 
> create ImageData objects yourself by making a simple ECMAScript object.

Isn't this obvious? I mean, JS doesn't have typing, and so any interface 
that JS can represent is implementable in JS...


On Sun, 13 May 2007, Maciej Stachowiak wrote:
> >
> > (Given that you can create them yourself I'm not sure why ImageData 
> > has readonly attributes, but maybe that would save some additional 
> > checking...)
> 
> Ironically, due to the readonly attributes you couldn't just use a 
> vanilla JS object as the implementation, even though you have to accept 
> is as a value.

I can make them non-readonly if people want.


> Also, if it's meant to be required for implementations to allow that, 
> the spec should say so - it's not normally assumed that any JS object 
> with the right properties can be used anywhere that an interface can be 
> used.

Isn't it?


On Mon, 14 May 2007, Anne van Kesteren wrote:
> 
> Just to keep this list in sync with #whatwg. It was suggested to give 
> both putImageData and getImageData a "high resolution" boolean parameter 
> which would indicate what type of ImageData object you would get back. 
> This would be fine by me. I'm not sure if that's needed right away 
> though.

That's another option, indeed. We could address this when someone 
implements a high-res <canvas>.


> > Also, if it's meant to be required for implementations to allow that, 
> > the spec should say so - it's not normally assumed that any JS object 
> > with the right properties can be used anywhere that an interface can 
> > be used.
>
> Agreed.

I've added an example.


On Mon, 14 May 2007, Philip Taylor wrote:
> 
> Looking at all the places I can see getImageData/putImageData being
> used, there appears to be four main categories:
> 
> Reading a single pixel:
>  http://canvaspaint.org/paint.js
>  http://www.sweetvision.com/2007/03/25/enhanceing-the-javascript-color-picker-with-the-canvas-element/
>  http://canvex.lazyilluminati.com/tests/tests/
> 
> Drawing pixels:
>  http://svn.sourceforge.net/viewvc/jsmsx/trunk/msx.js?view=markup
>  http://www.thescripts.com/forum/thread521112.html
> 
> Image filtering:
>  http://oxine.opera.com/gallery/canvas/2dgame/sepia.html [not
> actually ImageData, but close]
>  http://tech.groups.yahoo.com/group/canvas-developers/files/buttons.html
> 
> Copying images:
>  http://groups.google.com/group/mozilla.dev.apps.firefox/browse_thread/thread/742111eaf2e3e0da/b8777a855fcfa32b
>  http://forums.mozillazine.org/viewtopic.php?=&p=2759433
> 
> For reading a single pixel, any form of getImageData should be fine - 
> perfect accuracy isn't expected, and it just needs to correspond to 
> nearly what the user thinks they're pointing at.
> 
> For drawing pixels, you could use fillRect(x, y, 1, 1) but it's 
> impractically slow - http://canvex.lazyilluminati.com/misc/filter.html 
> in Firefox goes over 50 times slower with fillRect vs putImageData, so 
> something like putImageData is very useful. For cases like jsMSX, which 
> wants to draw to individual <canvas> pixels using get/putImageData, the 
> ImageData needs to contain canvas pixels so the JS code doesn't have to 
> do complex ImageData<->canvas pixel mapping itself.
> 
> For copying images, drawImage should be used unless you need to copy
> outside the browser (e.g. to a server or local storage), and toDataURL
> should be used if you do want to (because ImageData is horribly
> space-inefficient), so I don't think these cases are relevant to
> ImageData at all. (I'm not thinking about toDataURL for now.)
> 
> For image filtering, you probably do want perfect accuracy/resolution to 
> get the best possible output. In some cases (like blurring) you also 
> want to know the mapping between ImageData pixels and <canvas> pixels, 
> so that the effect of the filter can be independent of the resolution. 
> Particularly in those cases, you have to be quite careful to get it 
> right if you're only ever testing in browsers with a 1:1 pixel mapping, 
> though it's not that hard once you understand the issues and if you can 
> find someone to test and report bugs.

The use cases it was intended for were the first and last. I'm not really 
sure I understand the second use case.


> So, I was considering the function
>    getImageData(sx, sy, sw, sh, hires)

I think that's an interesting idea, but I think what we should do is wait 
for someone to implement a high res canvas, and if stuff breaks, then we 
retroactively redefine getImageData to return low res data, and then add 
the high-res flag.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'