[whatwg] getImageData/putImageData comments

[Ian Hickson]

On Fri, 12 Jun 2009, Boris Zbarsky wrote:
> Ian Hickson wrote:
> > > 2)  The description of putImageData says it "Paints the data from the
> > >     given ImageData object onto the canvas".  It may be worth
> > >     specifying that this uses the SOURCE operator, though this is
> > >     clear later on when defining what the method _really_ does.
> > 
> > That's the author description, which is non-normative and trades precision
> > for accessibility, so I've not added this text. I recommend not reading
> > those sections as an implementor.
> 
> I was reading it as an author; as an author I assumed that if I'm painting
> data with an alpha < 1 I'll see things under it (which is not hthe case with
> PutImageData).

Ah, ok. Fixed.


> > The long and short of this is that if we solve this problem today, the
> > solution will be abused as much as the current API, and we'll have to
> > introduce yet another solution when high-res backing stores are common. So
> > instead I'm hoping that (a) authors won't screw this up, and (b) high-res
> > backing stores will be implemented sooner rather than later. If we fail with
> > (a), which is more likely if (b) is delayed, then we'll just introduce a
> > higher-res API later, and designate this one a lost cause.
> 
> OK.  This was sort of discussed in a follow-on thread to my mail, but my 
> last comment about there being two fundamentally different use cases for 
> PutImageData never got any responses... [...]
> 
> It was the first use I tried to put canvas to: visualizing something by 
> computing color values for pixels and then painting to the canvas (in my 
> case Julia sets).

Yeah, I guess if you want to generate low-res bitmap data using an 
array-like API, the current <canvas> doesn't really help you.


> > The putImageData() API is really intended for filters and generated 
> > patterns, both of which would want to be as high-res as possible.
> 
> People are using it for things that are not filters or generated 
> patterns as we speak...  And generally getting it wrong (in the sense 
> that their stuff will break on high-res displays).

Not much we can do about that yet.


On Sat, 13 Jun 2009, Boris Zbarsky wrote:
> > > It's not that hard, it's an extra four or five lines of code to fill 
> > > in multiple pixels in a square (two nested for-loops and an 
> > > expression or two to work out what the limit is). Compared to the 
> > > maths such code would be doing anyway, this is trivial stuff.
> > 
> > The hard part is realizing you have to do it...
> 
> And in particular, the fact that it's not just filling in multiple 
> pixels in a square...
> 
> For another example, consider an algorithm that wants to reduce the size of
> the image by 1px horizontally (e.g. content-aware image resizing as demoed
> using canvas at
> <http://labs.pimsworld.org/wp-content/uploads/2009/04/demo-content-aware-image-resizing-2/>).
> If we're shrinking by one device pixel, then suddenly we can't even create a
> canvas of the right size for the new image, because canvas size is specified
> in integer CSS pixels.  Note that the algorithm used here, basically
> off-the-shelf, would in fact want to work on device pixels the way the spec is
> written right now; in general it would want to work on whatever pixels
> imagedata is in.

If you want to shrink an image, redraw it using drawImage() with different 
dimensions.


> If we want to shrink by one CSS pixel, so we can actually draw the 
> canvas, then each shrink step needs to apply the algorithm N times, 
> right?  But what if N is, say, 2.5, as it can be in Safari?  It's been 
> pointed out before that 1 css pixel can map to a fractional number of 
> device pixels there...  That algorithm can't very well be applied 2.5 
> times.  Of course my Julia set example would also need to do something a 
> lot more interesting than just "fill in multiple pixels in a square" 
> when the css-to-device pixel ratio is non-integral.
> 
> Note that I don't have a proposed solution here; I'm just pointing out 
> that the authoring pitfalls here are a lot more complicated than you 
> seem to give them credit for.

Yeah, I can see that there are cases that are a lot more complex than it 
would first seem.

I don't know what we can do about it either, though.


On Sat, 13 Jun 2009, Robert O'Callahan wrote:
> On Sat, Jun 13, 2009 at 6:57 PM, Ian Hickson <ian at hixie.ch> wrote:
> > 
> > There's no practical difference as far as I can tell between hoping 
> > that we can reuse the API, and then finding we can't, and introducing 
> > a second API for high-res screens; and just giving up now and saying 
> > that it's a low-res API, and then adding a second API later when we 
> > have high-res screens. The effect is more or less the same. Given 
> > that, I think it's worth at least trying to see if we get away with it 
> > and can in fact use this API in high-res situations later.
> 
> The difference to me as an implementor is that as the spec stands, I have to
> choose between
> 1) Implement high-resolution canvas backing store and implementing image
> data per spec, breaking most of the current scripts that are using
> image-data for all the users who can actually take advantage of that
> high-resolution backing store
> 2) Implement high-resolution canvas backing store and quietly violate the
> spec so that the current generation of image-data-using scripts continue to
> work
> 3) Don't implement high-resolution backing store, which at least means I
> don't have to choose between violating the spec and breaking content

Do 1. The earlier the better, because if we do it now we can still break 
pages without huge problems, and if we do, we might get away with using 
this API without having to introduce a new one later.

So please, if you are at all able to, do 1, as soon as possible.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'