[whatwg] proposed canvas 2d API additions

[Vladimir Vukicevic]

On 5/5/06, Ian Hickson <ian at hixie.ch> wrote:
> On Fri, 28 Apr 2006, Vladimir Vukicevic wrote:
> > ImageData createImageData(in string format, in string width, in string
> > height, in Array data);
> >
> > for creating "ImageData" out of an arbitrary set of generated data (e.g.
> > evaluating some function, drawing the results).  This would be
> > especially needed because you can't assign to data in an ImageData
> > structure (since you have it readonly); can only change the values of
> > its members.
>
> You could always create an arbitrarily sized ImageData structure by
> creating a blank <canvas> and using getPixels on that canvas. This would
> let you initialise the array to white, transparent black, or whatever is
> necessary.

You could, but that would involve a lot more overhead than you might
want.  Then again, you could just call getPixels on whatever canvas
you have and use that structure.  The spec needs to say (if it doesn't
already, I haven't read what you've added yet) that getPixels returns
a static snapshot of the canvas contents, and isn't modified by any
further drawing to the canvas (until getPixels is called again, which
returns a new structure).

> > >    ImageData getImageData(in float x, in float y, in float w, in float h);
> > >    void drawImageData(in float x, in float y, in ImageData d);
> >
> > I would keep x/y/w/h as integers, and explicitly specify that they're
> > not affected by the CTM.  If they are, you can't guarantee a lossless
> > round-trip (since if you shift over by half a pixel you have to do lots
> > of resampling, etc.).
>
> I agree that they shouldn't be affected by the CTM, but I disagree that
> they should be integers. e.g. in cases like:
>
>    HTML                          CSS
>    <canvas height=1 width=1>     canvas { height: 100%; width: 100%; }
>    </canvas>
>
> ...where the JS then uses the coordinate space 0..1,0..1 the author might
> want to grab the top corner by grabbing the 0,0,0.25,0.25 rect.

So, I really don't like this -- we need to nail down space the
getPixels/setPixels coordinates should be in.  I still think that they
should always be in the canvas space, no matter how many pixels they
refer to in the rendered content or in the device space.  Note that in
your example, the canvas can still be a 1x1 pixel canvas (and, I
believe, will be in all current implementations) -- that one pixel
will just cover the entire page.  There's nothing that says that the
canvas resolution gets increased by rendering it into a
bigger-than-1:1-pixel-mapping frame.  Like I said a few messages ago..
there are three coordinate spaces in play here:

<display space> (CSS, variable) --- <device space> (N pixels per 1
canvas pixel; canvas backing store) --- <canvas space> (base "1 unit",
which is normally a pixel, which is what the canvas 2d context deals
in)

display space is affected by CSS, and starts out the same size as the
pixel region covered by the canvas device space.  We should ignore
display space in all the canvas APIs, and only consider the canvas
space, and if necessary, the device space.  getPixels/setPixels are
one place where we need to consider the device space in the returned
data, but I don't think that the API should operate in device space
and especially not in display space.  The canvas space is one that the
author directly knows the bounds of (because they specified it
explicitly in the canvas width/height attributes), and can therefore
decide exactly which portion of the canvas to pull out via getPixels. 
If it was in any other space, you would either have to deal with
normalized floats (0.0 .. 1.0 range -- which would be somewhat
imprecise; if I have a 3x3 canvas, how do I pull out the 1x3 region in
the upper left?), or you'd have to have a way to figure out the
scaling factor that I mentioned before calling getPixels or
(setPixels).

> > Also... should the RGBA data be returned with premultiplied alpha or
> > not?  Premultiplied tends to be better for the math, non-premultiplied
> > tends to be easier to understand.  (That is, 50% opaque green is
> > (0,255,0,128) if non-premultiplied, or (0,128,0,128) if premultiplied.)
>
> For consistency with the rest of the API I'd say non-premultiplied, but I
> have no strong opinion on this.

Ok, non-premultiplied it is.  This may cause some potentially
unexpected data loss -- that is, something that is fully transparent
will always come back as fully transparent black, because if the
backing store is premultiplied all the color informaiton is lost --
but as long as that's mentioned in the spec it should be fine.

    - Vlad