[whatwg] I believe source rectangles for HTML5 Canvas drawImage are specified incorrectly

[Justin Novosad]

On Mon, Dec 17, 2012 at 10:33 AM, Kevin Gadd <kevin.gadd at gmail.com> wrote:

> A simple way to create an Image that represents a subregion of another
> Image or Canvas would be a nice solution here, since for
> implementations that need a temporary image anyway this lets them
> ensure it's only created once, and it lets you avoid the costs
> associated with temporary canvases. A few questions come to mind,
> though:
>
> Will it be possible to accept a canvas as the first argument instead
> of an Image?
>
That is a trickier case.  What happens when you draw to the canvas after
the creating the Image. Do the draws affect the Image?
If it is meant to behave like a snapshot, then pixels we need to be copied
so that they can persist.


> If you have a few live references to subregions of a larger image,
> will that prevent browsers like Chrome from discarding the decoded
> pixels?

I don't see that being a problem. A subimage could reference the image
resource the same way a regular image element does. You can already have
multiple image elements referencing the same resource (e.g. two image tags
with same "src")


>
> Will creating the subregion imply a copy and the associated garbage
> collection overhead? If so, API consumers would probably end up having
> to create some sort of caching library to keep cached copies of
> subregions around, and then those cached subregions might leak
> forever. I think what you'd want here is for it to be a reference to
> the subregion within the existing image, which means if it's a
> reference to a subregion of a canvas, when the canvas changes the
> subregion changes too. Would it break things for the semantics to work
> that way - i.e. it is now possible for the contents of an Image to
> change?
>
The contents of an image can change if you change the value of its "src"
attribute, which results in a heavy invalidation that triggers a re-layout
of the page.


> How would a new overload of the Image constructor be feature-detected in
> JS?
>
It would be possible for the subregion-ness of an Image element to be
expressed by attributes on the Image element (subregion x, y, w, h)
If we do it that way, one could test for attribute existence.


> If this becomes the correct way to solve this problem, what happens to
> existing implementations that provided alternative sampling behavior
> (like Chrome)? Will they get changed to match the spec, breaking apps
> until the new Image constructor is rolled out?
>

If an app only works in WK browsers, then there is already a problem with
it (and with the web platform).
Anyways, we can come up with ways to assure a smooth transition, but that
is a discussion for a different forum.


>
> Also, the more I think about it, the more the garbage collection
> impact of invoking a constructor for every blit seems like a potential
> problem. Even in generational GCs like V8, allocating a bunch of
> objects isn't cheap. Given that JS provides no way to hold weak
> references, it wouldn't be straightforward to cache and evict Image
> objects for each particular source rectangle used when drawing.
>

Right now, you have to store your sprite maps in globals to avoid this
problem, right?
I guess it would be the same for sprite (sub image) arrays.  I don't think
that is much worse that the status quo.


>
> -kg
>
> On Mon, Dec 17, 2012 at 7:23 AM, Justin Novosad <junov at chromium.org>
> wrote:
> >
> >
> > On Sun, Dec 16, 2012 at 11:52 PM, Rik Cabanier <cabanier at gmail.com>
> wrote:
> >>
> >>
> >>
> >> On Wed, Dec 12, 2012 at 10:24 AM, Justin Novosad <junov at chromium.org>
> >> wrote:
> >>>
> >>>
> >>>
> >>> On Wed, Dec 12, 2012 at 12:39 PM, Rik Cabanier <cabanier at gmail.com>
> >>> wrote:
> >>>>
> >>>>
> >>>> What would be the next step? Should we define a new version of
> drawImage
> >>>> with the extra parameter?
> >>>
> >>>
> >>> That is one option, but I think a context attribute (like
> >>> imageSmoothingEnabled) is also worth considering.
> >>> Perhaps image smoothing could be an enum rather than boolean with an
> >>> additional mode that prevents color bleeding.
> >>
> >>
> >> It seems a bit too expensive to add a variable to the graphics context
> >> that is checked for just this call.
> >> Maybe just a new drawImage call (like drawNonSmoothedImage?) is all that
> >> is needed.
> >
> >
> > Yes. That sounds quite reasonable to me, but we can find a better name.
> The
> > name "drawNonSmoothedImage" suggests that the image won't be smoothed at
> > all, which is not the case.  It's hard to find a name that correctly
> > describes the right behavior without getting too technical.  I am
> thinking
> > "drawSubImage", in the sense that the sub region delimited by the source
> > rectangle is treated as if it were a whole image.  This gives me another
> > idea: we could just have a new Image constructor that creates a new image
> > element that is a subregion of another:
> > var mySprite = new Image(spriteMap, x, y, w, h);
> > This can be implemented in a lightweight way that just references the
> data
> > of the source image.
> >
> >>
> >>
> >>>
> >>>
> >>>>
> >>>> If so, we probably want the description of the existing drawImage call
> >>>> to change so it leaves the resampling issue undefined unless WebKit
> is OK
> >>>> with changing their current behavior.
> >>>
> >>>
> >>> IMHO: Undifined behavior is a spec bug. If we have a problem with the
> >>> spec, we fix the spec, we don't just each do our own thing.
> >>>
> >>
> >
>
>
>
> --
> -kg
>