[whatwg] effect of smoothing on drawImage (was: Bicubic filtering on context.drawImage)

[Rik Cabanier]

I created a test case that rotates your test image:
http://jsfiddle.net/6vh5q/9/
According to the spec [1] the red line should show up when you rotate the
image, regardless of smoothing.

Chrome/Firefox with cairo or core graphics backend: no red line
IE/Firefox with d2d backend: checkered red when smoothing is off, solid
with smoothing on (= this is correct)
Safari: checkered red with smoothing off but no red with smoothing on (?)

It might be surprising to an author that there's content from outside the
source rect "leaks" in when smoothing is turned off.
I'm unsure if it's important enough to address in the spec.

On Sat, Mar 22, 2014 at 9:43 PM, K. Gadd <kg at luminance.org> wrote:

> On windows with smoothing disabled I get it in every browser except
> Chrome. Maybe this is due to Direct2D, and for some reason it
> antialiases the edges of the bitmaps? It's nice to hear that it
> doesn't happen on other configurations.
>
> It's important to test this when drawing with transforms active because
> you may not see the full set of artifacts without them. (The application
> where I first observed this is rotating/scaling these bitmaps with
> scaling disabled.)
>
> Copying to a temporary canvas is an interesting idea; is it possible for
> typical browser implementations to optimize this or does it forcibly
> degrade things to a pair of individual draw calls (with full state changes
> and 6-vertex buffers) for every bitmap rendered?
>
> I don't really have any problems with the behavior when smoothing is
> enabled; sorry if this was unclear.
>
> -kg
>
> On Sat, Mar 22, 2014 at 9:09 PM, Rik Cabanier <cabanier at gmail.com> wrote:
> >
> > On Fri, Mar 21, 2014 at 10:47 PM, K. Gadd <kg at luminance.org> wrote:
> >>
> >> Hi, the attached screenshots and test case in
> >> https://bugzilla.mozilla.org/show_bug.cgi?id=782054 demonstrate how
> >> the issue affects 2D games that perform scaling/rotation of bitmaps.
> >> There are other scenarios I probably haven't considered as well. As
> >> far as I can tell the mechanism used to render these quads is
> >> rendering quads that are slightly too large (probably for coverage
> >> purposes or to handle subpixel coordinates?) which results in
> >> effectively drawing a rectangle larger than the input rectangle, so
> >> you sample a bit outside of it and get noise when texture atlases are
> >> in use.
> >>
> >> Interestingly, I raised this on the list previously and it was pointed
> >> out that Chrome's previous ('correct' for that test case) behavior was
> >> actually incorrect, so it was changed. If I remember correctly there
> >> are good reasons for this behavior when bilinear filtering is enabled,
> >> but it's quite unexpected to basically get 'antialiasing'
> >> on the edges of your bitmaps when filtering is explicitly disabled.
> >> Getting opted into a different filter than the filter you expect could
> >> probably be similarly problematic but I don't know of any direct
> >> examples other than the gradient fill one.
> >
> >
> > ah. I remember looking at your test case.
> > I made a simpler version that shows the issue:
> > http://codepen.io/adobe/pen/jIzbv
> > According to the spec, there should be a faint line [1]:
> >
> > If the original image data is a bitmap image, the value painted at a
> point
> > in the destination rectangle is computed by filtering the original image
> > data. The user agent may use any filtering algorithm (for example
> bilinear
> > interpolation or nearest-neighbor). When the filtering algorithm
> requires a
> > pixel value from outside the original image data, it must instead use the
> > value from the nearest edge pixel. (That is, the filter uses
> 'clamp-to-edge'
> > behavior.) When the filtering algorithm requires a pixel value from
> outside
> > the source rectangle but inside the original image data, then the value
> from
> > the original image data must be used.
> >
> > You were told correctly that the Chrome behavior is incorrect. When doing
> > smoothing, Chrome is not looking outside the bounds of the source image,
> so
> > you don't get the faint line.
> > This is also an issue with the cairo and core graphics backends of
> Firefox.
> > Safari and IE seem to work correctly.
> >
> > I will log bugs against Chrome and Firefox so we can get interoperable
> > behavior here.
> >
> > I was not able to reproduce the issue of smoothing was disabled. If you
> want
> > smoothing but not the lines, you can do a drawimage to an intermediate
> > canvas with the same resolution as the source canvas. I verified that
> this
> > works.
>

1:
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-drawimage