[whatwg] Bicubic filtering on context.drawImage

[K. Gadd]

Hi, the attached screenshots and test case in
https://bugzilla.mozilla.org/show_bug.cgi?id=782054 demonstrate how
the issue affects 2D games that perform scaling/rotation of bitmaps.
There are other scenarios I probably haven't considered as well. As
far as I can tell the mechanism used to render these quads is
rendering quads that are slightly too large (probably for coverage
purposes or to handle subpixel coordinates?) which results in
effectively drawing a rectangle larger than the input rectangle, so
you sample a bit outside of it and get noise when texture atlases are
in use.

Interestingly, I raised this on the list previously and it was pointed
out that Chrome's previous ('correct' for that test case) behavior was
actually incorrect, so it was changed. If I remember correctly there
are good reasons for this behavior when bilinear filtering is enabled,
but it's quite unexpected to basically get 'antialiasing'
on the edges of your bitmaps when filtering is explicitly disabled.
Getting opted into a different filter than the filter you expect could
probably be similarly problematic but I don't know of any direct
examples other than the gradient fill one.

A list of resampling methods defined by the spec would be a great
overengineered (not in a bad way) solution, but I think you really
only need to worry about breaking existing apps - so providing an
escape valve to demand bilinear (this is pretty straightforward,
everything can do bilinear) instead of the 'best' filtering being
offered is probably enough for future-proofing. It might be better to
default to bilinear and instead require canvas users to opt into
better filtering, in which case a list of available filters would
probably be preferred, since that lets the developer do feature
detection.

-kg

On Fri, Mar 21, 2014 at 9:38 PM, Rik Cabanier <cabanier at gmail.com> wrote:
> Hi Katelyn,
>
> would this solved by creating a list of resampling methods that are clearly
> defined in the spec?
> Do you have a list in mind?
>
>
> On Sat, Mar 15, 2014 at 4:14 AM, K. Gadd <kg at luminance.org> wrote:
>>
>> In game scenarios it is sometimes necessary to have explicit control
>> over the filtering mechanism used, too. My HTML5 ports of old games
>> all have severe rendering issues in every modern browser because of
>> changes they made to canvas semantics - using filtering when not
>> requested by the game, sampling outside of texture rectangles as a
>> result of filtering
>
>
> Can you give an example of when that sampling happens?
>
>>
>> , etc - imageSmoothingEnabled doesn't go far enough
>> here, and I am sure there are applications that would break if
>> bilinear was suddenly replaced with bicubic, or bicubic was replaced
>> with lanczos, or whatever. This matters since some applications may be
>> using getImageData to sample the result of a scaled drawImage and
>> changing the scaling algorithm can change the data they get.
>>
>> One example I can think of is that many games bilinear scale a tiny
>> (2-16 pixel wide) image to get a large, detailed gradient (since
>> bilinear cleanly interpolates the endpoints). If you swap to another
>> algorithm the gradient may end up no longer being linear, and the
>> results would change dramatically.
>>
>> On Fri, Mar 14, 2014 at 1:45 PM, Simon Sarris <sarris at acm.org> wrote:
>> > On Fri, Mar 14, 2014 at 2:40 PM, Justin Novosad <junov at google.com>
>> > wrote:
>> >>
>> >>
>> >> Yes, and if we fixed it to make it prettier, people would complain
>> >> about a
>> >> performance regression. It is impossible to make everyone happy right
>> >> now.
>> >> Would be nice to have some kind of speed versus quality hint.
>> >
>> >
>> > As a canvas/web author (not vendor) I agree with Justin. Quality is very
>> > important for some canvas apps (image viewers/editors), performance is
>> > very
>> > important for others (games).
>> >
>> > Canvas fills a lot of roles, and leaving a decision like that up to
>> > browsers where they are forced to pick one or the other in a utility
>> > dichotomy. I don't think it's a good thing to leave "debatable" choices
>> > up
>> > to browser vendors. It ought to be something solved at the spec level.
>> >
>> > Either that or end users/programmers need to get really lucky and hope
>> > all
>> > the browsers pick a similar method, because the alternative is a
>> > (admittedly soft) version of "This site/webapp best viewed in Netscape
>> > Navigator".
>> >
>> > Simon Sarris
>
>