[whatwg] Canvas 2D context proposal: point/linear filtering modes

[Tab Atkins Jr.]

On Fri, Oct 28, 2011 at 12:26 PM, Ashley Gullen <ashley at scirra.com> wrote:
> Hi WHATWG,
>
> I'm running a startup heavily invested in HTML5 2D game development (see
> www.scirra.com).  I apologise if what I'm about to mention has been
> discussed before, but I haven't been involved before now.
>
> The retro (pixellated style) is still very popular in 2D gaming, and many
> new games are still being made with this style.  (Have a quick scroll
> through http://www.tigsource.com/ and notice how many retro games are still
> hitting headlines.)  I've noticed most browsers today are using bilinear
> filtering for scaling.  Compared to point filtering (aka nearest neighbour)
> this makes retro style games look blurry.  See here:
> http://www.scirra.com/images/point-vs-linear.png
>
> Bilinear filtering does in fact result in a better quality display for many
> modern games.  However, it's unsuitable for retro style games as the image
> demonstrates: seeing an entire game look that blurry is likely to make the
> game unplayable.  So I believe there's a real need for both, but the
> standard leaves this as an implementation detail: "The user agent may use
> any filtering algorithm (for example bilinear interpolation or
> nearest-neighbor)."  I propose that the standard is amended so both modern
> and retro style games can use the most appropriate sampling algorithm.
>
> For example:
> context.globalFilter = "point"; // set point filtering (nearest-neighbour)
> context.globalFilter = "bilinear"; // set bilinear filtering
>
> Some questions I anticipate:
>
> "Won't this slow down software renderers?"
> - Bilinear filtering can seriously impact performance in software rendered
> implementations of the canvas 2D context.  Generally bilinear is only
> supported on hardware accelerated implementations.  I think the best
> solution is to allow implementors to choose the default, and not require
> support for bilinear filtering (although it should be recommended).  Modern
> games using point filtering suffer a small degradation in visual quality,
> but retro games using bilinear filtering suffer a massive degradation in
> visual quality.  So bilinear filtering could be optional, and optionally be
> set as the default, but the ability to set point filtering should be
> required.
>
> "Why not just scale the artwork yourself with nearest-neighbour and use that
> in the game?"
> - HTML5 games have to run on a variety of screen sizes, ranging from
> handsets to HD displays.  A good solution to this is to use the scale()
> method to fit to the window size.  However, this scaling is done with
> bilinear filtering in all current major browsers.  Retro artwork drawn large
> and scaled down looks blurry, and artwork drawn small and scaled large looks
> blurry too.  Also, rotating, scaling and sub-pixel positioning individual
> objects can still make artwork look blurry, drawn at any size.
>
> "Why not use WebGL?"
> - The WebGL API is much more complex than the Canvas 2D API.  To make a
> retro style 2D game, Canvas 2D is the obvious choice, and allows a much
> quicker rate of development.  This also avoids the issue of driver support,
> hardware support and browser support associated with WebGL.
>
> Overall, I think this proposal is very simple, straightforward to
> standardise and implement, genuinely useful, and would significantly
> encourage 2D gaming in HTML5 for comparitively little effort.  Is it
> possible that this could be included in the standard?  Any
> comments/suggestions/vendor-specific extensions I've missed?

The CSS Working Group is interested in this topic more generally for
all images or videos.  There was previously a property in the CSS3
Image Values spec for this; it's currently punted to the level 4 spec,
but you can find it in an older WD like
<http://www.w3.org/TR/2011/WD-css3-images-20110712/#image-rendering>.

Currently there are two values defined: 'auto' means "do what you
want, preserving image quality", which will generally make pixel art
blurry; and 'crisp-edges' which means "do what you want, but don't
blur edges or colors", which is expected to often use
nearest-neighbor, but is allowed to use more complex pixel-art scaling
algorithms.  There's been a persistent request to explicitly ask for
nearest-neighbor, so I expect that will be addressed at some point
with a value named 'pixelated' or something.

Once this property is implemented, browsers should respect it for
scaling <canvas>, so you won't need anything explicit in the API.

~TJ