[whatwg] Enabling LCD Text and antialiasing in canvas

[Ian Hickson]

On Thu, 29 Mar 2012, Jeremy Apthorp wrote:
> On Thu, Mar 29, 2012 at 10:25 AM, Jeremy Apthorp <jeremya at chromium.org>wrote:
> > On Thu, Mar 29, 2012 at 8:41 AM, Ian Hickson <ian at hixie.ch> wrote:
> >> On Fri, 13 Jan 2012, Jeremy Apthorp wrote:
> >> >
> >> > I'd like to draw non-antialiased lines in a <canvas>. Currently it 
> >> > seems that the only way to do this is to directly access the pixel 
> >> > data.
> >> >
> >> > Is there a reason there's no way to turn off antialiasing?
> >>
> >> What's the use case?
> >
> > Pixel-art style games.
> 
> Specifically: even with the new image smoothing stuff in place for 
> drawImage, a 1:2 diagonal line will still be anti-aliased (only the 
> antialiasing will look silly scaled up to 2x).

Do you have an example of a game where lines are drawn using a line API 
without antialiasing, then scaled up? Most "pixel art" games I've seen 
tend to use bitmaps for that kind of thing.


On Mon, 12 Nov 2012, Justin Novosad wrote:
> 
> For many types of apps, DOM-based rendering is uncompetitively slow
> [so we should make text rendering in canvas more controllable]

This seems like something we should fix, not something we should work 
around by having people use canvas instead. Using canvas has all kinds of 
terrible side-effects, like reducing the likely accessibility of the page, 
making searcheability much worse, etc.

Also, do you have any metrics showing the magnitude of this problem on 
real-world sites that might consider using canvas instead?

 
> If LCD text were enable-able, authors would have to be mindful of a 
> number of caveats in order to avoid rendering artifacts.

Do we have any reason to believe the majority of authors would make the 
right decisions here?

(The main reason we haven't provided control over things like antialiasing 
is that many authors tend to make terribly bad decisions.) (Before anyone 
gets offended, by the way: that you are reading this almost guarantees 
that you are above average in terms of authoring ability.)


On Tue, Nov 13, 2012 at 9:37 PM, Robert O'Callahan wrote:
>
> We'd have to define what happens when you use subpixel antialiasing 
> "incorrectly", because we can be pretty sure authors will use it 
> incorrectly and expect to get interoperable behavior.

That's certainly true.


> Mozilla supports a "mozOpaque" attribute which makes the canvas buffer 
> RGBX (initialized to solid black) and enables subpixel antialiasing for 
> most text drawing. That might be enough to address your use-cases.

I haven't specified this; if other vendors intend to implement this let me 
know and I can spec it. I'm not sure it's worth it though.


On Wed, 14 Nov 2012, Robert O'Callahan wrote:
> On Wed, Nov 14, 2012 at 8:09 AM, Justin Novosad <junov at chromium.org> wrote:
> > 
> > Are there precedents for exposing features with documented caveats? 
> > (excluding caveats that were discovered after the fact)
> 
> Yes, and many of them have been extremely problematic, because Web 
> authors will ignore the caveats.

Right. I'd really like to avoid adding more if we can help it.


On Wed, 14 Nov 2012, Justin Novosad wrote:
> 
> There is a recent improvement in Chrome called "deferred 2D canvas 
> rendering" (enabled by default as of Chrome 23).  It is a mechanism that 
> records 2d canvas commands during JS execution, and only executes them 
> for real when the render buffer needs to be resolved (draw to screen, 
> getImageData, toDataURL, etc.).  If you want to check it out, the guts 
> are in Skia: SkGPipe is a sort of FIFO for graphics commands, 
> SkDeferredCanvas is a wrapper that manages the GPipe and automatically 
> flushes it and applies some command culling optimizations.
> 
> So to come back to the problem of with and without subpixel AA buffers: 
> if rendering is deferred, the non-AA buffer would never get rasterized 
> (and possibly never even allocated), unless it needs to be.  Obviously 
> there are practical limitations, for example we cannot store an 
> unlimited stream of recorded commands, so if the canvas draws 
> indefinitely without ever being cleared, at some point we have to 
> rasterize the non-AA buffer just so that we can safely discard the 
> recording data. Also, if at record time the necessary conditions for 
> subpixel AA are not met, perhaps we just forget about it.
> 
> I admit this is a complex solution for implementors, but it makes the 
> management of subpixel-AA safety transparent to web authors.

I think it'd be reasonable (for some definition of reasonable that 
relates to whether it's compatible with the spec, anyway) for implementors 
to do this today, without having to expose any control to the author.


On Thu, 15 Nov 2012, Fred Andrews wrote:
> 
> The canvas that scripts draw into could be over-sized with the UA down 
> sampling this to fit the target size and taking into account the 
> sub-pixel screen layout when doing so.

On Thu, 15 Nov 2012, Justin Novosad wrote:
>
> Obviously, that would be costly (x3 pixels), but I think it is a very 
> realistic solution and relatively low hanging fruit. The over-sizing of 
> the canvas would have to be handled under the hood by the UA though, 
> because it depends on LCD component ordering and orientation, which 
> means querying the OS/display driver. A lot of the kinks with the 
> over-sized canvas approach have already been ironed out for solving the 
> problem of High-DPI support ( put/getImageDataHD ), and I like the idea 
> of unifying the two. Implementing this would mostly be a matter of 
> adding per color component compositing of canvases. Also, the pixel 
> aspect ratio would have to be taken into account for line drawing.

getImageDataHD() requires that the pixels be square, but so long as that 
is taken into consideration (e.g. by dropping down to square pixels if the 
author calls putImageDataHD()) I think this could probably be made to work 
within the spec's current requirements.


> Regarding the concerns about accessibility, I think the problem can be 
> solved by using HitRegions with labels.

Oh it _can_ be solved. That's not the problem. Accessibility is not about 
what is _possible_, it's about what actually _is_. Most authors, 
realistically speaking, aren't goin to be using hit regions sufficiently 
for us to declare victory here.


> Come to think of it, there should be an option to make the UA do this 
> automatically: create a HitRegion with a label every time text is drawn 
> to a canvas.

I considered doing that, but it gets really fiddly when you're doing 
things like text that fades over multiple frames. In the end I decided 
that the magic wasn't worth it, as it would likely screw up more often 
than it would actually help.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'