[whatwg] High-density canvases

[Ian Hickson]

On Mon, 9 Sep 2013, Rik Cabanier wrote:
> On Mon, Sep 9, 2013 at 5:00 PM, Ian Hickson <ian at hixie.ch> wrote:
> >
> > So my understanding is that the reason this feature failed is that 
> > there's existing content that assumes a 1:1 ratio, and having an 
> > automatic high-density mode was making some pages end up with canvases 
> > with four canvas pixels per CSS pixel (linearly) -- two from the 
> > browser making a native canvas, times two from the page scaling the 
> > canvas for high DPI displays. This is a factor of sixteen over a 1:1 
> > canvas, a factor of four more than it should be for high DPI, and a 
> > big waste of resources.
> >
> > As much as sites do this manually, though, it's a huge pain in the 
> > neck to have to worry about pixel density when you're creating your 
> > canvas and drawing on it, especially if you're not drawing sprites on 
> > it.
> >
> > While we're talking about annoying things, there's also the annoyance 
> > that canvases tend to not take zoom into account (either 
> > density-affecting zoom like page zoom on desktop, or "transparent" 
> > zoom like pinch-zoom on mobile for non-mobile-optimised sites, which 
> > the site isn't supposed to know about): you have to remember to listen 
> > for onresize, and then manually blow away your canvas and recreate it 
> > at the right density and then squeeze it into place so that the 
> > coordinate space matches what your code is expecting while the 
> > <canvas> is actually sized for the display.
> >
> > There's also the issue of full-bleed canvases where every time the 
> > container changes, you have to remember to re-update the canvas 
> > coordinate space and repaint because otherwise your pretty page gets 
> > all warped.
> >
> > It would be nice to fix these all at once, and I think we can, by 
> > introducing a configuration option on getContext(), in the style of 
> > WebGL:
> >
> >    getContext('2d', { density: 'autosize' });
> >
> > This would trigger the following behaviour: When the context is 
> > created, and subsequently when the <canvas> changes size (e.g. due to 
> > being sized with CSS relative units and the element they're relative 
> > to changing), or when the display density changes size (e.g. due to 
> > page zoom), then:
> >
> >    - the width and height of the canvas bitmaps get updated to match the
> >      new native size of the <canvas>, at native density.
> >
> >    - the coordinate space of the canvas (context.width/context.height)
> >      gets updated to match the size of the <canvas> in CSS pixel units.
> >
> >    - a 'resize' event gets fired at the <canvas>.
> 
> Will this workflow introduce flickering?

So long as the resize event (and thus the redrawing) is synchronous with 
the clearing, I don't see why it would.


> I'm unsure if these calls are really needed. An author can already do 
> this today with a couple of lines of JS.

It's a heck of a lot more than a couple of lines to do it right today.


> By doing the actual resizing in the event handler, I believe flickering 
> is avoided.

Not sure what you mean here.


> I share Dean's worry that it will lead to bad redrawing if people don't 
> test all scenarios that causes resizing.

Can you be more specific? What scenarios do you think would result in bad 
redrawing?


> (Temporary canvases would also have to be resized manually)

We'd have to provide a way to link canvases so that they maintain the same 
density. It's on my list of things to make sure I handle if I spec this.


On Tue, 10 Sep 2013, Stephen White wrote:
>
> For posterity, here were our objections to the original high-DPI canvas 
> spec:
> 
>    - The API feels like a short-term hack to automagically do something
>    that the developer may or may not want done (e.g., if the game/app was
>    tuned for particular resolution, or for pixel-exact rendering) that we'll
>    be stuck with in the web platform long after its short-term usefulness has
>    expired

This is avoided in the new proposal by making it opt-in.


>    - It doesn't scale well to non-integer devicePixelRatios

Can you elaborate on this? I don't see why the new proposal would have 
this problem, but I also don't see why the old one would, so I don't know 
if it's because I don't understand the problem or if it's because I'm 
missing something from the old proposal.


>    - It is easy for developers to implement the above behaviour in JS if
>    desired

I wouldn't say "easy".


> I think the new proposal addresses the first point, since it's opt-in. I 
> don't think the second point is a problem, since [get|put]ImageData() 
> will be back to manipulating exact backing store pixels, so no 
> non-integer resizing will be required. The third point becomes moot.

Agreed.


> One question: now that some browsers are including browser zoom (page 
> zoom) in window.devicePixelRatio, will/should the new proposal 
> automatically cause a resize callback on page zoom, in order to preserve 
> 1:1 device pixels?

My intent was to do so, yes. In practice I presume it'd be up to the 
browser to decide how often to actually do this (e.g. if page zoom is 
being smoothly animated, you may wish to only do it every few frames).


> (Note that I think this is a problem with current JS-based 
> implementations of canvas auto-scale as well, although perhaps there's a 
> DOM event for this that you can listen to; I might just be showing my 
> ignorance here.)

Currently page zoom should trigger a 'resize' event, but I expect few 
pages check.


On Tue, 10 Sep 2013, Justin Novosad wrote:
>
> There is another closely related issue that's been discussed before: 
> adding a redraw callback to 2d canvas.  In the past we discussed this 
> for solving the problem of recoverring from a gpu context loss, but it 
> seems there may be better reasons to consider adding a redraw callback 
> such as freeing memory consumed by canvas backing stores that are in 
> background tabs, and re-building the content when needed. This 
> discussion was revived in the past few days on the chromium graphics-dev 
> mailing list: 
> https://groups.google.com/a/chromium.org/forum/?fromgroups#!topic/graphics-dev/CQJXpXxO6dk
> 
> The idea is still embryonic and we're brainstorming in this chromium 
> issue: crbug.com/287823
> 
> I think that discussion should be merged with this thread because a 
> resize event is another case where one may want to redraw. It would be 
> great to solve all of these issues together.

I think it would make eminent sense to also fire the event ('resize', I 
guess) if the context was lost or if the canvas was about to be made 
visible again after the browser dropped the rendering, yes.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'