[whatwg] [canvas] imageRenderingQuality property

[Ian Hickson]

On Mon, 2 Jun 2008, Vladimir Vukicevic wrote:
> 
> I'd like to propose adding an imageRenderingQuality property on the 
> canvas 2D context to allow authors to choose speed vs. quality when 
> rendering images (especially transformed ones).

How can an author know which is appropriate?


> This is modeled on the SVG image-rendering property, at 
> http://www.w3.org/TR/SVG/painting.html#ImageRenderingProperty:
> 
>   attribute string imageRenderingQuality;
> 
> 'auto' (default): The user agent shall make appropriate tradeoffs to 
> balance speed and quality, but quality shall be given more importance 
> than speed.
> 
> 'optimizeQuality': Emphasize quality over rendering speed.
> 
> 'optimizeSpeed': Emphasize speed over rendering quality.

This seems to be something that is completely inappropriate for SVG or for 
canvas. The UA should make its own tradeoffs, likely tradeoffs that don't 
even remotely fit into the categories above (e.g. adapting to scripts that 
do animations to converge on the best quality possible at 30fps, or 
optimise for memory usage).


On Mon, 2 Jun 2008, Oliver Hunt wrote:
>
> Um, could you actually give some kind of reasoning for these?  I am not 
> aware of any significant performance issues in Canvas that cannot be 
> almost directly attributed to JavaScript itself rather than the canvas.

On Mon, 2 Jun 2008, Vladimir Vukicevic wrote:
> 
> Sure; bilinear filtering is slower than nearest neighbour sampling, and 
> in many cases the app author would like to be able to decide that 
> tradeoff (or, at least, to be able to say "I want this to go as fast as 
> possible, regardless of quality").  Some apps might also render to a 
> canvas just once, and would prefer to do it at the highest quality 
> filtering available even if it's more expensive than the default.

Why not just have the UA run in a high quality mode the first time it is 
painted on, but if the script tries to paint again within a certain amount 
of time, switch to high speed?

Trusting the author to do this right seems like a fantastically bad idea.


On Mon, 2 Jun 2008, David Hyatt wrote:
> On Jun 2, 2008, at 4:34 PM, Vladimir Vukicevic wrote:
> > 
> > Yeah, I agree -- I thought that there was some plan somewhere to 
> > uplift a bunch of these SVG CSS properties into general usage?  I know 
> > that Gecko uplifted text-rendering, we should figure out what else 
> > makes sense to pull up.  (If image-rendering were uplifted, it would 
> > apply to <canvas>, for the scaling/transformation of the canvas 
> > element itself as opposed to the canvas rendering content.)
> 
> Right, that would be my expectation as well (that the CSS property could 
> be applied to <canvas> to control the quality when rendering the canvas 
> buffer itself).

That neatly moves the problem away from the canvas API, and thus it 
wouldn't be my problem necessarily, but I still don't think it'd be a good 
idea. :-)


On Mon, 2 Jun 2008, Oliver Hunt wrote:
>
> That's exactly what i would be afraid of people doing.  If I have a fast 
> system why should i have to experience low quality rendering?  It should 
> be the job of the platform to determine what level of performance or 
> quality can be achieved on a given device.  Typically such a property 
> would be considered a "hint", and as such would likely be ignored.
> 
> If honouring this property was _required_ rather than being a hint you would
> hit the following problems:
>
> * Low power devices would have a significant potential for poor 
> performance if a developer found that their desktop performed well so 
> set the requirement to high quality.
>
> * High power devices would be forced to use low quality rendering modes 
> when perfectly capable of providing better quality without significant 
> performance penalty.
> 
> Neither of these apply if the property were just a hint, but now you 
> have to think about what happens to content that uses this property in 
> 18 months time. You've told the UA to use a low quality rendering when 
> it may no longer be necessary, so now the UA has a choice it either 
> always obeys the property meaning lower quality than is necessary so 
> that new content performs well, or it ignores the property in which case 
> new content performs badly.
> 
> The correct behaviour would be for the UA to work out itself what it can 
> do, which it needs to be able to do anyway, in order to satisfy the 
> "auto" option.

That line of reasoning seems correct to me.


On Mon, 2 Jun 2008, Vladimir Vukicevic wrote:
> 
> If web apps misuse the property, then bugs should be filed on those apps 
> that incorrectly use the property, and the app developer should fix 
> them.

That's naive, I'm afraid. In practice, sites are written and abandoned, 
and don't get fixed.


On Tue, 3 Jun 2008, Robert O'Callahan wrote:
> 
> These hint properties are opt-in for UAs. If you don't like the idea, 
> just treat all values as "auto".

There's not much point us adding the feature if multiple UAs aren't going 
to implement it.

Also, if it's a hint, then it's untestable, and we wouldn't be able to 
prove interoperability, which again makes it something not really worth 
adding to the spec.


On Mon, 2 Jun 2008, Vladimir Vukicevic wrote:
> 
> The web platform shouldn't prevent developers from exercising control 
> over how their content is rendered; most developers, as you say, 
> probably shouldn't change anything from the default 'auto'.  But the 
> capability should be there. Arbitrarily deciding what developers can and 
> can't do isn't interesting from the perspective of creating a 
> full-featured platform, IMO.
> 
> No matter how fast smooth/bilinear filtering is, something more complex 
> is always going to be slower, and something less complex is always going 
> to be faster.  If those perf differences are significant to the web app, 
> no matter how small, you're going to want to be able to have that 
> control.  If they're not, then you should just be using 'auto' and let 
> the UA handle it.

On Mon, 2 Jun 2008, David Hyatt wrote:
> 
> I completely agree.  Almost any feature can be abused.  It's not our job 
> to withhold features just because they can be abused.  It's also worth 
> pointing out that this is a common graphical knob supported by Cairo and 
> CoreGraphics. It is a proprietary MS CSS property and an SVG CSS 
> property.  In other words, it seems to be a pretty widely implemented 
> feature and as such seems like it would be a worthwhile addition to 
> canvas.
> 
> If we add this, we should also add support for text rendering quality as 
> well, since canvas is picking up the ability to draw text.

The point is that you have no way to know whether you need to enable the 
speed optimisation or the quality optimisation. Say you're doing something 
which runs fine on a high-end Dell machine, runs ok on a Wii, and runs 
terribly on an iPhone. What setting do you use?


On Mon, 2 Jun 2008, Oliver Hunt wrote:
>
> The issue is not that certain operations are slower than others, the 
> issue is that anything that requires the developer to choose between 
> performance/quality is going to become obsolete as the performance trade 
> offs are constantly moving and are not the same from UA to UA, from 
> platform to platform.  I think the issue of performance is a complex one 
> that will not benefit in the long term from a simple on off switch.  
> Conceivably we could introduce new rendering primitives, such as 
> CanvasSprite, CanvasLayer, or some such which would, i suspect, provide 
> a similar benefit, but be more resilient in the face of changing 
> performance characteristics.

The decision would have to change over time as well, indeed, something 
which is just as hard for the author to deal with as multiple platforms.


On Tue, 3 Jun 2008, Jerason Banes wrote:
>
> If you don't mind someone weighing in who's been working with the 
> Nintendo Wii for the past year and a half, I have found that the 
> "quality" setting in Flash is tremendously useful. While the march of 
> Moore's law has ensured that Flash on the desktop is zippy, it has also 
> created a gap whereby smaller devices are powerful enough to compete 
> with last generation desktops. For these devices, the quality setting is 
> more than a "nice to have". It's absolutely critical to obtaining decent 
> performance.

Wouldn't it be better for the platform to decide that for itself?


> The algorithm's I've used over on WiiCade automatically adjust for high
> quality when on the desktop, and low to medium quality when on the Wii. This
> provides a more consistent experience to users of both systems. In addition,
> many game provide a quality setting in their options screen. (Similar to the
> rendering features options in most 3D games.) This allows the user to adjust
> the rendering speed manually if he finds his system is too slow or the game
> in question is more than fast enough on the Wii.

Why wouldn't the platform just support that natively instead of requiring 
every Web developer to do it manually?

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'