[whatwg] [canvas] request for {create, get, put}ImageDataHD and ctx.backingStorePixelRatio

[Ian Hickson]

On Tue, 20 Mar 2012, Edward O'Connor wrote:
> 
> Unfortunately, lots of <canvas> content (especially content which calls 
> {create,get,put}ImageData methods) assumes that the <canvas>'s backing 
> store pixels correspond 1:1 to CSS pixels, even though the spec has been 
> written to allow for the backing store to be at a different scale 
> factor.

I've fixed the text so that those methods now always return 96dpi data.


> I'd like to propose the addition of a backingStorePixelRatio property to 
> the 2D context object. Just as window.devicePixelRatio expresses the 
> ratio of device pixels to CSS pixels, ctx.backingStorePixelRatio would 
> express the ratio of backing store pixels to CSS pixels. This allows 
> developers to easily branch to handle different backing store scale 
> factors.

I've added window.screen.canvasResolution which returns the resolution 
that is being used for 2D canvases created during the current task.


> Additionally, I think the existing {create,get,put}ImageData API needs 
> to be defined to be in terms of CSS pixels, since that's what existing 
> content assumes.

Done.


> I propose the addition of a new set of methods for working directly with 
> backing store image data. (New methods are easier to feature detect than 
> adding optional arguments to the existing methods.) At the moment I'm 
> calling these {create,get,put}ImageDataHD, but I'm not wedded to the 
> names. (Nor do I want to bikeshed them.)

Done.

I've also added toDataURLHD and toBlobHD.


On Tue, 20 Mar 2012, James Robinson wrote:
>
> If we are adding new APIs for manipulating the backing directly, can we 
> make them asynchronous? This would allow for many optimization 
> opportunities that are currently difficult or impossible.

I haven't done this, because it would make the API rather weird. But I am 
happy to do it if people think the API weirdness is a cost worth paying.

Note that technically getImageData() doesn't have to block -- it's array 
access on ImageData that has to block. It would be possible to implement 
getImageData() in such a way that the ImageData object is lazily filled. 
You'd end up blocking later if the author really needed the data, but it's 
possible to write code that doesn't block (though you wouldn't necessarily 
know how long to wait, I guess).


On Tue, 20 Mar 2012, Boris Zbarsky wrote:
> On 3/20/12 6:36 PM, Glenn Maynard wrote:
> > The drawing calls that happen after would need to be buffered (or 
> > otherwise flush the queue, akin to calling glFinish), so the 
> > operations still happen in order.
> 
> The former seems like it could get pretty expensive and the latter would 
> negate the benefits of making it async, imo.

Having the operations not occur in order would make the API quite 
difficult to use, so if that's not an option, I don't think it's worth it.


On Wed, 21 Mar 2012, Maciej Stachowiak wrote:
> On Mar 20, 2012, at 12:00 PM, James Robinson wrote:
> > 
> > If we are adding new APIs for manipulating the backing directly, can 
> > we make them asynchronous? This would allow for many optimization 
> > opportunities that are currently difficult or impossible.
> 
> Neat idea to offer async backing store access. I'm not sure that we 
> should tie this to backing store access at true backing store resolution 
> vs at CSS pixel nominal resolution, because it will significantly raise 
> the barrier to authors recoding their existing apps to take full 
> advantage of higher resolutions. With Ted's proposal, all they would 
> have to do is use the HD versions of calls and change their loops to 
> read the bounds from the ImageData object instead of assuming. If we 
> also forced the new calls to be async, then more extensive changes would 
> be required.
> 
> I hear you on the benefits of async calls, but I think it would be 
> better to sell authors on their benefits separately.

I think it depends how strong the benefits are. In this particular case, I 
tend to agree that the benefits aren't really worth tying them together, 
and possibly not worth providing the async model as a separate API at all.

Maybe we could have an attribute on ImageData that says whether an array 
index read would have to block on getting the data or whether it's ready, 
maybe coupled with an event that says when it's ready?

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'