[whatwg] [canvas] request for {create, get, put}ImageDataHD and ctx.backingStorePixelRatio

[Oliver Hunt]

On Apr 16, 2012, at 11:07 AM, Darin Fisher <darin at chromium.org> wrote:

> 
> Carrots and Sticks.
> 
> Aren't we missing an opportunity here?  By giving web developers this easy
> migration path, you're also giving up the opportunity to encourage them to
> use a better API.  Asynchronous APIs are harder to use, and that's why we
> need to encourage their adoption.  If you just give people a synchronous
> version that accomplishes the same thing, then they will just use that,
> even if doing so causes their app to perform poorly.
> 
> See synchronous XMLHttpRequest.  I'm sure every browser vendor wishes that
> didn't exist.  Note how we recently withdrew support for synchronous
> ArrayBuffer access on XHR?  We did this precisely to discourage use of
> synchronous mode XHR. Doing so actually broke some existing web pages.  The
> pain was deemed worth it.
> 
> GPU readback of a HD buffer is going to suck.  Any use of this new API is
> going to suck.
> 
> -Darin
> 

Any use of imagedata i've seen assumes that they can avoid intermediate states in the canvas ever being visible, if you make reading and writing the data asynchronous you break that invariant and suddenly makes things much harder for the user.

The reason we don't want IO synchronous is because IO can take a potentially unbound amount of time, if you're on a platform that makes a memcpy take similarly unbound time, i recommend that you work around it.

Anyway, the sensible approach to imagedata + hardware backed canvas is to revert to a software backed canvas, as once someone has used imagedata once, they're likely to do it again (and again, and again) so it is probably a win to just do everything in software at that point.  Presumably you could through in heuristics to determine whether or not it's worth going back to the GPU at some point, but many of the common image data use cases will have awful perf if you try to keep them on the GPU 100% of the time.

--Oliver

> 
> 
>> 
>> 
>>> 
>>> - James
>>> On Mar 20, 2012 10:29 AM, "Edward O'Connor" <eoconnor at apple.com>
>> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> Unfortunately, lots of <canvas> content (especially content which calls
>>>> {create,get,put}ImageData methods) assumes that the <canvas>'s backing
>>>> store pixels correspond 1:1 to CSS pixels, even though the spec has been
>>>> written to allow for the backing store to be at a different scale
>>>> factor.
>>>> 
>>>> Especially problematic is that developers have to round trip image data
>>>> through a <canvas> in order to detect that a different scale factor is
>>>> being used.
>>>> 
>>>> I'd like to propose the addition of a backingStorePixelRatio property to
>>>> the 2D context object. Just as window.devicePixelRatio expresses the
>>>> ratio of device pixels to CSS pixels, ctx.backingStorePixelRatio would
>>>> express the ratio of backing store pixels to CSS pixels. This allows
>>>> developers to easily branch to handle different backing store scale
>>>> factors.
>>>> 
>>>> Additionally, I think the existing {create,get,put}ImageData API needs
>>>> to be defined to be in terms of CSS pixels, since that's what existing
>>>> content assumes. I propose the addition of a new set of methods for
>>>> working directly with backing store image data. (New methods are easier
>>>> to feature detect than adding optional arguments to the existing
>>>> methods.) At the moment I'm calling these {create,get,put}ImageDataHD,
>>>> but I'm not wedded to the names. (Nor do I want to bikeshed them.)
>>>> 
>>>> 
>>>> Thanks for your consideration,
>>>> Ted
>>>> 
>> 
>>