[whatwg] [canvas] request for {create, get, put}ImageDataHD and ctx.backingStorePixelRatio

[Darin Fisher]

On Mon, Apr 16, 2012 at 11:17 AM, Oliver Hunt <oliver at apple.com> wrote:

>
> On Apr 16, 2012, at 11:07 AM, Darin Fisher <darin at chromium.org> wrote:
>
> >
> > Carrots and Sticks.
> >
> > Aren't we missing an opportunity here?  By giving web developers this
> easy
> > migration path, you're also giving up the opportunity to encourage them
> to
> > use a better API.  Asynchronous APIs are harder to use, and that's why we
> > need to encourage their adoption.  If you just give people a synchronous
> > version that accomplishes the same thing, then they will just use that,
> > even if doing so causes their app to perform poorly.
> >
> > See synchronous XMLHttpRequest.  I'm sure every browser vendor wishes
> that
> > didn't exist.  Note how we recently withdrew support for synchronous
> > ArrayBuffer access on XHR?  We did this precisely to discourage use of
> > synchronous mode XHR. Doing so actually broke some existing web pages.
>  The
> > pain was deemed worth it.
> >
> > GPU readback of a HD buffer is going to suck.  Any use of this new API is
> > going to suck.
> >
> > -Darin
> >
>
> Any use of imagedata i've seen assumes that they can avoid intermediate
> states in the canvas ever being visible, if you make reading and writing
> the data asynchronous you break that invariant and suddenly makes things
> much harder for the user.
>

I agree with Charles Pritchard that it is only the reading of pixel data
that should be asynchronous.

I think developers could learn to cope with this new design just as they do
with other asynchronous facets of the platform.



>
> The reason we don't want IO synchronous is because IO can take a
> potentially unbound amount of time, if you're on a platform that makes a
> memcpy take similarly unbound time, i recommend that you work around it.
>

Of course, GPU readbacks do not compare to network IO.  However, if the
goal is to achieve smooth animations, then it is important that the main
thread not hitch for multiple animation frames.  GPU readbacks are
irregular in duration and can sometimes be quite expensive if the GPU
pipeline is heavily burdened.



>
> Anyway, the sensible approach to imagedata + hardware backed canvas is to
> revert to a software backed canvas, as once someone has used imagedata
> once, they're likely to do it again (and again, and again) so it is
> probably a win to just do everything in software at that point.  Presumably
> you could through in heuristics to determine whether or not it's worth
> going back to the GPU at some point, but many of the common image data use
> cases will have awful perf if you try to keep them on the GPU 100% of the
> time.
>

I don't think it is OK if at application startup (or animation startup)
there is a big UI glitch as the system determines that it should not
GPU-back a canvas.  We have the opportunity now to design an API that does
not have that bug.

Why don't you want to take advantage of this opportunity?

-Darin





> >
> >
> >>
> >>
> >>>
> >>> - James
> >>> On Mar 20, 2012 10:29 AM, "Edward O'Connor" <eoconnor at apple.com>
> >> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> Unfortunately, lots of <canvas> content (especially content which
> calls
> >>>> {create,get,put}ImageData methods) assumes that the <canvas>'s backing
> >>>> store pixels correspond 1:1 to CSS pixels, even though the spec has
> been
> >>>> written to allow for the backing store to be at a different scale
> >>>> factor.
> >>>>
> >>>> Especially problematic is that developers have to round trip image
> data
> >>>> through a <canvas> in order to detect that a different scale factor is
> >>>> being used.
> >>>>
> >>>> I'd like to propose the addition of a backingStorePixelRatio property
> to
> >>>> the 2D context object. Just as window.devicePixelRatio expresses the
> >>>> ratio of device pixels to CSS pixels, ctx.backingStorePixelRatio would
> >>>> express the ratio of backing store pixels to CSS pixels. This allows
> >>>> developers to easily branch to handle different backing store scale
> >>>> factors.
> >>>>
> >>>> Additionally, I think the existing {create,get,put}ImageData API needs
> >>>> to be defined to be in terms of CSS pixels, since that's what existing
> >>>> content assumes. I propose the addition of a new set of methods for
> >>>> working directly with backing store image data. (New methods are
> easier
> >>>> to feature detect than adding optional arguments to the existing
> >>>> methods.) At the moment I'm calling these {create,get,put}ImageDataHD,
> >>>> but I'm not wedded to the names. (Nor do I want to bikeshed them.)
> >>>>
> >>>>
> >>>> Thanks for your consideration,
> >>>> Ted
> >>>>
> >>
> >>
>
>