[whatwg] createImageData

[Oliver Hunt]

On May 13, 2008, at 4:28 PM, Vladimir Vukicevic wrote:

>
> On May 13, 2008, at 4:10 PM, Oliver Hunt wrote:
>>>
>>>> My experience implementing this in WebKit showed a pure byte  
>>>> array backing store was significantly faster than using boxed  
>>>> values.
>>>
>>> Faster for which operation, though?  The put, or the actual  
>>> manipulation?  It's a tradeoff, really; if you're doing limited  
>>> pixel manipulation, but lots of putImageData, you can optimize  
>>> that directly by just calling putImageData once to an offscreen  
>>> canvas and then blitting that with drawImage.  If you're doing  
>>> lots of pixel manipulation but only one putImageData, I guess you  
>>> can use a JS array for your intermediate ops to avoid the checking  
>>> overhead, and set the image data pixels all at once (though again  
>>> paying the checking penalty per pixel), but having cheap  
>>> putImageData.
>>>
>>> Throwing the error at putImageData time lets the implementation  
>>> optimize in whatever way is most convenient/performant (either at  
>>> pixel operation time by setting an error bit in the ImageData  
>>> object which is checked by putImageData, or at putImageData time),  
>>> and is (IMO) more flexible.. given that errors are an exceptional  
>>> case, I don't think the spec should force the checking per pixel.
>>
>> I found it faster in general across quite a few tests.  I would  
>> argue that if you are using ImageData in a way that leads to you  
>> writing to the same pixel multiple times you should improve your  
>> algorithms (this is just the generic over painting issue).
>
> I dunno, some kind of iterative algorithm that you want to visualize  
> at random timesteps.  You could keep the output in a separate array  
> and copy over when you want to render it.

I'm not sure what you mean.  By my interpretation that would require  
more work when you validate on blit, because while you would not have  
to validate when you copy the invalidated region of your buffer into  
the ImageData buffer it would not need to be validated, but when you  
then go to blit the ImageData, putImageData *must* revalidate the  
entirety of the ImageData buffer (hmmm, i suppose the UA could try  
tracking dirty regions in the ImageData buffer to minimise  
revalidation? i suspect this would not be significantly better than  
just validating on every pixel put)

If you actually meant you were using the ImageData buffer as your  
working buffer then you would possibly be doing excessive  
revalidation, but in my experience such a case would be atypical (and  
we're more interested in the performance of the normal case vs. edge  
cases) and the cost of clamping, etc is dwarfed by the dispatch cost  
just to do the assignment (at least in WebKit) so i'm not sure there  
would a significant loss in performance anyway.

>
>
>> A very reall issue to consider though is the case where I've been  
>> very careful to only update those pixels that need to be updated.   
>> If the ImageData is not clamped, etc on put then *every* blit must  
>> do a complete revalidation of the entire ImageData data buffer.
>
> Yep, that's true.
>
>> I think we need a list of use cases for ImageData, off the top of  
>> my head i can think of:
>> * filters -- in general a single write per pixel, potentially  
>> multiple reads
>> * Generated images -- still arguably single write per pixel
>> * I'm not sure what to call this -- but things like http://jsmsxdemo.googlepages.com/jsmsx.html
>>
>> I honestly can't think of something that would (sanely) expect to  
>> be writing multiple times to the same pixel between blits, but i  
>> should note i haven't actively spent any significant time trying to  
>> come up with these.  That said in all of the above cases the cost  
>> of immediate clamping is technically the same as delaying the  
>> clamp, although it also has the benefit of allowing reduced memory  
>> usage.
>
> Yeah, those are all good use cases -- it just seems like requiring  
> immediate clamping is basically specifying for a specific  
> implementation, when the overall goal is "checking for invalid  
> data".  Specifying that the error should come from putImageData  
> would give implementations more flexibility, without limiting error  
> checking.  (You could argue that it's easier to get a precise error  
> location by checking on pixel assignment, but I don't think that the  
> potential cost and loss of flexibility is worth it.  Once authors  
> know that they have an error in their data, they can take other  
> action to track it down.)

There is no implementor freedom here -- you either clamp immediately,  
or you do not clamp immediately, if one UA does not clamp then it will  
have different behaviour from the other UAs leading to behavioural  
incompatibility (eg. one site may expect the values to be clamped  
immediately, and therefore not work in UAs that don't clamp, another  
may believe that the clamping happens later leading it to break in  
those UAs that clamp immediately).

This is not to say there's no room for implementation variation -- an  
implementation *could* maintain the data as an array of boxed values  
provided it performed the requisite toNumber and clamping and rounding  
operations, or it could store as an array of unboxed types -- there  
are potential advantages in doing it in either way.  The issue is when  
clamping occurs, which isn't an implementation detail, it's  
fundamental to how the data structure works.

>    - Vlad

--Oliver