[whatwg] Need clarification on DOM exceptions thrown by canvas 2D drawImage

[Ian Hickson]

On Mon, 8 Aug 2011, Jeff Muizelaar wrote:
> > On Mon, 8 Aug 2011, Justin Novosad wrote:
> >> 
> >> This inquiry is regarding this page of the specification: 
> >> http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html 
> >> In section 4.8.11.1.10 Images, about drawImage(), it is stated that 
> >> "If one of the sw or sh arguments is zero, the implementation must 
> >> raise an INDEX_SIZE_ERR exception"  There are no other references to 
> >> other circumstances under which INDEX_SIZE_ERR should be thrown, and 
> >> there is no indication of what the correct behavior is when the 
> >> source rectangle is completely or partially outside the bounds of the 
> >> source image.

On Tue, 9 Aug 2011, Philip Taylor wrote:
> 
> The spec used to throw exceptions on out-of-bounds source rectangles,
> but that causes breakage because floats are imprecise (e.g.
> http://www.jigzone.com/xmockup/oCanvasBug.php failed in Opera because
> 79.00000000000001 > 79 as 64-bit double, whereas other browsers
> presumably rounded to 32-bit float first), so it had to be changed.
> (http://html5.org/r/5373 first, then changed again because of
> http://www.w3.org/Bugs/Public/show_bug.cgi?id=10799 to be consistent
> with filtering behaviour.)

On Mon, 8 Aug 2011, Jeff Muizelaar wrote:
> On 2011-08-08, at 4:58 PM, Ian Hickson wrote:
> > 
> > A bit lower down in the same section, the spec says: "When the 
> > filtering algorithm requires a pixel value from outside the original 
> > image data, it must instead use the value from the nearest edge pixel. 
> > (That is, the filter uses 'clamp-to-edge' behavior.)"
> 
> The clamp-to-edge behavior doesn't really work well with Coregraphics' 
> drawImage call. This means that this behaviour is not implemented in 
> Firefox on OS X and I expect WebKit doesn't implement it for a similar 
> reason. I was actually hoping the spec could be changed to the simpler 
> behaviour of just clamping the source rectangle to the bounds of the 
> image. This behaviour is easy to implement on all platforms and is still 
> quite reasonable.

On Tue, 9 Aug 2011, Philip Taylor wrote:
> 
> Does the clamp-to-edge behaviour work fine when the source rectangle
> is entirely inside the image? e.g. the image
> 
> 8800
> 8800
> 0088
> 0088
> 
> (where each digit is a pixel) drawn at 2x scale with bilinear
> filtering should give
> 
> 88862000
> 88862000
> 88862000
> 66653222
> 22235666
> 00026888
> 00026888
> 00026888
> 
> because of the filtering requirements. If CoreGraphics can't do that 
> then it's broken (per the spec) regardless of how source rectangles are 
> handled. Or is it able to do clamp-to-edge fine up to the edge of the 
> source image, just not extend that beyond the image when the source 
> rectangle is expanded further?

On Tue, 9 Aug 2011, Justin Novosad wrote:
> 
> [...] the clamp-to-edge is essential for many existing canvas-based 
> games that use large images as sprite maps.  Without clamping to the 
> edge of the source rectangle you would get color bleeding between 
> adjacent sprites when the draw operation magnifies the sprite, or even 
> just when it is not perfectly pixel-aligned.

On Tue, 9 Aug 2011, Boris Zbarsky wrote:
> 
> We're talking about clamping to the edge of the backing store, not 
> clamping to the edge of the source rectangle, unless I missed something.  
> There's no clamping to the edge of the source rectangle specified in the 
> spec, as far as I can tell.

On Tue, 9 Aug 2011, Glenn Maynard wrote:
> 
> It sounds like it would make fast GPU-accelerated blitting difficult, 
> even in fairly simple cases.

On Tue, 9 Aug 2011, Justin Novosad wrote:
> 
> Not that bad.
> I can think of two ways:
> A) create a cropped texture just for the sub rectangle
> B) implement your own clamping in the shader
> 
> I think A) is problematic because some platforms do not have 
> non-power-of-two textures, though that problem is slowly fading away. 
> Chrome does B)

On Tue, 9 Aug 2011, Jeff Muizelaar wrote:
> 
> Afaict, it is able to do clamp-to-edge fine up to the edge of the source 
> image
> 
> For example, I get something like the following in CoreGraphics:
> 
> 88871000
> 88871000
> 88871000
> 77761111
> 11116777
> 00017888
> 00017888
> 00017888
> 
> I can't explain the 7/1 vs. 6/2 but it does the right thing as far as 
> edge padding goes.

On Fri, 19 Aug 2011, Justin Novosad wrote:
> 
> I just wanted to revive this thread because the issue of what to do with 
> source rectangles that are partially outside of the source image in 
> calls to canvas drawImage is still not completely resolved.
> 
> The discussion continued here: 
> https://bugs.webkit.org/show_bug.cgi?id=65709 It is now spun-off into a 
> separate issue: https://bugs.webkit.org/show_bug.cgi?id=66574
> 
> In the discussion for 65709, we think we figured out what the right 
> thing to do is. Nonetheless, it would probably be best if the matter 
> could be settled definitively and formally in the specification.
 
On Mon, 8 Aug 2011, Justin Novosad wrote:
> On Mon, Aug 8, 2011 at 4:58 PM, Ian Hickson <ian at hixie.ch> wrote:
> > On Mon, 8 Aug 2011, Justin Novosad wrote:
> >>
> >> This inquiry is regarding this page of the specification:
> >> http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html
> >> In section 4.8.11.1.10 Images, about drawImage(), it is stated that "If
> >> one of the sw or sh arguments is zero, the implementation must raise an
> >> INDEX_SIZE_ERR exception"  There are no other references to other
> >> circumstances under which INDEX_SIZE_ERR should be thrown, and there is
> >> no indication of what the correct behavior is when the source rectangle
> >> is completely or partially outside the bounds of the source image.
> >
> > A bit lower down in the same section, the spec says: "When the filtering
> > algorithm requires a pixel value from outside the original image data, it
> > must instead use the value from the nearest edge pixel. (That is, the
> > filter uses 'clamp-to-edge' behavior.)"
> 
> Well no one is doing that! That would mean that if the source
> rectangle is partially out of bounds, say the right half, then that
> half would be composed of horizontal streaks.

On Mon, 8 Aug 2011, Justin Novosad wrote:
>
> I think clamping should only apply to a 0.5 pixel margin outside the 
> bounds of the source rectangle/source image intersection.

I've updated the spec to say that the drawImage call should clip the 
destination rectangle to the space that can be drawn on given the source 
image and source rectangle. This keeps the edges of images crisp, while 
making everything well-defined. (I hope!)

Let me know if this is inadequate or unsatisfactory.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'