[whatwg] Fwd: RE: Inconsistent behaviour of globalCompositeOperation property - Drawing model discussion

Tue Jan 4 15:05:30 PST 2011

If Microsoft, Opera, and Mozilla all subscribed to the current version of the spec, it looks like WebKit is the only hold out.
In the discussions that I followed, I noticed no strong arguments, other then my own (concerning resource consumption, which Robert O'Callahan aparently rejected as invalid) in favor of changing the specification such that only covered pixels are affected by compositing.

Given the statements above I no longer think that changing the spec in this regard is a good thing, but I still believe that the disappearance of shadows for the source-in and copy modes and the strange result when shadows are drawn and the composite operation is source-out should be corrected. To do this I suggested the following in my previous e-mail, but I got no comments about my suggestion so I repeat it here (please excuse my insistence):

Replace steps 3 to 6 of the drawing model, with:

3. When shadows are drawn<http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#when-shadows-are-drawn>, composite B (source) with A (destination) using destination-over operation.

4. Multiply the alpha component of every pixel in A by globalAlpha<http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-globalalpha>.

5. Composite A within the clipping region<http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#clipping-region> over the current canvas bitmap using the current composition operator.

Please consider the proposal above.

Thanks to everyone for considering my thoughts,

Carol Szabo

________________________________
From: rocallahan at gmail.com [mailto:rocallahan at gmail.com] On Behalf Of ext Robert O'Callahan
Sent: Tuesday, January 04, 2011 4:40 PM
To: Szabo Carol (Nokia-MS/Boston)
Cc: chuck at jumis.com; whatwg at whatwg.org
Subject: Re: [whatwg] Fwd: RE: Inconsistent behaviour of globalCompositeOperation property - Drawing model discussion

On Wed, Jan 5, 2011 at 8:12 AM, <carol.szabo at nokia.com<mailto:carol.szabo at nokia.com>> wrote:
Please see my in-line comments below:

________________________________
Version 1:

4.8.11.1.13 Drawing model

When a shape or image is painted, user agents must follow these steps, in the order given (or act as if they do):

1. Render the shape or image onto an infinite transparent black bitmap, creating image M1, as described in the previous sections except that for the purpose of this step every pixel of the image will be considered to be fully opaque white and the current fillStyle will be considered to be solid fully opaque white and the strokeStyle will be considered fullyOpaque white as well

2. When shadows are drawn, render the shadow from image M1, using a fully opaque white shadow color as described in the shadows section, creating image M2.

3. Let C1 be a region obtained by intersecting the canvas's cliping region with the set of pixels in the canvas that correspond to pixels in M1 (by having the same coordinates) that are not fully transparent.

4. Let C2 be a region obtained by intersecting the canvas's cliping region with the set of pixels in the canvas that correspond to pixels in M2 (by having the same coordinates) that are not fully transparent.

5. Render the shape or image onto an infinite transparent black bitmap, creating image A, as described in the previous sections. For shapes, the current fill, stroke, and line styles must be honored, and the stroke must itself also be subjected to the current transformation matrix.

6. When shadows are drawn, render the shadow from image A, using the current shadow styles, creating image B.

7. When shadows are drawn, multiply the alpha component of every pixel in B by globalAlpha.

8. When shadows are drawn, composite B with the current canvas bitmap, cliping results to region C2 defined above, using the current composition operator.

9. Multiply the alpha component of every pixel in A by globalAlpha.

10. Composite A with the current canvas bitmap, cliping results to region C1 defined above, using the current composition operator.

Making a binary fully-transparent/not-fully-transparent per-pixel decision to create regions C1 and C2 seems like it can't be right in the presence of antialiasing.

Suppose we have a path filled with black and operator "copy". Any pixel on the edge of that path that gets any nonzero coverage value from antialiasing will end up solid black with this proposal. That's going to look very ugly. We'll want a solution where any canvas pixel which has a very small amount of coverage by the path will be mostly unchanged in the final result.

I do not understand why pixels touched by antialiasing are going to be solid black.
Yes, I made a mistake. The actual result will be mostly-transparent black, but that is equally unacceptable.

In step 1, every pixel which is very slightly covered by the path will be filled with mostly-transparent white.
In step 3, all such pixels will be added to C1.
In step 5, those pixels will be set to mostly-transparent black in image A.
In step 10, for those pixels we'll composite mostly-transparent black onto the canvas with operator "copy", setting the canvas pixels to mostly-transparent black.

The core problem is steps 3 and 4. Making a binary decision for each pixel whether it's "in" or "out" of the shape simply can't work well  when coverage-based antialiasing is being used.

If you generalize C1 and C2 to be alpha masks, or rephrase the approach so that it permits such generalization, then this could work. But you have to be careful about how you use partial alpha values in step 10.

The way I understand antialiasing (and maybe I am wrong), pixels that are partly touched retained partly their old color and transparency and get parly the new color and transparency. More precisely the resulting transparency and color components an average of the color component being painted and the previous color component weighted by the coverage fraction of the pixel. Hence partially covered pixels are partially transparent, thus the background behind the canvas should shine through and the partially covered pixels won't be entirely black unless that background is black as well.

That's right, but your proposal interacts with this process.

I agree with you though that there are cases when inappropriately using globalCompositeOperation can yield ugly and perhaps surprising results, such as in the case you described if the canvas is completely red before the operation and it is put on a page that has green background, thus the shape will acquire an unexpected slightly green rim between black and red.

Yes, it's easy to produce ugliness. With great power comes great responsibility :-).

Version 2:

4.8.11.1.13 Drawing model

When a shape or image is painted, user agents must follow these steps, in the order given (or act as if they do):

1. Render the shape or image onto an infinite transparent black bitmap, creating image A, as described in the previous sections. For shapes, the current fill, stroke, and line styles must be honored, and the stroke must itself also be subjected to the current transformation matrix.

2. When shadows are drawn, render the shadow from image A, using the current shadow styles, creating image B.

3. When shadows are drawn, multiply the alpha component of every pixel in B by globalAlpha.

4. When shadows are drawn, composite, using the current composition operator, B with the current canvas bitmap, cliping results to the cliping region of the canvas and to the pixels that would have taken the shadow's color if the composition operator would have been source-over and the shadow would have been fully opaque and the globalAlpha would have been 1.

5. Multiply the alpha component of every pixel in A by globalAlpha.

6. Composite, using the current composition operator, A with the current canvas bitmap, cliping results to the cliping region of the canvas and to the pixels that would have taken the shape's or image's pixel color if the composition operator would have been source-over and the image would have been fully opaque, the fillStyle and strokeStyle would have been a solid fully opaque color, and the globalAlpha would have been 1

Again, this needs to be modified to take into account the possibility that some pixels are partially covered.

Again, as above, Porter-Duff does not allow for partially applying the composition operator. For every pair of source and target pixels, the operator is applied.

Right. In practice implementations go beyond Porter-Duff to take some kind of "mask alpha" into account to support coverage-based antialiasing.

I myself have no particular quarel with the current spec language other than the fact that it looses shadows and that it is very expensive to implement with graphics primitives that I know of, due to the need to alocate large intermediary images to be composited later, rather then drawing directly on the target bitmap using an appropriate pen and compositor.

This depends on what your underlying implementation can do and what the workload is. In practice usage of operators other than "over" is quite rare. Copying one canvas to another is probably the main use. You can optimize the case where the source fills the destination clip region --- in those cases, which I think are most cases in practice, of course it doesn't matter whether the operation is bounded by the source shape or not.

Given that Microsoft have indicated they're happy with the current spec, and are presumably implementing it, I think we should get their explicit approval before we change the spec here. I'm still happy with either the current spec or your proposed change (after the issues have been addressed).

My understanding from some older thread was that Microsoft favored the approach where only pixels covered by the shape are composited, but that information may be outdated.

Yes, they seem to have changed their minds.

Rob
--
"Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true." [Acts 17:11]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20110104/0d1e17b7/attachment-0002.htm>