[whatwg] Drawing shapes on canvas and feedback thereon

[Ian Hickson]

On Fri, 27 Sep 2013, Ruben Rodriguez II wrote:
> On 09/05/2013 05:22 PM, Ian Hickson wrote:
> >
> > Not always, only if you don't draw the line aligned with the pixel 
> > grid (e.g. you draw a diagonal line, or a horizontal or vertical line 
> > that isn't centered in the middle of pixels on the pixels grid, or a 
> > horizontal or vertical line whose width isn't an integral number of 
> > pixels, etc).
> > 
> > The options, on a pixel grid display, are:
> > 
> >   - don't honour the position precisely -- this leads to very ugly
> >     artifacts when animating (lines jerk around), and basically means that
> >     the graphics aren't accurate.
> > 
> >   - instead of describing the shapes as vectors, describe them using
> >     programs that can adapt to the position and size they're being drawn
> >     at, such that they automatically snap to the pixel grid in a pretty
> >     fashion -- this is what fonts do.
> > 
> >   - try to trick the eye by using anti-aliasing when things don't line up
> >     exactly on the pixel grid.
> > 
> > The first two really aren't plausible options for <canvas>.
>
> I would just like to note that sometimes we do not WANT to draw precise 
> shapes. :) Many people enjoy the aesthetic of 2d pixel-based graphics, 
> and it should be a viable choice for the graphical style of a game, for 
> instance. Canvas makes this more difficult, and it shouldn't be so!

I don't really understand what you mean by "the aesthetic of 2D 
pixel-based graphics". Can you elaborate? Maybe with examples? How does 
the 2D Canvas API make this more difficult?


> Why can't we have a global option to turn this off if we want to? I'm 
> not trying to advocate for throwing away all antialiasing... I 
> understand that most applications will probably want it by default, and 
> agree with having it as the default.
> 
> Basically all it is is
> http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#image-smoothing
> ... except that we're not talking about scaling.

What does "turn this off" mean, exactly? What are you turning off? If you 
just "turn off" the current algorithms for drawing lines, you don't get 
different lines, you get no lines. What is it you want instead? Consider 
in particular lines drawn on high-res displays, lines drawn from 0,0.5 to 
100,0.5, animations, etc.


On Fri, 27 Sep 2013, Jasper St. Pierre wrote:
>
> The issue here is that the canvas API does not specify how pixels are 
> sited on the canvas: if you imagine pixels as enlarged squares on a grid 
> (shush, I know), does an X coordinate of 5 name the center of the 
> square, or the intersection between 4th and 5th squares?

It is defined:

"The CanvasRenderingContext2D 2D rendering context represents a flat 
linear Cartesian surface whose origin (0,0) is at the top left corner, 
with the coordinate space having x values increasing when going right, and 
y values increasing when going down. The x-coordinate of the right-most 
edge is equal to the width of the rendering context's scratch bitmap in 
CSS pixels; similarly, the y-coordinate of the bottom-most edge is equal 
to the height of the rendering context's scratch bitmap in CSS pixels."

The answer is the intersection.


> Neither of these are perfect solutions when you have transforms; perhaps 
> optional support for crisp lines or otherwise turning off AA would be a 
> good idea.

It is indeed the idea that led to this part of the thread. :-)

The problem is, it's not clear what that would mean exactly:

   http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Sep/0040.html


On Sat, 28 Sep 2013, Rik Cabanier wrote:
>
> Yeah, this should be specified somewhere.
> As you point out, luckily everyone is in agreement.

It's not luck. :-)


On Sat, 28 Sep 2013, Rik Cabanier wrote:
> On Fri, Sep 27, 2013 at 2:08 PM, Ian Hickson <ian at hixie.ch> wrote:
> > On Thu, 5 Sep 2013, Rik Cabanier wrote:
> > > On Thu, Sep 5, 2013 at 3:22 PM, Ian Hickson <ian at hixie.ch> wrote:
> > > > On Sat, 10 Aug 2013, Rik Cabanier wrote:
> > > > >
> > > > > I was wondering if this is something that happens in Flash as 
> > > > > well. It turns out that there's an option called "hinting: Keep 
> > > > > stroke anchors on full pixels to prevent blurry lines." [...]
> > > > >
> > > > > I think canvas should have a similar feature...
> > > >
> > > > Can you elaborate on how exactly you would want this to work? How 
> > > > would you avoid the alignment and distortion problems when 
> > > > applying this to anything less trivial than a rectangle?
> > >
> > > Basically, this would *just* move the control points and the width 
> > > of paths so the strokes are always aligned to the pixel grid (This 
> > > would take pixel density and transformations into account). After 
> > > this, you would draw as usual.
> >
> > Can you define "aligned to the pixel grid"?
> >
> > If I have a line from x1,y to x2,y, followed by an arc from x2,y back 
> > to x1,y with radius r, what should happen and why?
> 
> Align the anchor points of all the segments. Don't change any of the 
> anti-aliasing behavior.

How does this differ from simply always using integers for coordinates?


> > What if they're draw as separate paths?
> 
> I'm unsure if I follow. That shouldn't make a different. What might be 
> different however, is if you draw a diagonal line in 1 segment or 2 
> since the middle point will be aligned to the grid in the latter case.

Consider a case like this:

   http://goo.gl/VFsLqj

How do you keep the diagonal line exactly touching the arc?


On Wed, 16 Oct 2013, Rik Cabanier wrote:
> > > >
> > > > Once the Path API is implemented, we can add any number of 
> > > > features, such as adding a path so as to form the union of both 
> > > > paths, adding a path so as to form the intersection of both paths, 
> > > > etc.
> > >
> > > OK. In order to make that possible, 'addPath', 
> > > 'addPathByStrokingPath', 'addText' and 'addTextByStrokingText' 
> > > should be cut from the interface.
> 
> Paths are a construct that can be filled with a fill rule or stroked. 
> After this, they are no longer paths but regions of a certain color.

That's one way to design the API, but it doesn't seem to be the only way. 
Another way is the way the API is currently written: you have paths, and 
when you fill them, you get pixels. No need for the intermediate "region" 
concept. I don't understand why you don't like that approach. You can do 
unions and so forth with just paths, no need for regions.


> The same is true for text. (Your recent addition that describes a stroke 
> as sweeping a line following the path, is starting to align with that.)

I don't understand why the new description would be in any way different 
than the previous one. It's clearer, hopefully, and maybe less ambiguous 
in some cases, but it's still just describing converting one path into 
another path.


> The path object should represent the path in the graphics state. You 
> can't add a stroked path or text outline to the graphics state and then 
> fill/stroke it.

Why not?


> currentPath which was added by WebKit and Blink, makes that clear.

I don't see how currentPath impacts this one way or the other.


> > > > Sure, for that you need a new API function, e.g. union, which as 
> > > > I've mentioned before, I think would be a logical addition to the 
> > > > API in due course. But I don't think we should add features too 
> > > > quickly, lest we get too far ahead of browsers.
> > >
> > > So please remove the APIs such as AddPathByStrokingPath that rely on 
> > > these difficult algorithms.
> 
> Those APIs have to know how to turn strokes and text into paths (which 
> is not trivial and no UA does by default). In addition if you want to 
> have useful output (= not getting unexpected interactions of the winding 
> rules), the APIs would need to use planarization for stroke and text 
> outlines. See 
> http://blogs.adobe.com/webplatform/2013/01/31/revised-canvas-paths/ for 
> a description of the issue.

We seem to be going around in circles. We're in agreement that eventually 
we should add APIs for combining paths such that we get the equivalent of 
the union of their fill regions. I agree that converting text into paths 
is non-trivial (lots of stuff browsers do is non-trivial, that's kind of 
the point -- if it was trivial, we could leave it for authors). But I 
don't see how we get from there to you wanting the existing APIs removed.


On Tue, 29 Oct 2013, Jürg Lehni wrote:
>
> Regardless of good practices, I still believe that Path is too general a 
> name for a new prototype that is introduced at this point.

Personally I think the name is fine, and it's shipped. However, if other 
vendors want to use another name, then I'll follow whatever the majority 
of implementors use, and I hope the other vendors will adopt it too.


On Mon, 4 Nov 2013, Jürg Lehni wrote:
> 
> Instead of exposing constructors, why not simply expose the methods that 
> create them?

On Mon, 4 Nov 2013, Anne van Kesteren wrote:
> 
> Objects not having constructors is a bad API practice we are moving away 
> from.

On Wed, 6 Nov 2013, Jürg Lehni wrote:
> 
> I'm not sure everybody thinks so. There are whole libraries out there 
> that avoid the use of 'new' in favor of functions that create the 
> objects for you (e.g. Two.js, Raphael.js, etc).

We're moving towards using constructors more because that's the JavaScript 
language idiom for creating new objects. Old DOM APIs didn't use 
constructors because they were designed to be used outside browsers as 
well, but we've since given up with that design goal for Web APIs.


> We're used to calling document.createElement(), etc. To me it feels 
> strange that all of a sudden, browser provided functionality is not 
> bound to a context, but is implemented by such a free-floating 
> constructor.

It's still bound to a context. The constructors are properties of the 
global object.


On Tue, 12 Nov 2013, Elliott Sprehn wrote:
>
> I don't think this is just "in my mind". I've polled lots of developers 
> over here and other companies and every single one of them when asked 
> "what does new Path() do if you saw it in a web page?" or "what does the 
> Path object do in JS?" said it related to part of the URL. I haven't 
> found a single person that thinks the "JS Path object" is a graphics 
> thing.

I don't think that's a problem in practice, though. It's not like you'd 
come across Path objects out of context.


On Wed, 13 Nov 2013, Jussi Kalliokoski wrote:
> 
> Actually I opened this thread out of interest because I thought this was
> that this was related to files.

Hm, I guess I'm wrong that it's not a problem in practice then. :-)


> Path is also too generic even in the context of graphics. If we later on 
> want to add a path object for 3-dimensional paths, you end up with Path 
> and Path3D?

Why wouldn't we use Path for 2D and 3D? (It's hard to know without a 
concrete proposal for what a 3D path would be.)


The thread concluded with several people agreeing that we should rename 
Path to Path2D. As noted above, if this is what browsers implement, I will 
make sure to update the spec accordingly. Right now I'm only aware of one 
implementation, which calls it Path, so I've not yet updated the spec.


On Sun, 27 Oct 2013, Rik Cabanier wrote:
> 
> I think we should update the algorithm [at 
> http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#trace-a-path 
> ] so the problem is broken into stages: [...]

Before saying what you want, please describe the problem.


On Mon, 28 Oct 2013, Justin Novosad wrote:
>
> If I understand correctly, the need to produce a path that is winding 
> agnostic is for cases where we are not stroking to the display. It is 
> for cases where the API produces a stored path that could be further 
> manipulated.  The only examples of this that come to my mind right now 
> are the addPathByStrokingPath and addPathByStrokingText methods of the 
> Path object interface.  Is that what you had in mind?  Perhaps the 
> requirement for generating winding agnostic paths should be specific to 
> those two methods?

The only methods for which the spec currently requires user agents to 
ensure that they create only paths that wind clockwise, and for which this 
has any practical impact as far as I can tell, are the Path methods 
addText(), addPathByStrokingText(), and addPathByStrokingPath().


> Also it may be helpful to provide an accurate/formal description of what 
> winding agnostic means.

The spec doesn't use that term (I don't know that it's actually possible 
to create a winding agnostic path in the general case -- consider a figure 
of eight inside an ellipse, for instance). The only places where it 
currently has a requirement of this nature is where the paths are 
generated entirely by the UA in a manner that they can be guaranteed to 
not cross themselves.



On Mon, 28 Oct 2013, Rik Cabanier wrote [of the "trace a path" algorithm]:
>
> 1. Step 4:
> 
> Add a straight closing line to each closed subpath in path connecting 
> the last point and the first point of that subpath; change the last 
> point to a join (from the previously last line to the newly added 
> closing line), and change the first point to a join (from the newly 
> added closing line to the first line).
> 
> Is this needed?

Yes, otherwise closed subpaths wouldn't have their closing line dashed.


> A closed subpath already drew a line [2].
> 2: http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-closepath

I see no text there that draws a line.


> 2. Calculating dashing is a bit strange
> For instance, let's say with have a dash array of [5 5 3 3] with a dash
> offset of 0 and apply it to a path of length 20.
> According to the spec:
> - pattern width = 16
> - subpath width = 20
> - offset = *16* (from step 3: "While offset is less than pattern width,
> increment it by pattern width")
> - position = *-16* (from step 5: "Let position be zero minus offset.")
> 
> If you follow the steps, you have to go through the loop twice before
> drawing starts.
> Maybe we can change step 3:
> 
> While offset is greater than pattern width, decrement it by pattern width.
> While offset is less than *zero*, increment it by pattern width.

Yeah, this was clearly an error in the spec. Fixed.


> 3. Step 20:
> 
> If position is greater than width, then jump to the step labeled convert.
> 
> I think that should be:
> 
>  If position is greater than *subpath *width, then jump to the step labeled
> convert.

Yeah, copy/pasta from the old version of the algorithm. Fixed. Thanks.


On Sat, 2 Nov 2013, Rik Cabanier wrote:
> 
> Sorry, I misunderstood. Yes, we want to move to separate path/shape 
> objects that can be set atomically (=much faster) on the canvas context.

Why? With Path objects, you don't have to set them at all -- just paint 
straight from the Path object using stroke(path) or fill(path).


On Mon, 4 Nov 2013, Jürg Lehni wrote:
> 
> I like this feature a lot. One advantage to not underestimate is the 
> amount of effort it takes to change existing code to make use off the 
> new Path feature, while staying backward compatible with older browsers 
> that don't implement this spec. For example, in Paper.js it took only 
> three added lines of code to use cached paths if they exist rather than 
> redrawing them each time.

Being able to get a Path from the context if it exists seems reasonable, 
though it's non-trivial to define what that is given the way that 
transforms affect the context implied path.


On Mon, 4 Nov 2013, Rik Cabanier wrote:
> 
> In light of this, does anyone have objections to these 2 new methods:
> 
> Path getCurrentPath();
> void setCurrentPath(Path);

(The right question is "does anyone want these methods". The bar is 
higher than just no objections.)

What's the use case for setCurrentPath()?

The functionality of a getCurrentPath() method seems reasonable for the 
reasons Jürg gave above.

Another way to do it would be to have the Path constructor take a 
rendering context object. That would be more consistent with how Path 
objects work so far.


On Mon, 4 Nov 2013, Robert O'Callahan wrote:
>
> If you return a path in user-space, what do you get if you call
> getCurrentPath with a singular transform?
>   ctx.moveTo(0,0);
>   ctx.lineTo(1,1);
>   ctx.scale(0,0);
>   var p = ctx.getCurrentPath();
> ?

The scale() call here has no effect. The default path is affected by 
transforms when you add the path segments, not when you paint. It would be 
consistent to do the same when converting it to a Path object.

This would, of course, mean that this wouldn't have the expected effect 
with transforms, since they'd be amplified:

   c.beginPath();
   c.scale(2,2);
   c.moveTo(0,0);
   c.lineTo(1,1);
   var p = new Path(c);
   c.stroke(); // strokes the current default path from 0,0 to 2,2
   c.stroke(p); // strokes p from 0,0 to 4,4 !

   c.resetTransform();
   c.stroke(p); // strokes the current default path from 0,0 to 2,2


On Mon, 4 Nov 2013, Rik Cabanier wrote:
> 
> However, for your example, I'm unsure what the right solution is. The 
> canvas specification is silent on what the behavior is for 
> non-invertible matrices.

What question do you think the spec doesn't answer?

> I think setting scale(0,0) or another matrix operation that is not
> reversible, should remove drawing operations from the state because:
> - how would you stroke with such a matrix?

You'd get a point.

> - how do patterns operate? the same for gradient fills.

This is defined:

"If a radial gradient or repeated pattern is used when the transformation 
matrix is singular, the resulting style must be transparent black 
(otherwise the gradient or pattern would be collapsed to a point or line, 
leaving the other pixels undefined). Linear gradients and solid colors 
always define all points even with singular tranformation matrices."

> - how would you pass this to the underlying graphics library?

That's an implementation detail.

> - certain operators such as 'arc' rely on doing the transform in reverse.

If the transform is scale(0,0), an arc will always just result in a point.


On Mon, 4 Nov 2013, Rik Cabanier wrote:
> 
> After pondering this some more and looking at the different
> implementations, I propose the following:
> if the user sets a non-invertible matrix, the canvas context should be in a
> state that ignores all path drawing operations, stroke/fill calls and all
> other ctm operations (apart from setTransform). setCurrentPath is also
> ignored and getCurrentPath should return an empty path.
> If the ctm becomes invertible again (from a setTransform or a restore),
> drawing operations pick up again with the currentPoint that was active when
> the non-invertible matrix was set.

I don't understand why the current text in the spec is not sufficient.


On Tue, 5 Nov 2013, Dirk Schulze wrote:
> 
> Ian’s response so far was that it doesn’t need any further definition. 
> That is why no implementation changed the behavior since then.

As far as I can tell, the spec is clear. Several of the browsers are 
clearly wrong.


On Tue, 5 Nov 2013, Rik Cabanier wrote:
> 
> Is this the link:
> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org//2013-January/038798.html
> I don't see any replies...

http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Jul/0241.html

On Tue, 5 Nov 2013, Rik Cabanier wrote:
> 
> What's lost in his reply is that Canvas is supposed to transform the 
> existing path to the new CTM when you change the matrix.

I'm not sure what you mean there. Changing the transformation matrix 
shouldn't affect the current default path, per spec. stroke() and fill(), 
with no arguments, are only affected by the current transformation matrix 
insofar as it affects the actual stroke style or fill style.


> So for this case: [lightly edited to give concrete values]
> 
> ctx.beginPath();
> ctx.moveTo(100,100);
> ctx.lineTo(200,100);
> ctx.scale(0,0);
> ctx.lineTo(300,100);
> ctx.fill();
> ctx.strokeStyle = 'red';
> ctx.stroke();

As far as I can tell, you should get a black triangle with coordinates 
100,100, 200,100, and 0,0, with no stroke. (The line width gets scaled to 
zero, but the fill is defined across all space as black, by default.)

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'