[whatwg] Canvas and Paths

[Ian Hickson]

On Thu, 28 Nov 2013, Rik Cabanier wrote:
> On Thu, Nov 28, 2013 at 8:30 AM, Jürg Lehni <lists at scratchdisk.com> 
> wrote:
> >
> > I meant to say that it I think it would make more sense if the path 
> > was in the current transformation matrix, so it would represent the 
> > same coordinate values in which it was drawn, and could be used in the 
> > same 'context' of transformations applied to the drawing context later 
> > on.
> 
> No worries, it *is* confusing. For instance, if you emit coordinates and 
> then scale the matrix by 2, those coordinates from getCurrentPath will 
> have a scale of .5 applied.

That's rather confusing, and a pretty good reason not to have a way to go 
from the current default path to an explicit Path, IMHO.

Transformations affect the building of the current default path at each 
step of the way, which is really a very confusing API. The Path API on the 
other hand doesn't have this problem -- it has no transformation matrix. 
It's only when you use Path objects that they get transformed.


> > So this is not how most implementations currently have it defined.
> 
> I'm unsure what you mean. Browser implementations? If so, they 
> definitely do store the path in user coordinates. The spec currently 
> says otherwise [1] though.

I'm not sure what you're referring to here.


> It would be a very fast way to set a cached path in the graphics state

What would the purpose of this be? You can just pass the path to the 
relevant functions instead, no?


> Another use case is to allow authors to quickly migrate to hit regions.
> 
> ctx.beginPath(); ctx.lineTo(...); ...; ctx.fill();
> ... // lots of complex drawing operation for a control
> ctx.beginPath(); ctx.lineTo(...); ...; ctx.stroke();
> 
> 
> To migrate that to a region (with my proposed shape interface [1]):
> 
> var s = new Shape();
> 
> ctx.beginPath(); ctx.lineTo(...); ...; ctx.fill(); s.add(new
> Shape(ctx.currentPath));
> ...
> ctx.beginPath(); ctx.lineTo(...); ...; ctx.stroke(); s.add(new
> Shape(ctx.currentPath, ctx.currentDrawingStyle));
> 
> ctx.addHitRegion({shape: s, id: "control"});

Why not just add ctx.addHitRegion() calls after the fill and stroke calls?


On Fri, 6 Dec 2013, Jürg Lehni wrote:
> 
> Instead of using getCurrentPath and setCurrentPath methods as a 
> solution, this could perhaps be solved by returning the internal path 
> instead of a copy, but with a flag that would prevent further 
> alterations on it.
> 
> The setter of the currentPath accessor / data member could then make the 
> copy instead when a new path is to be set.
> 
> This would also make sense from a a caching point of view, where storing 
> the currentPath for caching might not actually mean that it will be used 
> again in the future (e.g. because the path's geometry changes completely 
> on each frame of an animation), so copying only when setting would 
> postpone the actual work of having to make the copy, and would help 
> memory consummation and performance.

I don't really understand the use case here.


On Fri, 29 Nov 2013, Rik Cabanier wrote:
> On Mon, Nov 25, 2013 at 3:55 PM, Ian Hickson <ian at hixie.ch> wrote:
> > On Sat, 28 Sep 2013, Rik Cabanier wrote:
> > > On Fri, Sep 27, 2013 at 2:08 PM, Ian Hickson <ian at hixie.ch> wrote:
> > > > On Thu, 5 Sep 2013, Rik Cabanier wrote:
> > > > > On Thu, Sep 5, 2013 at 3:22 PM, Ian Hickson <ian at hixie.ch> wrote:
> > > > > > On Sat, 10 Aug 2013, Rik Cabanier wrote:
> > > > > > >
> > > > > > > I was wondering if this is something that happens in Flash 
> > > > > > > as well. It turns out that there's an option called 
> > > > > > > "hinting: Keep stroke anchors on full pixels to prevent 
> > > > > > > blurry lines." [...]
> > > > > > >
> > > > > > > I think canvas should have a similar feature...
> > > > > >
> > > > > > Can you elaborate on how exactly you would want this to work? 
> > > > > > How would you avoid the alignment and distortion problems when 
> > > > > > applying this to anything less trivial than a rectangle?
> > > > >
> > > > > Basically, this would *just* move the control points and the 
> > > > > width of paths so the strokes are always aligned to the pixel 
> > > > > grid (This would take pixel density and transformations into 
> > > > > account). After this, you would draw as usual.
> > > >
> > > > Can you define "aligned to the pixel grid"?
> > > >
> > > > If I have a line from x1,y to x2,y, followed by an arc from x2,y 
> > > > back to x1,y with radius r, what should happen and why?
> > >
> > > Align the anchor points of all the segments. Don't change any of the 
> > > anti-aliasing behavior.
> >
> > How does this differ from simply always using integers for 
> > coordinates?
> 
> It would simplify the process for the developer as it might be difficult 
> to determine what an "integer" coordinate is, especially if there is a 
> complex CTM in effect. For instance, if the scale is .5, you'd have to 
> round to a multiple of 2.

I don't really understand when the snapping happens in this proposal. 
Consider a rotation transformation matrix. How exactly does the snapping 
work? In a space that's transformed by 45 degress, how would lines drawn 
at 44, 45, and 46 degrees to the horizontal, all starting at the same 
point, end up being rendered?


> > > What if they're draw as separate paths?
> > >
> > > I'm unsure if I follow. That shouldn't make a different. What might 
> > > be different however, is if you draw a diagonal line in 1 segment or 
> > > 2 since the middle point will be aligned to the grid in the latter 
> > > case.
> >
> > Consider a case like this:
> >
> >    http://goo.gl/VFsLqj
> >
> > How do you keep the diagonal line exactly touching the arc?
> 
> You would not :-) If the pixel that would cover over 50%, it would be 
> filled.
> 
> If the developer is interested in geometric precision, he shouldn't 
> request sharp lines.

I think there's a difference between being "interested in geometric 
precision" and interested in things not being ugly. In fact, I don't 
really see when you would want sharp lines, but _not_ want them to 
necessarily line up in a tight-fitting manner.


> > You can do unions and so forth with just paths, no need for regions.
> 
> How would you do a union with paths?
> If you mean that you can just aggregate the segments, sure but that doesn't
> seem very useful.

You say, here are some paths, here are some fill rules, here are some 
operations you should perform, now give me back a path that describes the 
result given a particular fill rule.

A shape is just a path with a fill rule, essentially. Anything you can do 
with one you can do with the other.


> > > The path object should represent the path in the graphics state. You 
> > > can't add a stroked path or text outline to the graphics state and 
> > > then fill/stroke it.
> >
> > Why not?
> 
> As designed today, you could fill it, as long as you use non-zero 
> winding. If you use even-odd, the results will be very wrong. (ie where 
> joins and line segments meet, there will be white regions)

I think "wrong" here implies a value judgement that's unwarranted.


> Stroking will be completely wrong too, because joins and end caps are 
> drawn separately, so they would be stroked as separate paths. This will 
> not give you the effect of a double-stroked path.

I don't understand why you think joins and end caps are drawn separately. 
That is not what the spec requires.


> > We seem to be going around in circles. We're in agreement that 
> > eventually we should add APIs for combining paths such that we get the 
> > equivalent of the union of their fill regions. I agree that converting 
> > text into paths is non-trivial (lots of stuff browsers do is 
> > non-trivial, that's kind of the point -- if it was trivial, we could 
> > leave it for authors). But I don't see how we get from there to you 
> > wanting the existing APIs removed.
> 
> I want them removed because they will most likely not behave in the way 
> that an author expects. When he "adds" 2 paths, he wouldn't expect that 
> there is 'interference' between them.

I don't see why not. It's exactly what happens today if you were to just 
add the same path creation statements together into the current default 
path and fill or stroke that.


> > The only methods for which the spec currently requires user agents to 
> > ensure that they create only paths that wind clockwise, and for which 
> > this has any practical impact as far as I can tell, are the Path 
> > methods addText(), addPathByStrokingText(), and 
> > addPathByStrokingPath().
> 
> don't forget tracing a path.

That's addPathByStrokingPath(). (stroke() is essentially just 
addPathByStrokingText() followed by a fill().)


> > On Mon, 4 Nov 2013, Rik Cabanier wrote:
> > >
> > > In light of this, does anyone have objections to these 2 new 
> > > methods:
> > >
> > > Path getCurrentPath();
> > > void setCurrentPath(Path);
> >
> > (The right question is "does anyone want these methods". The bar is 
> > higher than just no objections.)
> >
> > What's the use case for setCurrentPath()?
> 
> It would negate the need for a 'fill(path)', 'stroke(path)' and
> 'clip(path)'.

That's not a use case, it's just a different way of implementing the same 
use case. Why is it better? It seems distinctly more awkward and less 
idiomatic to have to set an implicit property and then call a method, than 
just call the method with the path.

Analogously, we don't say:

   Math.angle = Math.PI * 3;
   var x = Math.sin();

...we say:

   var x = Math.sin(Math.PI * 3);


> It also make it more intuitive since you can call: 
> ctx.setCurrentPath(ctx2.getCurrentPath());

I don't understand how that is intuitive. Why would you do this?

Without compelling use cases, I don't think we should add these features.


> > On Mon, 4 Nov 2013, Robert O'Callahan wrote:
> > >
> > > If you return a path in user-space, what do you get if you call
> > > getCurrentPath with a singular transform?
> > >   ctx.moveTo(0,0);
> > >   ctx.lineTo(1,1);
> > >   ctx.scale(0,0);
> > >   var p = ctx.getCurrentPath();
> > > ?
> >
> > The scale() call here has no effect. The default path is affected by
> > transforms when you add the path segments, not when you paint. It would be
> > consistent to do the same when converting it to a Path object.
> 
> No, the CTM definitely affects what you paint. Stroke width and 
> gradients for instance.

That's not the path, but sure.


> On Sat, 2 Nov 2013, Rik Cabanier wrote:
> > On Mon, 4 Nov 2013, Jürg Lehni wrote:
> > >
> > > I like this feature a lot. One advantage to not underestimate is the 
> > > amount of effort it takes to change existing code to make use of the 
> > > new Path feature, while staying backward compatible with older 
> > > browsers that don't implement this spec. For example, in Paper.js it 
> > > took only three added lines of code to use cached paths if they 
> > > exist rather than redrawing them each time.
> >
> > Being able to get a Path from the context if it exists seems 
> > reasonable, though it's non-trivial to define what that is given the 
> > way that transforms affect the context implied path.
> 
> Why is it non-trivial? UA have to calculate this path (wrt the current 
> CTM anyway).
>
> > This would, of course, mean that this wouldn't have the expected 
> > effect with transforms, since they'd be amplified:
> >
> >    c.beginPath();
> >    c.scale(2,2);
> >    c.moveTo(0,0);
> >    c.lineTo(1,1);
> >    var p = new Path(c);
> >    c.stroke(); // strokes the current default path from 0,0 to 2,2
> >    c.stroke(p); // strokes p from 0,0 to 4,4 !
> >
> >    c.resetTransform();
> >    c.stroke(p); // strokes the current default path from 0,0 to 2,2
> 
> No, these will look the same. 'new Path(c)' establishes the same path in 
> the same user coordinate space.

That's... highly unintuitive and very confusing.

Consider these. Which is wrong?

    c.beginPath();
    c.scale(2,2);
    c.moveTo(0,0);
    c.lineTo(1,0);
    c.resetTransform();
    c.lineTo(0,1);
    var p = new Path(c);
    c.stroke(); // strokes the current default path from 0,0 to 2,0 to 0,1
    c.stroke(p); // strokes p from 0,0 to 2,0 to 0,1

    c.beginPath();
    c.scale(2,2);
    c.moveTo(0,0);
    c.lineTo(1,0);
    c.lineTo(0,0.5);
    c.resetTransform();
    var p = new Path(c);
    c.stroke(); // strokes the current default path from 0,0 to 2,0 to 0,1
    c.stroke(p); // strokes p from 0,0 to 2,0 to 0,1?

    c.beginPath();
    c.scale(2,2);
    c.moveTo(0,0);
    c.lineTo(1,0);
    c.lineTo(0,0.5);
    var p = new Path(c);
    c.resetTransform();
    c.stroke(); // strokes the current default path from 0,0 to 2,0 to 0,1
    c.stroke(p); // strokes p from 0,0 to 2,0 to 0,1?

    c.beginPath();
    c.scale(2,2);
    c.moveTo(0,0);
    c.lineTo(1,0);
    c.lineTo(0,0.5);
    var p = new Path(c);
    c.stroke(); // strokes the current default path from 0,0 to 2,0 to 0,1
    c.resetTransform();
    c.stroke(p); // strokes p from 0,0 to 2,0 to 0,1?

    c.beginPath();
    c.scale(2,2);
    c.moveTo(0,0);
    c.lineTo(1,0);
    c.lineTo(0,0.5);
    var p = new Path(c);
    c.stroke(); // strokes the current default path from 0,0 to 2,0 to 0,1
    c.stroke(p); // strokes p from 0,0 to 2,0 to 0,1?
    c.resetTransform();

This is why it's non-trivial.


> > On Mon, 4 Nov 2013, Rik Cabanier wrote:
> > >
> > > However, for your example, I'm unsure what the right solution is. 
> > > The canvas specification is silent on what the behavior is for 
> > > non-invertible matrices.
> >
> > What question do you think the spec doesn't answer?
> >
> > > I think setting scale(0,0) or another matrix operation that is not 
> > > reversible, should remove drawing operations from the state because: 
> > > - how would you stroke with such a matrix?
> >
> > You'd get a point.
> 
> How would you get a point? the width is scaled to 0.

That's how you get a point -- scale(0,0) essentially reverts everything to 
a zero dimensional point.


> > > - how do patterns operate? the same for gradient fills.
> >
> > This is defined:
> >
> > "If a radial gradient or repeated pattern is used when the 
> > transformation matrix is singular, the resulting style must be 
> > transparent black (otherwise the gradient or pattern would be 
> > collapsed to a point or line, leaving the other pixels undefined). 
> > Linear gradients and solid colors always define all points even with 
> > singular tranformation matrices."
> 
> How is this better than not drawing?

I'm not arguing that it's better, just that's it's defined. We went 
through a lot of effort to make sure all these things were defined, taking 
into account author feedback, implementor feedback, etc.


On Wed, 8 Jan 2014, Rik Cabanier wrote:
> 
> I'll reiterate my proposal:
> 1. remove all the addxxx methods from the Path2D object [3]
> Path object are just containers for segments. Aggregating segments will
> generally not give the desired results since the segments will interact
> (see [1]).
> AddPath *could* be kept if people see a strong use case.
> 
> 2. introduce a new class 'Shape2D'

When making proposals, it's better to start from use cases and then 
explore those, rather than starting from APIs and then defending them. 
Without knowing what the use cases are, it's hard to evaluate the 
proposals. It's especially hard to evaluate proposals against each other.


On Mon, 13 Jan 2014, Rik Cabanier wrote:
> On Mon, Jan 13, 2014 at 12:31 PM, Justin Novosad <junov at google.com> 
> wrote:
> 
> > Maybe I am missing something, but I think this API could be 
> > implemented entirely in JS without any new canvas APIs.
> 
> That is true. Note that this is also true for the current Path proposal

Not really. The Path object has many features that need browser support:

 - filling or stroking a path doesn't blow away the current default path
 - paths can be stroked then further manipulated
 - text can be added to a Path and then manipulated
 - text can be added to a Path in a way that follows a Path

None of these features can be done with the non-Path API in JS.


> The Path APIs are simply containers of path segments with no drawing 
> intent. This means that stroking and filling has to be resolved at 
> drawing time and since those are expensive to resolve, they might impact 
> performance.

UAs can certainly pre-compute anything, or cache anything, that would be 
expensive. In particular, I would expect a browser to cache the expensive 
aspects of filling a Path, since that Path will most likely be filled 
again in the future. That's kind of the point of Path.

If there's specific optimisations that can't be done with Path, do bring 
them up, though. That's the kind of use case information that is useful.


On Tue, 14 Jan 2014, Rik Cabanier wrote:
> 
> Another use-case for the shape object would be WebGL.
> Today, users have to draw shapes and text on a 2d canvas and send over the
> bitmap to WebGL as a texture. If we could teach WebGL to take Shape2D
> object, it would have access to scalable graphics which would lower memory
> use and have better quality.
> The engineers at NVidia have a paper [1] that describes how this could work.

Couldn't the same be done with Path object? e.g. WebGL could have some 
object that you construct from a path and fill rule, which you can then 
reuse in a WebGL context.


On Fri, 24 Jan 2014, Joe Gregorio wrote:
> 
> In general we would like to see a layered approach to drawing objects, 
> starting with something primitive like Path, and at the most general a 
> Display List[1] that contains the full set of capabilities of 
> CanvasRenderingContext2D.  That layering could be done in two or three 
> objects, either [Path, DisplayList] or [Path, Shape2D, DisplayList]. In 
> all cases you can use the lower level objects to construct higher level 
> objects, i.e. use one or more Paths to build a Shape2D, and use Paths 
> and Shape2Ds to define a DisplayList.

Can you elaborate on the DisplatList part of this? Would this need to be 
explicit in the JavaScript API, or is this something you see as implicit 
in the implementation?


> The add* methods could be simplified to:
> 
>   void addPath(Path path, SVGMatrix? transformation);
>   void addPathByStrokingPath(Path path, CanvasDrawingStyles styles,
> SVGMatrix? transformation);
>   void addText(DOMString text, CanvasDrawingStyles styles, SVGMatrix?
> transformation, unrestricted double x, unrestricted double y, optional
> unrestricted double maxWidth);
>   void addTextAlongPath(DOMString text, CanvasDrawingStyles styles,
> SVGMatrix? transformation, Path path, optional unrestricted double
> maxWidth);

This really is just removing the addPathByStrokingText() method and 
splitting the overloaded addText() method into addText() and 
addTextAlongPath(), right?

Since people have started implementing this now, I've also now added a 
constructor that builds a path from a bunch of other paths and a fill 
rule, and creates a new (fill-rule-agnostic) path that is the "union", 
shape-wise, of the input paths.


> The functionality of the addPathByStrokingText methods below can be done by
> applying the above methods.

Yeah, the addPathByStrokingText() method is just intended to allow you to 
render text outlines without having to go through two Paths. It's purely a 
convenience method. Happy to drop it if implementors don't think it's 
worth it.


On Mon, 27 Jan 2014, Rik Cabanier wrote:
> 
> My issue with this add* methods is that they likely will not do what an
> author expects since the path segments will interact in unexpected ways.

They interact in exactly the ways that they do with the current default 
path. It's just like concatenating the path creation method calls.

I agree that once that is deployed we should add a method to create a path 
from other paths in a way that causes the new path to describe the union 
of the fill regions of the original paths (given some fill rule).


> In addition, I think no browser follows the spec for "tracing a path" [1].

The solution to that is to fix the browsers.


On Mon, 27 Jan 2014, Joe Gregorio wrote:
>
> How about simplifying to just these two add* methods:
> 
>   void addPath(Path path, SVGMatrix? transformation);
>   void addText(DOMString text, SVGMatrix? transformation, unrestricted
> double x, unrestricted double y, optional unrestricted double maxWidth);
> 
> Note the removal of the CanvasDrawingStyles from addText.

This drops the ability to stroke a path, which seems like a major loss, as 
well as the ability to draw text along a path, which would be rather 
disappointing (and is one of the things authors kept asking for).

We can't drop the CanvasDrawingStyles argument since that's what specifies 
the font.


On Mon, 27 Jan 2014, Rik Cabanier wrote:
>
> For instance, let's say you request the path for the letter 'o'. It's
> represented by 2 closed paths.
> What winding rule should you use when you fill that path? Should the paths
> be oriented so you have to use a NZO fill or so you can use any winding
> rule?
> 
> What happens if the path for a letter intersect with another path (or
> another letter)?

Good point. I've changed the algorithm a bit so that the resulting path 
that is added has no ambiguous overlap like this.


On Tue, 18 Feb 2014, Rik Cabanier wrote:
>
> Some people were unhappy with the SVGMatrix in the argument list but 
> unless we have something better soon, it's fine to implement. (We can 
> always use a union later)
> 
> Ian, could you remove the other APIs from the spec:

I don't think the arguments for removing these are compelling. The 
problems with the APIs have been addressed (e.g. there's no ambiguity in 
the case of overlapping text), the use cases are clear (e.g. drawing text 
around an arc or drawing a label along a line graph's line), and the API 
now supports the constructs to do unions of fill regions.


> No one has implemented them and they are confusing the browser vendors.

I don't think they're confusing anyone.


> Until we have support for shapes, the output of these methods won't be 
> stable.

These methods have been very stable. They have barely changed since they 
were added, except for some minor tweaks to fix bugs.


On Sun, 9 Mar 2014, Rik Cabanier wrote:
> On Wed, Mar 5, 2014 at 1:46 PM, Rik Cabanier <cabanier at gmail.com> wrote:
> >
> > The mozilla implementation uses different "backends" for the canvas 2D 
> > context. Even within the same document, different canvas objects can 
> > be on top of different graphics libraries. For best performance, the 
> > Path2D object should use the same graphics interface as the canvas 
> > context you're applying it to. If this is not the case, the path 
> > segments have to be walked and converted which is a costly operation.
> 
> I was thinking about this a bit more and an alternate solution would be 
> to pass an optional 2D context to the Path2D constructors. A UA could 
> ignore that context, or it could see it as a hint that the path is going 
> to be used with that context.

I don't understand why this needs to be done ahead of time. Just cache the 
calls until you're used against a backend, and then convert them to the 
optimal form at that time.


On Mon, 10 Mar 2014, Rik Cabanier wrote:
> 
> The current Path2D interface might be unacceptably slow under certain 
> circumstances and there's currently no way for authors to work around 
> this. There has to be a hint. If not, I don't see a way that firefox 
> will ship this.

I don't understand why it's slow.


On Mon, 10 Mar 2014, Joe Gregorio wrote:
> 
> I thought the speed of Path2D came from the reuse, that is, that the 
> build cost wasn't a big concern because the speed came from reusing that 
> same object over and over at up to 60fps?

That's certainly the idea, yes.


On Mon, 10 Mar 2014, Rik Cabanier wrote:
> 
> It's decoding/re-encoding of an already constructed path.

I don't understand why you need to decode/re-encode. Just "compile" the 
path to optimal form when you use it the first time, and cache the result.


> Yes. The concern is that the re-encoding will happen every time you use 
> the path.

Why would you re-encode? If you've already used the path with a backend, 
you'll already have it in encoded form.


> We could code it in such a way that the path is retargeted when it's 
> used. It's a bit strange since fill and stroke are not supposed to 
> "change" the path.

I don't understand what would change here.


On Mon, 10 Mar 2014, Justin Novosad wrote:
> 
> Can't the implementation just perform that work lazily the first time 
> the path is rasterized, and retain the cached result for subsequent use 
> of the path object?

On Mon, 10 Mar 2014, Tab Atkins Jr. wrote:
> 
> This is also my question.  Given that generating a path for a particular 
> context isn't a magic bullet *anyway* (because the details of the 
> context can change), I don't understand why caching isn't the answer.

On Mon, 10 Mar 2014, Rik Cabanier wrote:
> 
> At usage time, the path could be retargeted to a new backend.

If the backend changes, knowing the backend at creation time doesn't help.

If it doesn't, then the cost seems to be the same either way.


> I don't think that should be done as a cached copy since that would 
> require too many resources. I will see if this is an acceptable solution 
> for mozilla.

How many resources could a path possibly take?


On Mon, 10 Mar 2014, Justin Novosad wrote:
> 
> Isn't caching ideal for that situation? In the case of re-targeting, you 
> can either replace the cached encoding, or append the new encoding to a 
> collection of cached encodings.  Both of those options seem more 
> effective than to stick to an encoding type that was baked-in at 
> construction time. It may also be great to have a heuristic to chose 
> whether to discard the previously cached re-encoding. Something like: if 
> we are re-encoding because the destination backing type changed due to a 
> resize, then discard previous encodings; if re-encoding because the path 
> is drawn to multiple canvases, then retain multiple cached encodings.

That makes sense to me.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'