[whatwg] Adding SVG Filter-like functionality to Canvas 2D Context

[Hans Schmucker]

On Tue, Jul 7, 2009 at 12:15 AM, Robert O'Callahan<robert at ocallahan.org> wrote:
> On Tue, Jul 7, 2009 at 9:21 AM, <hansschmucker at gmail.com> wrote:
>>
>> On Tue, Jul 7, 2009 at 2:09 AM, hansschmucker at gmail.com> wrote:
>> > SVG Filters are a relatively easy spec, where the most important parts
>> > can be implemented in a matter of hours.
>> On Jul 6, 2009 10:54pm, Robert O'Callahan <robert at ocallahan.org> wrote:
>> > Speaking as an implementor of SVG filters, I don't believe you :-).
>>
>> I said most parts (by which I meant the actual image manipulations, which
>> are defined pretty well in the SVG spec, not the surrounding issues of when
>> to do drawing updates and so on that wouldn't apply to a Canvas anyway) :) .
>> OK, I made it sound simpler than it is, but when you get right down to it,
>> just the drawing code isn't terribly complicated.
>
> Some of the filters aren't complicated, but some are. You're also talking
> about duplicating the object model. And looking forward, if we do some
> JIT-based or GPU-based filter acceleration, duplicating that work would
> really suck. I would say the drawing code has to be shared even if the
> object model is not.
>
>> On Jul 6, 2009 10:54pm, Robert O'Callahan <robert at ocallahan.org> wrote:
>> > I agree that we need a convenient way to use filters with canvas. But I
>> > would rather not create an entirely new object model.
>> > How about a drawImageWithFilter method that takes the same set of
>> > parameters as drawImage plus a reference to an SVG element? The image is
>> > used for Source/SourceAlpha and the size of the image establishes the object
>> > bounding box for filter and filter primitive units. Since drawImage can draw
>> > a canvas, this seems completely flexible.
>>
>> I'm fearing that the dependency on SVG would bring a lot of problems if a
>> browser vendor wants to include just Canvas, like WebKit on the Palm Pre.
>
> Webkit on iPhone and I think Android, and Fennec too, support SVG. If Palm
> decides differently, that's their call and they can take the consequences. I
> am deeply averse to duplicating functionality across specs "just in case"
> someone implements one but not the other. It punishes authors and vendors of
> full-function browsers.

It may just be me, but I think that the success of Canvas, which is
way ahead of HTML5 in general, is largely due to the fact that it's
pretty much standalone. You don't have to read through hundreds of
pages of other documentation, you just read the Canvas spec and that's
pretty much it. Same for parties who want to implement it, there's
virtually no big dependency.

Of course, how you actually implement it (i.e. by sharing code with
the SVG filters) is not even something a spec is supposed to dictate,
you can share as much code as you want as long as the result is what
the spec says.

>> Also, the SVG model is pretty complex when you include all the ways to
>> cross-reference data. What would happen if a filter would want to load an
>> image? Canvas can't work asyc, so either the whole call would fail or we'd
>> lock up the browser for ages.
>
> You'd just use whatever image you've got, just like SVG filter drawing does
> normally. It's no different from doing drawImage with a partially loaded
> image.
Is this really useful for Canvas? Partial results are fine when all
you do is filter, but if the filter is part of a larger drawing
routine, then the result could be pretty undesirable. And since
there's no direct way to check for the status of the resource, all the
Canvas user could do would be trying again and again until it works
should we decide that methods should throw an error if a resource
isn't loaded yet.

>> Security issues would also increase tenfold as Canvas has a pretty simple
>> security model. If we'd allow normal SVG filters, we'd probably need to flag
>> the Canvas as "dirty", since SVG flters allow for loading of data from many
>> sources, including what's currently on the screen.
>
> You mean BackgroundImage? That makes no sense for canvas usage anyway, we'd
> just ignore it. Foreign image usage would taint the canvas, that's easy to
> do.
Nope, I meant the filter loading an SVG, loading a HTML via
foreignElement, loading another SVG via iframe, loading an image from
another server. BackgroundImage is of no use for Canvas anyway.

>> Including a dedicated filter API for Canvas would keep it simple and
>> self-contained, allowing for much easier implementation, wouldn't it?
> No, not really. Duplicating code is evil. Adding layers of abstraction so
> that the same code can be used through different interfaces is less evil,
> but it's still bad.
I think that abstraction would provide an OK compromise, but that's up
to the implementing parties.

>> It would also give us a chance to strip out SVG stuff that doesn't make
>> any sense for Canvas or is too complicated for too little benefit.
> You see that as a chance. I see it as an opportunity for endless arguments.
> And experience suggests that over time both sets of functionality will grow
> to meet all users' requirements.
But do you really want to explain to everybody who wants to use it
that the colorspace is not normal RGBA32, but some strange....
something? That the applied rectangle is actually 1.2x the given size
in any dimension? There is simply a lot of stuff in SVG that's meant
for graphics artists, not programmers. The usage would be different in
Canvas from what it is in SVG.

> BTW with an HTML5 parser you can write <svg> content directly in HTML, which
> makes it really easy to write an SVG filter to use with <canvas>.
And suddenly we have another dependency :) But seriously: What good is
a filter that is supplied somewhere in the document body to a dynamic
graphic. Why would I want to meddle with the DOM, when really all I
want to set is a local property? Maybe I could pipe the SVG in via
E4X, but that's just another dependency, that doesn't really do any
good.

> One thing I would change is SVG filters should not clip to the filter region
> or the filter primitive subregions. That's actually a change that should be
> made to SVG (and I think the SVG WG is planning to make it), but for canvas
> that should be the default.
I think I'll have to look that one up :)

--hans