[whatwg] whatwg Digest, Vol 94, Issue 44

[Charles Pritchard]

On 1/25/12 7:39 AM, whatwg-request at lists.whatwg.org wrote:
>> >  Generally, I think that often a hybrid approach to Canvas, where you draw
>> >  into multiple Canvas elements and use CSS transforms, animations (and now
>> >  filters) for positioning and effects can give you the best of both worlds...
>> >
> I disagree. I would much rather see filter functionality available to both
> Canvas and CSS. That functionality is clearly in both Canvas' and CSS'
> wheelhouses, and since they are both implemented by the browser vendor, why
> not have that low-level functionality available to both Canvas and CSS? It
> seems crazy to me to have that low-level code sitting in the browser and
> then restrict it to CSS. Why would anyone want to do that?


Implementers may run shaders on the GPU.

You don't want to have your canvas bitmap uploaded to the gpu, filtered, 
then sent back to the cpu. There are undefined issues on quality. The 
image may be compressed in transit. The CSS layer is a very much 
separated from the Canvas rasterization process. This is a good thing.

I did ask about adding filters to Canvas years ago, I think in '09. So I 
want you to know, I am supportive. I worked on a large project with 
dozens of filters and blend modes atop Canvas and WebGL.

I want to write pixel shaders in JS so as to maintain backward 
compatibility and use the pixel shader with CSS semantics.

It does not help with the low level code issue you're talking about. But 
that's an area that's always been difficult. We have no direct access to 
deflate, either, nor many of the other low-level code assets sitting in 
the browser. With deflate, we could reasonably construct RGB and Indexed 
PNGs. But we don't have it, and that's ok.

Robert O has done a lot of work on Workers running filters on an <audio> 
stream:
https://dvcs.w3.org/hg/audio/raw-file/tip/streams/StreamProcessing.html

We can extend that proposal to work on Uint8ClampedArray (previously, 
CanvasPixelArray) and/or a Uint32Array.
It may be more efficient to treat the data as a Uint32Array. Fast is 
good. It'd work on <video> and <canvas>.

JS with typed arrays, especially Uint32Array, can be optimized very well 
by the compiler, and it can run across multiple cores. It's unlikely 
that the hard coded UA implementation of a few filter effects will have 
the long-term value of script shaders.

-Charles