[whatwg] An API to resize and rotate images client-side

[Ian Hickson]

On Thu, 20 May 2010, David Levin wrote:
>
> Twice when this was brought up on whatwg developers out of the blue
> mentioned that the image resizing was a useful thing for them (once early in
> this thread and once long ago when canvas in workers was brought up).
> 
> In addition to that anecdotal evidence, here are several other places this
> comes up which I can list quickly:
> 
> - For example, take Facebook. If I upload a huge photo to Facebook, it 
> seems to upload the whole thing and then resizes it on the server (down 
> to something much smaller than 1600 X 1200).
>
> - This is similar for other social sites like dating sites or Orkut that 
> only allow a maximum size of photo. Typically, either they force the 
> user to resize the image (which is a horrible experience) or they resize 
> the image on the client using gears (with workers and canvas) or flash, 
> etc. (or canvas but for more than one browser that may hang the UI).
>
> - Similarly Gmail now allows dragging images into email 
> (http://gmailblog.blogspot.com/2010/05/drag-images-into-messages.html). 
> The full resolution image isn't necessary for this. It would be better 
> to have a resized image.
>
> - Something like Google Docs or Wave which show real time participation 
> of other people typing would benefit from getting a thumbnail of an 
> inserted image to other people in the conversation. (One could envision 
> this for any real time chat/communication website.)
>
> - When you upload photos to picasaweb from the Picasa client, it offers 
> to resize them to 1600X1200 before uploading them. Also, it offers an 
> option to upload a thumbnail first before uploading the bigger picture, 
> so the album can appear even quicker (just at a really low resolution). 
> Ideally, a website could do something similar.

On Tue, 25 May 2010, David Levin wrote:
>
> http://webkit.org/demos/canvas-perf/canvas.html
> 
> Firefox 3.7a4 (no D2D)
> 
> Direct image copy: 39ms
> Indirect copy with (via ImageData): 160ms
> Copy with 2x scale: 646.5ms
> Copy with 0.5x scale: 42.5ms
> Copy with rotate: 358ms
> 
> Firefox 3.7a4 (D2D)
> 
> Direct image copy: 115ms
> Indirect copy with (via ImageData): 365.5ms
> Copy with 2x scale: 246ms
> Copy with 0.5x scale: 48.5ms
> Copy with rotate: 100.5ms
> 
> Chrome 4.1.249.1064 (45376)
> 
> Direct image copy: 32.5ms
> Indirect copy with (via ImageData): 207.5ms
> Copy with 2x scale: 378.5ms
> Copy with 0.5x scale: 27.5ms
> Copy with rotate: 367ms
> 
> While the GPU does help in some scenarios, unfortunately it must still 
> take some time to do its work, so it doesn't enable us to do sync apis 
> that don't hang the UI.

The logical conclusion is that we should make one of the following choices:

 1. Provide dedicated asynchronous APIs for this use case.
    For example, a method to go from an image URL to a Blob representing 
    that image at a different size and/or rotation.

 2. Provide generic APIs for that can handle this use case amongst others.
    For example, porting the 2D canvas API to workers.

 3. Not address this use case (yet).

For #1, my preference would be to add two methods to <img>, one to make 
<img> resize and rotate the image and then fire 'load' again, and one to 
obtain the current image as a Blob, much like toDataURL() on <canvas> 
(where a similar toBlob() would also be useful).

For #2, we'd need to provide an Image object (a non-DOM version of 
HTMLImageElement) and a Canvas object (a non-DOM version of 
HTMLCanvasElement) in workers.

But unless we have a critical mass of browser vendors agreed on one of 
these approaches, we have to default to #3. Currently it seems only the 
Chrome team is especially interested in either #1 or #2.

My recommendation, in the absence of enthusiasm from other browser 
vendors, would be for the Chrome team to experiment with #1 or #2. If I 
have misread the current situation and there is a willingness to implement 
#1 or #2 in more browsers, then please let me know. I'd be happy to spec 
one of those options. (#1 would be easy to do; #2 might take longer, since 
it is significantly more work.)

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'