[whatwg] Rich Paste & DataTransfer / DataTransferItems API

[Daniel Cheng]

On Mon, Mar 14, 2011 at 20:14, Glenn Maynard <glenn at zewt.org> wrote:

> On Mon, Mar 14, 2011 at 10:47 PM, Daniel Cheng <dcheng at chromium.org>
> wrote:
> > Is there an alternate bitmap format that's likely to be widely supported
> > across different browsers? I picked PNG since if the compression proves
> to
> > be expensive, we can simply try less hard to compress the data (at the
> > extreme end, we don't compress it at all and it basically just becomes a
> > bitmap container).
>
> Yeah, but if the user can't say whether the user wants compressed data
> or not, just tweaking the compression level is always going to be bad
> for one use case (canvas) or the other (uploads).
>
> Using PNG even for uncompressed data does sound better than BMP.
> Alpha channel, 16bpp color, color management, etc. can all still
> apply.
>
> --
> Glenn Maynard
>

Isn't there an image resize API? IMO, that sounds like things that should be
handled by that API.

On Mon, Mar 14, 2011 at 20:15, Jeb Boniakowski <jebjeb at gmail.com> wrote:

> Glenn's point is a good one and the problem isn't whether the bitmap
> will be supported across the browsers, its whether the bitmap will
> make it to the system pasteboard in the first place (assuming its
> being put there from a source other than the browser.  While the
> browser could push pngs or whatever, imagine how frustrating it would
> be to users if you could paste after copying from a browser window but
> not from a random desktop app).
>
> At least on OS X, it looks like applications make sure to make copied
> images available in tiff and pict formats, presumably so every
> drag/paste target doesn't have to support converting from every image
> format (also IIRC tiff and pict are pretty simple 'buffer of pixels w/
> a header' formats, right?)
>
> So the AFAICT possibilities are:
>  - Do what Glenn said and design an async API, which is more
> complicated for the spec, implementors, and users
>  - Cheat.  e.g. monitor the system clipboard in a background thread
> and begin converting anything matching the system's image formats in
> the background/when usage is low, and then only block if you don't
> convert in time.  This is...gross, but would be simple to use.
>
> jeb.
>
> On Mon, Mar 14, 2011 at 11:47 PM, Daniel Cheng <dcheng at chromium.org>
> wrote:
> > I probably sent it with the wrong from: address.
> >
> > On Mon, Mar 14, 2011 at 19:21, Glenn Maynard <glenn at zewt.org> wrote:
> >>
> >> (Was Daniel's reply off-list?)
> >>
> >> On Mon, Mar 14, 2011 at 9:14 PM, Jeb Boniakowski <jebjeb at gmail.com>
> wrote:
> >> > On Mon, Mar 14, 2011 at 9:22 PM, Daniel Cheng <dcheng at google.com>
> wrote:
> >> >> I'm currently 60-75% complete landing the patches for image paste
> >> >> support in
> >> >> Chrome. I've chosen to expose image/png instead of a raw bitmap
> through
> >> >> event.clipboardData.items in Chrome as a Blob.
> >> >> DataTransferItem::getAsFile()
> >>
> >> That's a good starting point.  I'd suggest that a different entry
> >> point is needed to support this fully.  A couple things that come to
> >> mind:
> >>
> >> PNG compression is expensive; on a larger image it may take 5-10
> >> seconds; more for more aggressive compressors.  This wants an async
> >> API, so compression doesn't block and compression can be threaded.  (A
> >> mechanism that can support Progress Events would make a lot of sense,
> >> too--something for much later, of course, but should at least be kept
> >> in mind, since func(successCb, errorCb) interfaces don't handle that
> >> nicely.)
> >>
> >> The same thing applies to Canvas:
> >>
> >>
> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-December/029492.html
> >>
> >>
> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-February/030381.html
> >> (Ian's reply)
> >>
> >> Canvas users would also want to be able to load a pasted image without
> >> redundant compression and decompression steps.
> >>
> >> --
> >> Glenn Maynard
> >
> > Is there an alternate bitmap format that's likely to be widely supported
> > across different browsers? I picked PNG since if the compression proves
> to
> > be expensive, we can simply try less hard to compress the data (at the
> > extreme end, we don't compress it at all and it basically just becomes a
> > bitmap container).
> > It would definitely be nice if you didn't need to know the length of the
> > data when creating a Blob.
> > Daniel
>

It's actually fairly trivial to get bitmaps from the system clipboard on
most platforms (I can't speak for Linux because I haven't examined it as
much), so I'm not too worried about that.

Daniel