[whatwg] Fwd: Why can't ImageBitmap objects have width and height attributes? (and other e-mails)

[K. Gadd]

Re-sending this because the listserv silently discarded it (You guys should
fix it to actually send the notice...)

---------- Forwarded message ----------
From: K. Gadd <kg at luminance.org>
Date: Wed, Jul 17, 2013 at 6:46 PM
Subject: Re: [whatwg] Why can't ImageBitmap objects have width and height
attributes? (and other e-mails)
To: Ian Hickson <ian at hixie.ch>
Cc: whatwg at whatwg.org


Responses inline


On Wed, Jul 17, 2013 at 5:17 PM, Ian Hickson <ian at hixie.ch> wrote:

> On Tue, 18 Dec 2012, Kevin Gadd wrote:
> >
> > Is it possible to expose the width/height of an ImageBitmap, or even
> > expose all the rectangle coordinates? Exposing width/height would be
> > nice for parity with Image and Canvas when writing functions that accept
> > any drawable image source.
> >
> > Thanks for the prompt action here, this looks like a straightforward
> > solution.
>
> I've added height, width, and pixel density. Not sure what you meant by
> the other coordinates.
>

By 'the other coordinates' I mean that if you constructed it from a
subrectangle of another image (via the sx, sy, sw, sh parameters) it would
be good to expose *all* those constructor arguments. This allows you to
more easily maintain a cache of ImageBitmaps without additional bookkeeping
data.


>
> On Tue, 18 Dec 2012, Kevin Gadd wrote:
> >
> > Sorry, upon reading over the ImageBitmap part of the spec again I'm
> > confused: Why is constructing an ImageBitmap asynchronous?
>
> Because it might involve network I/O.
>
> > I thought any decoding isn't supposed to happen until drawImage, so I
> > don't really understand why this operation involves a callback and a
> > delay. Making ImageBitmap creation async means that you *cannot* use
> > this as a replacement for drawImage source rectangles unless you know
> > all possible source rectangles in advance. This is not possible for
> > many, many use cases (scrolling through a bitmap would be one trivial
> > example).
>
> Yeah, it's not supposed to be a replacement for drawImage().
>

This is why I was confused then, since I was told on this list that
ImageBitmap was a solution for the problem of drawing subrectangles of
images via drawImage (since the current specified behavior makes it
impossible to precisely draw a subrectangle). :(


>
>
> > Is it async because it supports using Video and Blob as the source?
>
> Mainly Blob, but maybe other things in the future.
>
>
> > I really love the feature set (being able to pass ImageData in is going
> > to be a huge boon - no more temporary canvases just to create images
> > from pixel data!) but if it's async-only I don't know how useful it will
> > be for the issues that led me to starting this discussion thread in the
> > first place.
>
> Can you elaborate on the specific use cases you have in mind?
>

The use case is being able to draw lots of different subrectangles of lots
of different images in a single frame.

>
>
> On Tue, 18 Dec 2012, Kevin Gadd wrote:
> >
> > How do you wait synchronously for a callback from inside
> > requestAnimationFrame?
>
> You return and wait for another frame.
>
>
> > Furthermore, wouldn't that mean returning once to the event loop for
> > each individual drawImage call you wish to make using a source rectangle
> > - so for a single scene containing lots of dynamic source rectangles you
> > could end up having to wait for dozens of callbacks.
>
> I don't understand. Why can't you prepare them ahead of time all together?
> (As in the example in the spec, for instance.)
>

You can, it's just significantly more complicated. It's not something you
can easily expose in a user-consumable library wrapper either, since it
literally alters the execution model for your entire rendering frame and
introduces a pause for every group of images that need the use of temporary
ImageBitmap instances. I'm compiling classic 2D games to JavaScript to run
in the browser, so I literally call drawImage hundreds or thousands of
times per frame, most of the calls having a unique source rectangle. I will
have to potentially construct thousands of ImageBitmaps and wait for all
those callbacks. A cache will reduce the number of constructions I have to
do per frame, but then I have to somehow balance the risk of blowing
through the entirety of the end user's memory (a very likely thing on
mobile) or create a very aggressive, manually flushed cache that may not
even have room for all the rectangles used in a given frame. Given that an
ImageBitmap creation operation may not be instantaneous this really makes
me worry that the performance consequences of creating an ImageBitmap will
make it unusable for this scenario.

(I do agree that if you're building a game from scratch for HTML5 Canvas
based on the latest rev of the API, you can probably design for this by
having all your rectangles known in advance - but there are specific
rendering primitives that rely on dynamic rectangles, like for example
filling a progress bar with a texture, tiling a texture within a window, or
scrolling a larger texture within a region. I've encountered all these in
real games.)


>
> > Each time you wait for that callback, won't any pending drawing
> > operations to your canvas get flushed, resulting in the scene slowly
> > rendering onscreen?
>
> Right, don't draw anything until you're ready to draw it all.
>
>
> On Wed, 19 Dec 2012, Robert O'Callahan wrote:
> >
> > It seems to me a synchronous constructor that creates an ImageBitmap for
> > a subimage of another ImageBitmap (but not other kind of source!) will
> > be needed.
>
> What's the use case?
>

Use case is explained above. To be clear, I think this is essential because
it is a synchronous operation (this form of ImageBitmap could potentially
not even involve a copy, though I understand if for some reason you can't
provide that) and it's an operation that is extremely common in
performance-sensitive 2D rendering. To me, the GC pressure from ImageBitmap
instances is bad enough; adding an event loop turn and a copy and
potentially another decode is just plain ridiculous. It'll force people to
go straight to WebGL, which would be a shame (especially due to the
compatibility penalty that results from that.)


>
>
> On Tue, 18 Dec 2012, Rik Cabanier wrote:
> >
> > Did you mean to say a synchronous constructor?
> > Reading the new spec (
> > http://www.whatwg.org/specs/web-apps/current-work/#imagebitmap), I still
> > fail to see how this is better than a drawimage with dx=dy=0 and
> > dw/dy=canvas width/height
>
> It's not. If you can use drawImage(), you should do so.
>

Yeah, I thought you were aware that this came up because I *can't* use
drawImage, and it turned out from discussion that it was impossible (or
undesirable) to fix the problems with drawImage.


>
> BitmapImage is mainly to have a way to send an <img> to a worker so you
> can draw images in a worker, since workers can't have <img> objects.
>

Didn't realize about the worker constraint, that makes sense.

 -kg