[whatwg] Canvas 2D memory management

[Rik Cabanier]

On Thu, Jul 18, 2013 at 2:03 PM, Justin Novosad <junov at google.com> wrote:

> On Thu, Jul 18, 2013 at 12:50 PM, Ian Hickson <ian at hixie.ch> wrote:
>
> > On Wed, 9 Jan 2013, Ashley Gullen wrote:
> > >
> > > Some developers are starting to design large scale games using our
> HTML5
> > > game engine, and we're finding we're running in to memory management
> > > issues.  Consider a device with 50mb of texture memory available.  A
> > > game might contain 100mb of texture assets, but only use a maximum of
> > > 30mb of them at a time (e.g. if there are three levels each using 30mb
> > > of different assets, and a menu that uses 10mb of assets).  This game
> > > ought to fit in memory at all times, but if a user agent is not smart
> > > about how image loading is handled, it could run out of memory.
> > >
> > > [...]
> > >
> > > Some ideas:
> > > 1) add new functions to the canvas 2D context, such as:
> > > ctx.load(image): cache an image in memory so it can be immediately
> drawn
> > > when drawImage() is first used
> > > ctx.unload(image): release the image from memory
> >
> > The Web API tries to use garbage collection for this; the idea being that
> > you load the images you need when you need them, then discard then when
> > you're done, and the memory gets reclaimed when possible.
> >
> > We could introduce a mechanism to flush ImageBitmap objects more
> forcibly,
> > e.g. imageBitmap.discard(). This would be a pretty new thing, though. Are
> > there any browser vendors who have opinions about this?
> >
> > We should probably wait to see if people are able to use ImageBitmap with
> > garbage collection first. Note, though, that ImageBitmap doesn't really
> > add anything you couldn't do with <img> before, in the non-Worker case.
> > That is, you could just create <img> elements then lose references to
> them
> > when you wanted them GC'ed; if that isn't working today, I don't see why
> > it would start working with ImageBitmap.
> >
>
> This is probably an area where most browsers could do a better job.
> Browsers should be able to handle the texture memory issues automatically
> without any new APIs, if they can't, then file bug reports.  If garbage
> collection is not kicking-in at the right time, report it to the vendor.
>

Does the JS VM know about the image bits? It seems not since they live on
the C++ side so the imageBitmap could look like a small object that is
GC'ed later.


> ImageBitmap should provide the same kind of pinning semantics as the
> suggested ctx.load/unload. However, one weakness of the current API is that
> upon construction of the ImageBitmap, the browser does not know whether the
> asset will be used with a GPU-accelerated rendering context or not. If this
> information were available, the asset could be pre-cached on the GPU when
> appropriate.  Maybe something like ctx.prefetch(image) would be appropriate
> for warming up the caches.


That seems too implementation specific.