[whatwg] Canvas 2D memory management

[Justin Novosad]

On Thu, Jul 18, 2013 at 12:50 PM, Ian Hickson <ian at hixie.ch> wrote:

> On Wed, 9 Jan 2013, Ashley Gullen wrote:
> >
> > Some developers are starting to design large scale games using our HTML5
> > game engine, and we're finding we're running in to memory management
> > issues.  Consider a device with 50mb of texture memory available.  A
> > game might contain 100mb of texture assets, but only use a maximum of
> > 30mb of them at a time (e.g. if there are three levels each using 30mb
> > of different assets, and a menu that uses 10mb of assets).  This game
> > ought to fit in memory at all times, but if a user agent is not smart
> > about how image loading is handled, it could run out of memory.
> >
> > [...]
> >
> > Some ideas:
> > 1) add new functions to the canvas 2D context, such as:
> > ctx.load(image): cache an image in memory so it can be immediately drawn
> > when drawImage() is first used
> > ctx.unload(image): release the image from memory
>
> The Web API tries to use garbage collection for this; the idea being that
> you load the images you need when you need them, then discard then when
> you're done, and the memory gets reclaimed when possible.
>
> We could introduce a mechanism to flush ImageBitmap objects more forcibly,
> e.g. imageBitmap.discard(). This would be a pretty new thing, though. Are
> there any browser vendors who have opinions about this?
>
> We should probably wait to see if people are able to use ImageBitmap with
> garbage collection first. Note, though, that ImageBitmap doesn't really
> add anything you couldn't do with <img> before, in the non-Worker case.
> That is, you could just create <img> elements then lose references to them
> when you wanted them GC'ed; if that isn't working today, I don't see why
> it would start working with ImageBitmap.
>

This is probably an area where most browsers could do a better job.
Browsers should be able to handle the texture memory issues automatically
without any new APIs, if they can't, then file bug reports.  If garbage
collection is not kicking-in at the right time, report it to the vendor.
ImageBitmap should provide the same kind of pinning semantics as the
suggested ctx.load/unload. However, one weakness of the current API is that
upon construction of the ImageBitmap, the browser does not know whether the
asset will be used with a GPU-accelerated rendering context or not. If this
information were available, the asset could be pre-cached on the GPU when
appropriate.  Maybe something like ctx.prefetch(image) would be appropriate
for warming up the caches.


> --
> Ian Hickson               U+1047E                )\._.,--....,'``.    fL
> http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
> Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'
>