[whatwg] Canvas 2D memory management

[Rik Cabanier]

On Wed, Jan 9, 2013 at 8:00 AM, Ashley Gullen <ashley at scirra.com> wrote:

> Some developers are starting to design large scale games using our HTML5
> game engine, and we're finding we're running in to memory management
> issues.  Consider a device with 50mb of texture memory available.  A game
> might contain 100mb of texture assets, but only use a maximum of 30mb of
> them at a time (e.g. if there are three levels each using 30mb of different
> assets, and a menu that uses 10mb of assets).  This game ought to fit in
> memory at all times, but if a user agent is not smart about how image
> loading is handled, it could run out of memory.
>
> We have a WebGL renderer which solves this by explicitly creating and
> deleting textures as necessary when switching levels, which guarantees that
> memory is managed efficiently.  It also has the additional benefit that all
> necessary textures are pre-loaded, so there's no janking during the game as
> the first drawImage() of a particular asset in the level uploads a texture.
>
> I would like to suggest memory management features for the canvas 2D
> rendering context.  By explicitly pre-loading images and releasing them at
> the end of the level we can guarantee that devices will not run out of
> memory, as well as making gameplay smoother.
>
> Some ideas:
> 1) add new functions to the canvas 2D context, such as:
> ctx.load(image): cache an image in memory so it can be immediately drawn
> when drawImage() is first used
>

Is this what you're looking for:
http://www.whatwg.org/specs/web-apps/current-work/#imagebitmap


> ctx.unload(image): release the image from memory
>

Releasing all reference to ImageBitmap should release it from memory. Are
you looking for a scheme that does not involve garbage collection?

Some of you concerns with memory management could be addressed with
WeakMaps.
Basically, you can put all your images in a WeakMap and during the draw
cycle, you pull them out and use them. If they're no longer there, it meant
that the garbage collector has kicked in to free up memory and you need to
reload (which will unfortunately cause a jank). I'm unsure how smart the
garbage collector is because you probably want the items in the WeakMaps to
be deleted last.


>
> 2) we can drawImage() every image on startup to force lazy-loading browsers
> to load everything that will be used, but there's still no way to indicate
> which images should be released at the end of a level.  This could be left
> for the browser to determine (perhaps releasing by least-recently-used),
> but perhaps this should be required in the specification?
>
> 3) leave current behavior as it is and suggest WebGL for this type of
> application
>
> My preference is option 1, but I don't know if this works for all use cases
> and will work nicely with implementations.  Any thoughts?
>
> Ashley Gullen
> Scirra.com
>