[whatwg] Canvas 2D memory management

[Justin Novosad]

On Thu, Jul 18, 2013 at 5:18 PM, Rik Cabanier <cabanier at gmail.com> wrote:

>
>
> On Thu, Jul 18, 2013 at 2:03 PM, Justin Novosad <junov at google.com> wrote:
>
>> On Thu, Jul 18, 2013 at 12:50 PM, Ian Hickson <ian at hixie.ch> wrote:
>>
>> > On Wed, 9 Jan 2013, Ashley Gullen wrote:
>> > >
>> > > Some developers are starting to design large scale games using our
>> HTML5
>> > > game engine, and we're finding we're running in to memory management
>> > > issues.  Consider a device with 50mb of texture memory available.  A
>> > > game might contain 100mb of texture assets, but only use a maximum of
>> > > 30mb of them at a time (e.g. if there are three levels each using 30mb
>> > > of different assets, and a menu that uses 10mb of assets).  This game
>> > > ought to fit in memory at all times, but if a user agent is not smart
>> > > about how image loading is handled, it could run out of memory.
>> > >
>> > > [...]
>> > >
>> > > Some ideas:
>> > > 1) add new functions to the canvas 2D context, such as:
>> > > ctx.load(image): cache an image in memory so it can be immediately
>> drawn
>> > > when drawImage() is first used
>> > > ctx.unload(image): release the image from memory
>> >
>> > The Web API tries to use garbage collection for this; the idea being
>> that
>> > you load the images you need when you need them, then discard then when
>> > you're done, and the memory gets reclaimed when possible.
>> >
>> > We could introduce a mechanism to flush ImageBitmap objects more
>> forcibly,
>> > e.g. imageBitmap.discard(). This would be a pretty new thing, though.
>> Are
>> > there any browser vendors who have opinions about this?
>> >
>> > We should probably wait to see if people are able to use ImageBitmap
>> with
>> > garbage collection first. Note, though, that ImageBitmap doesn't really
>> > add anything you couldn't do with <img> before, in the non-Worker case.
>> > That is, you could just create <img> elements then lose references to
>> them
>> > when you wanted them GC'ed; if that isn't working today, I don't see why
>> > it would start working with ImageBitmap.
>> >
>>
>> This is probably an area where most browsers could do a better job.
>> Browsers should be able to handle the texture memory issues automatically
>> without any new APIs, if they can't, then file bug reports.  If garbage
>> collection is not kicking-in at the right time, report it to the vendor.
>>
>
> Does the JS VM know about the image bits? It seems not since they live on
> the C++ side so the imageBitmap could look like a small object that is
> GC'ed later.
>

If there is any memory consumed by the ImageBitmap that is pinned for the
life-time of the object, then that memory needs to be declared to the JS VM
even if the data does not live in the JS heap. I know V8 and JavaScriptCore
have APIs for that and other engines probably do too, so the problem is
manageable.