[whatwg] Structured clone algorithm on LocalStorage

[Brett Cannon]

On Thu, Sep 24, 2009 at 01:45, Jeremy Orlow <jorlow at chromium.org> wrote:
[SNIP]
> These are all good suggestions, but I'm not sure even that API would be
> powerful enough for developers.
> <thought exercise>
> For example, how would you implement an offline webmail app?  Well, my first
> thought is to make each email a key/value.  But we can't iterate over ranges
> of keys, so that won't work.  Maybe instead we can make each key a folder
> for our mail?  We could store all the mails in an array.  But that could be
> a huge amount of data in each key--on the order of hundreds of megabytes for
> some users.  So, to optimize, I guess we could store the emails in their own
> keys and then just store large arrays of mail keys in each folder key.  This
> would also solve the problem of gmail-like "labels" (i.e. a many to one
> relationship between folders/labels to emails).  Oh yeah, and we'd need to
> have one key that's a list of all the folders.
> Great.  But now we want to allow offline searching of our emails.  Crap.
>  Well we can efficiently search arrays with a binary search.  But that takes
> a while to update.  Well, we can implement a balanced tree for each index
> and then store it in keys.  And have one key that has a list of all our
> index keys.  This seems reasonably efficient--as long as the trees don't get
> too big.  Even if the implementation is pretty slick, once they get to a
> certain size, just loading one key into memory could take a while.
>  Especially since the entire time we're updating/reading/whatever our keys,
> we're either starving other tabs from accessing the data or blocking their
> event loop.
> </thought exercise >
> So yes: I do think that you can create at least a simple web mail client.
>  And, as long as the JavaScript engine is fast enough to handle large,
> complex data structures (without running out of memory!), I suppose you
> probably could build just about anything on top of it.
> But realistically, it seems that (at a minimum) we need to add a way to
> iterate over ranges of keys and something with multiple storage areas (as
> you suggested).  Honestly, I can't think of anything else that seems super
> important right now.  (Though I know the gmail guys would say they need full
> text search.  :-)
> If it'd be helpful, I could maybe ask some developers here at Google how
> they'd like to use LocalStorage to get some more concrete use cases.

To give a data point I can tell you all about my PhD thesis work that
involves localStorage. Basically I am looking at how to automate the
persistence of JavaScript data objects (i.e. stuff JSON supports) both
locally for performance/offline reasons, and to the cloud for
synchronization/backup reasons. For the browser side I am using
localStorage to store the objects.

I have ended up using localStorage by using keys that represent
"pointers" for objects with the values being the JSON string for that
object. This essentially lets me treat localStorage as an object
store. With some other JavaScript trickery I am able to load only the
objects I need and avoid loading the rest of the object graph,
avoiding performance costs when trying to stringify a large object
graph at once.

Talking directly to the structured clones direction, I have to say
that I personally would rather have it revert back to strings. With my
persistence hat on I would say it would be a slight convenience, but
with my localStorage-browser-compatibility-library hat on I would say
that it is going to be a huge pain in the rear to add support for the
browsers already out there for minor benefit.

So if I can build an automated data object persistence object on top
of localStorage as it is implemented in the browsers today (sans
incompatibilities) then I think your assessment, Jeremy, is true that
localStorage can be used as the storage basis for things, and w/o
structured clones to boot.

-Brett