[whatwg] LocalStorage in workers

[Jonas Sicking]

On Wed, Sep 16, 2009 at 2:56 PM, Jeremy Orlow <jorlow at chromium.org> wrote:
> On Wed, Sep 16, 2009 at 2:27 PM, Jonas Sicking <jonas at sicking.cc> wrote:
>>
>> On Wed, Sep 16, 2009 at 12:58 PM, Drew Wilson <atwilson at google.com> wrote:
>> > I'm saying that an async API is overkill and unwieldy if all you need is
>> > WorkerLocalStorage.
>> > If you're going to route your localstorage access through an async API
>> > anyway, then you might as well proxy it to the parent page - there's
>> > very
>> > little advantage to doing it otherwise, other than access to lexically
>> > scoped resources from within your callback.
>>
>> Actually, there's a pretty big difference. With the current state of
>> affairs, if a worker wants to make a computation based on values in
>> the localStorage, and store the result in localStorage, this is
>> extremely hard.
>>
>> For example, say that a worker want to perform the following operation:
>>
>> localStorage.result = F(localStorage.n);
>>
>> where F(n) is the n:th value in the Fibonacci sequence.
>>
>> To do this today the worker would first have to call the main window
>> to get the localStoreage.n value. It could then calculate the result
>> of F(localStorage.n). It would then send a message to the main window
>> to store the result in localStorage.result. However in the meantime
>> localStorage.n might have changed, which causes an inconsistent state.
>>
>> So instead the worker has to send both the value of localStorage.n as
>> well as the result to the window. The window can then check if
>> localStorage.n has changed. If it has changed, the window has to send
>> the new value back to the worker, and then the worker has to redo its
>> calculation.
>>
>> This has several problems. It's bug prone since the developer might
>> not realize the race condition. It's very hard to do correctly. And
>> even when done correctly risks wasting a lot of cycles.
>>
>> An alternative solution is to do all calculations in the main window,
>> which has synchronous access to localStorage. But the whole point of a
>> worker is to avoid having to do heavy work in the window.
>>
>> However, with the solution Jeremy proposed, calculating the above
>> algorithm can be done in the worker after the worker while the worker
>> is inside the callback and thus have synchronous access to
>> localStorage.
>>
>> Say that instead of calculating Fibonacci numbers, we were storing a
>> database of emails in localStorage, and using a worker to synchronize
>> that database to a server. In this case it seems extermely complex to
>> have to communicate asynchronously through the window and deal with
>> race conditions where the user is modifying the email database at the
>> same time.
>
> True.
> The problem is that some page from the same origin might also try to access
> LocalStorage.  If it does, it'll block the entire event loop until the
> worker is finished.  I can't think of how to "fix" this in a way that's not
> racy.  My originally proposal was written in the hope that developers would
> be more cautious since they're doing things inside an async callback, but
> the more I think about it, the more I think this isn't realistic.
> I think we have 3 options:
> 1) Create a LocalStorage like API that can only be accessed in an async way
> via pages (kind of like WebDatabase).
> 2) Remove any atomicity/consistency guarantees from synchronous LocalStorage
> access within pages (like IE8 currently does) and add an async interface for
> when pages do need atomicity/consistency.
> 3) Come up with a completely different storage API that all the browser
> vendors are willing to implement that only allows Async access from within
> pages.  WebSimpleDatabase might be a good starting point for this.
>
> 1 is probably the simplest to implement, but it seems pretty hacky and it's
> likely not powerful enough for many advanced web apps (offline web mail
> would be an example).  If we do 2, many (most?) web developers will just use
> the sync interface and write racy apps.  3 will take the longest time to do,
> but is definitely the best long term solution.

I think 2 is right out. 1 is what we should have done in the first
place if we had thought about the multiple processes thing. The only
thing that's bad about 1 is that we're creating two extremely similar
features. So I'd say 1 is unfortunate rather than hacky.

3 is is something that I personally think we should do no matter what
as I'm not a big fan of the current SQL interface. But I wonder if we
might want to do 1 anyway. After all, localStorage and the SQL APIs
were both suggested. Presumably to allow localStorage to handle the
simple cases and SQL to handle the more complex ones.

/ Jonas