[whatwg] LocalStorage in workers

[James Robinson]

On Wed, Sep 16, 2009 at 11:34 AM, Michael Nordman <michaeln at google.com>wrote:

>
>
> On Wed, Sep 16, 2009 at 11:24 AM, James Robinson <jamesr at google.com>wrote:
>
>> On Wed, Sep 16, 2009 at 10:53 AM, Michael Nordman <michaeln at google.com>wrote:
>>
>>>
>>>
>>> On Wed, Sep 16, 2009 at 9:58 AM, Drew Wilson <atwilson at google.com>wrote:
>>>
>>>> Jeremy, what's the use case here - do developers want workers to have
>>>> access to shared local storage with pages? Or do they just want workers to
>>>> have access to their own non-shared local storage?
>>>> Because we could just give workers their own separate WorkerLocalStorage
>>>> and let them have at it. A worker could block all the other accesses to
>>>> WorkerLocalStorage within that domain, but so be it - it wouldn't affect
>>>> page access, and we already had that issue with the (now removed?)
>>>> synchronous SQL API.
>>>>
>>>> I think a much better case can be made for WorkerLocalStorage than for
>>>> "give workers access to page LocalStorage", and the design issues are much
>>>> simpler.
>>>>
>>>
>>> Putting workers in their own storage silo doesn't really make much sense?
>>> Sure it may be simpler for browser vendors, but does that make life simpler
>>>  for app developers, or just have them scratching their heads about how to
>>> read/write the same data set from either flavor of context in their
>>> application?
>>>
>>> I see no rhyme or reason for the arbitrary barrier except for browser
>>> vendors to work around the awkward implict locks on LocalStorage (the source
>>> of much grief). Consider this... would it make sense to cordon off the
>>> databases workers vs pages can see? I would think not, and i would hope
>>> others agree.
>>>
>>
>> The difference is that the database interface is purely asynchronous
>> whereas storage is synchronous.
>>
>
> Sure... we're talking about adding an async api that allows worker to
> access a local storage repository... should such a thing exist, why should
> it not provide access to the same repository as seen by pages?
>

Not quite - Jeremy proposed giving workers access to a synchronous API
(localStorage.*) but to only allow it to be called within the context of a
callback that the UA can run when it chooses.  It's another way to approach
the implicit locking since a UA would have to, in effect, hold the storage
mutex for the duration of the callback.  The page's context could still be
blocked for an indefinite amount of time by a worker thread.

Drew suggested isolating the worker's access to a separate storage 'arena'
so that there wouldn't be shared, synchronous access between the page
context and a worker context.  This way the synchronous Storage API can be
used essentially unchanged without having to deal with the more nasty parts
of synchronization.

- James


>
>> If multiple threads have synchronous access to the same shared resource
>> then there has to be a consistency model.  ECMAScript does not provide for
>> one so it has to be done at a higher level.  Since there was not a solution
>> in the first versions that shipped, the awkward implicit locks you mention
>> were suggested as a workaround.  However it's far from clear that these
>> solve the problem and are implementable.  It seems like the only logical
>> continuation of this path would be to add explicit, blocking synchronization
>> primitives for developers to deal with - which I think everyone agrees would
>> be a terrible idea.  If you're worried about developers scratching their
>> heads about how to pass data between workers just think about happens-before
>> relationships and multi-threaded memory models.
>>
>> In a hypothetical world without synchronous access to LocalStorage/cookies
>> from workers, there is no shared memory between threads except via message
>> passing.  This can seem a bit tricky for developers but is very easy to
>> reason about and prove correctness and the absence of deadlocks.
>>
>> - James
>>
>>
>>>
>>>
>>>
>>>>
>>>> -atw
>>>>
>>>> On Tue, Sep 15, 2009 at 8:27 PM, Jonas Sicking <jonas at sicking.cc>wrote:
>>>>
>>>>> On Tue, Sep 15, 2009 at 6:56 PM, Jeremy Orlow <jorlow at chromium.org>
>>>>> wrote:
>>>>> > One possible solution is to add an asynchronous callback interface
>>>>> for
>>>>> > LocalStorage into workers.  For example:
>>>>> > function myCallback(localStorage) {
>>>>> >   localStorage.accountBalance = localStorage.accountBalance + 100;
>>>>> > }
>>>>> > executeLocalStorageCallback(myCallback);  // TODO: Make this name
>>>>> better
>>>>> >  :-)
>>>>> > The interface is simple.  You can only access localStorage via a
>>>>> callback.
>>>>> >  Any use outside of the callback is illegal and would raise an
>>>>> exception.
>>>>> >  The callback would acquire the storage mutex during execution, but
>>>>> the
>>>>> > worker's execution would not block during this time.  Of course, it's
>>>>> still
>>>>> > possible for a poorly behaving worker to do large amounts
>>>>> of computation in
>>>>> > the callback, but hopefully the fact they're executing in a callback
>>>>> makes
>>>>> > the developer more aware of the problem.
>>>>>
>>>>> First off, I agree that not having localStorage in workers is a big
>>>>> problem that we need to address.
>>>>>
>>>>> If I were designing the localStorage interface today I would use the
>>>>> above interface that you suggest. Grabbing localStorage can only be
>>>>> done asynchronously, and while you're using it, no one else can get a
>>>>> reference to it. This way there are no race conditions, but also no
>>>>> way for anyone to have to lock.
>>>>>
>>>>> So one solution is to do that in parallel to the current localStorage
>>>>> interface. Let's say we introduce a 'clientStorage' object. You can
>>>>> only get a reference to it using a 'getClientStorage' function. This
>>>>> function is available both to workers and windows. The storage is
>>>>> separate from localStorage so no need to worry about the 'storage
>>>>> mutex'.
>>>>>
>>>>> There is of course a risk that a worker grabs on to the clientStorage
>>>>> and holds it indefinitely. This would result in the main window (or
>>>>> another worker) never getting a reference to it. However it doesn't
>>>>> affect responsiveness of that window, it's just that the callback will
>>>>> never happen. While that's not ideal, it seems like a smaller problem
>>>>> than any other solution that I can think of. And the WebDatabase
>>>>> interfaces are suffering from the same problem if I understand things
>>>>> correctly.
>>>>>
>>>>> There's a couple of other interesting things we could expose on top of
>>>>> this:
>>>>>
>>>>> First, a synchronous API for workers. We could allow workers to
>>>>> synchronously get a reference to clientStorage. If someone is
>>>>> currently using clientStorage then the worker blocks until the storage
>>>>> becomes available. We could either use a callback as the above, which
>>>>> blocks until the clientStorage is acquired and only holds the storage
>>>>> until the callback exists. Or we could expose clientStorage as a
>>>>> property which holds the storage until control is returned to the
>>>>> worker eventloop, or until some explicit release API is called. The
>>>>> latter would be how localStorage is now defined, with the important
>>>>> difference that localStorage exposes the synchronous API to windows.
>>>>>
>>>>> Second, allow several named storage areas. We could add an API like
>>>>> getNamedClientStorage(name, callback). This would allow two different
>>>>> workers to simultaneously store things in a storage areas, as long as
>>>>> they don't need to use the *same* storage area. It would also allow a
>>>>> worker and the main window to simultaneously use separate storage
>>>>> areas.
>>>>>
>>>>> However we need to be careful if we add both above features. We can't
>>>>> allow a worker to grab multiple storage areas at the same time since
>>>>> that could cause deadlocks. However with proper APIs I believe we can
>>>>> avoid that.
>>>>>
>>>>> / Jonas
>>>>>
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20090916/562fcd06/attachment-0001.htm>