[whatwg] Structured clone algorithm on LocalStorage

[Darin Fisher]

On Fri, Oct 2, 2009 at 8:08 PM, Jonas Sicking <jonas at sicking.cc> wrote:

> On Wed, Sep 30, 2009 at 10:11 PM, Darin Fisher <darin at chromium.org> wrote:
>
>> On Tue, Sep 29, 2009 at 11:48 PM, Jonas Sicking <jonas at sicking.cc> wrote:
>>
>>> On Tue, Sep 29, 2009 at 12:19 AM, Darin Fisher <darin at chromium.org>
>>> wrote:
>>> > On Thu, Sep 24, 2009 at 11:57 PM, Jonas Sicking <jonas at sicking.cc>
>>> wrote:
>>> >>
>>> >> On Thu, Sep 24, 2009 at 9:04 PM, Darin Fisher <darin at chromium.org>
>>> wrote:
>>> >> > On Thu, Sep 24, 2009 at 4:43 PM, Jonas Sicking <jonas at sicking.cc>
>>> wrote:
>>> >> >>
>>> >> >> On Thu, Sep 24, 2009 at 10:52 AM, Darin Fisher <darin at chromium.org
>>> >
>>> >> >> wrote:
>>> >> >> > On Thu, Sep 24, 2009 at 10:40 AM, Jonas Sicking <jonas at sicking.cc
>>> >
>>> >> >> > wrote:
>>> >> >> >>
>>> >> >> >> On Thu, Sep 24, 2009 at 1:17 AM, Darin Fisher <
>>> darin at chromium.org>
>>> >> >> >> wrote:
>>> >> >> >> > On Thu, Sep 24, 2009 at 12:20 AM, Jonas Sicking
>>> <jonas at sicking.cc>
>>> >> >> >> > wrote:
>>> >> >> >> >>
>>> >> >> >> >> On Wed, Sep 23, 2009 at 10:19 PM, Darin Fisher
>>> >> >> >> >> <darin at chromium.org>
>>> >> >> >> >> wrote:
>>> >
>>> > ... snip ...
>>> >
>>> >>
>>> >> >> >> >> > multi-core is the future.  what's the opposite of
>>> fine-grained
>>> >> >> >> >> > locking?
>>> >> >> >> >> >  it's not good ;-)
>>> >> >> >> >> > the implicit locking mechanism as spec'd is super lame.
>>> >> >> >> >> >  implicitly
>>> >> >> >> >> > unlocking under
>>> >> >> >> >> > mysterious-to-the-developer circumstances!  how can that be
>>> a
>>> >> >> >> >> > good
>>> >> >> >> >> > thing?
>>> >> >> >> >> > storage.setItem("y",
>>> >> >> >> >> >
>>> function_involving_implicit_unlocking(storage.getItem("x")));
>>> >> >> >> >>
>>> >> >> >> >> I totally agree on all points. The current API has big
>>> >> >> >> >> imperfections.
>>> >> >> >> >> However I haven't seen any workable counter proposals so far,
>>> and
>>> >> >> >> >> I
>>> >> >> >> >> honestly don't believe there are any as long as our goals
>>> are:
>>> >> >> >> >>
>>> >> >> >> >> * Don't break existing users of the current implementations.
>>> >> >> >> >> * Don't expose race conditions to the web.
>>> >> >> >> >> * Don't rely on authors getting explicit locking mechanisms
>>> >> >> >> >> right.
>>> >> >> >> >>
>>> >> >> >> >
>>> >> >> >> > The current API exposes race conditions to the web.  The
>>> implicit
>>> >> >> >> > dropping of the storage lock is that.  In Chrome, we'll have
>>> to
>>> >> >> >> > drop
>>> >> >> >> > an existing lock whenever a new lock is acquired.  That can
>>> happen
>>> >> >> >> > due to a variety of really odd cases (usually related to
>>> nested
>>> >> >> >> > loops
>>> >> >> >> > or nested JS execution), which will be difficult for
>>> developers to
>>> >> >> >> > predict, especially if they are relying on third-party JS
>>> >> >> >> > libraries.
>>> >> >> >> > This issue seems to be discounted for reasons I do not
>>> understand.
>>> >> >> >>
>>> >> >> >> I don't believe we've heard about this before, so that would be
>>> the
>>> >> >> >> reason it hasn't been taken into account.
>>> >> >> >>
>>> >> >> >> So you're saying that chrome would be unable implement the
>>> current
>>> >> >> >> storage mutex as specified in spec? I.e. one that is only
>>> released
>>> >> >> >> at
>>> >> >> >> the explicit points that the spec defines? That seems like a
>>> huge
>>> >> >> >> problem.
>>> >> >> >
>>> >> >> > No, no... my point is that to the application developer, those
>>> >> >> > "explicit"
>>> >> >> > points will appear quite implicit and mysterious.  This is why I
>>> >> >> > called
>>> >> >> > out third-party JS libraries.  One day, a function that you are
>>> using
>>> >> >> > might transition to scripting a plugin, which might cause a
>>> nested
>>> >> >> > loop, which could then force the lock to be released.  As a
>>> >> >> > programmer,
>>> >> >> > the unlocking is not explicit or predictable.
>>> >> >>
>>> >> >> Ah, indeed, this is a problem. However the unfortunate fact remains
>>> >> >> that so far no other workable solution has been proposed.
>>> >> >
>>> >> > OK, so we agree that the current solution doesn't meet the goals you
>>> >> > stated above :-(
>>> >>
>>> >> Well, it addresses them as long as users are aware of the risk, and
>>> >> properly document weather their various library functions will release
>>> >> the lock or not. However I agree that it's unlikely that they will do
>>> >> so correctly.
>>> >
>>> > I thought the point of not having lock APIs was that users shouldn't
>>> have
>>> > to understand locks ;-)  The issue I've raised here is super subtle.
>>>  We
>>> > have not succeeded in avoiding subtlety!
>>>
>>> I think we're mostly in agreement. What I'm not sure about is what you
>>> are proposing we do with localStorage? Remove it from the spec? Change
>>> the API? Something else?
>>>
>>>
>> I'm glad we agree.
>>
>> I'm not sure what we should do.  It seems like there is a "legacy API"
>>  argument for sticking with the current proposal even though it is flawed
>> and
>> HTML5 is not yet final.  (It has also not been implemented by browsers for
>> very long.)  Stated that way, it sounds like a weak argument for
>> preserving
>> the API as is, and we should just fix it to be better.
>>
>> My understanding is that removal is not a popular position.  However,
>> given
>> that more browsers are moving to be multi-process, I have to say that I'm
>> a
>> bit surprised there isn't more support for ditching the current
>> localStorage
>> API.
>>
>
> You're preaching to the choir :) I'd recommend talking to apple and
> microsoft directly. I don't know what their plans are regarding all this.
>

Fair enough :-)



>
>
>>
>>
>>>  >> >> > Moreover, there are other examples which have been discussed on
>>> the
>>> >> >> > list.  There are some DOM operations that can result in a frame
>>> >> >> > receiving
>>> >> >> > a DOM event synchronously.  That can result in a nesting of
>>> storage
>>> >> >> > locks,
>>> >> >> > which can force us to have to implicitly unlock the outermost
>>> lock to
>>> >> >> > avoid
>>> >> >> > deadlocks.  Again, the programmer will have very poor visibility
>>> into
>>> >> >> > when
>>> >> >> > these things can happen.
>>> >> >>
>>> >> >> So far I don't think it has been shown that these events need to be
>>> >> >> synchronous. They all appear to be asynchronous in gecko, and in
>>> the
>>> >> >> case of different-origin frames, I'm not even sure there's a way
>>> for
>>> >> >> pages to detect if the event was fired asynchronously or not.
>>> >> >
>>> >> > IE and WebKit dispatch some of them synchronously.  It's hard to say
>>> >> > which
>>> >> > is correct or if it causes any web compat isues.  I'm also not sure
>>> that
>>> >> > we
>>> >> > have covered all of the cases.
>>> >>
>>> >> It still seems to me that it's extremely unlikely that pages depend on
>>> >> cross origin events to fire synchronously. I can't even think of a way
>>> >> to test if a browser dispatches these events synchronously or not. Can
>>> >> you?
>>> >
>>> > i agree that it seems uncommon.  maybe there could be some odd app that
>>> > does something after resizing an iframe that could be dependent on the
>>> > event handler setting some data field.  this kind of thing is probably
>>> even
>>> > less common in the cross-origin case.
>>>
>>> But how would you read that data field in the cross-origin frame? I
>>> think it might be possible, but extremely hard.
>>>
>>>
>> Yeah.
>>
>> My concern is simply that I cannot prove that I don't have to worry about
>> this
>> problem.  Future web APIs might also inadvertently make matters worse.
>>
>
> I agree it's not ideal, but at the same time I don't think that not
> allowing synchronous cross-origin APIs is a huge burden. You campaigned
> heavily against that when we were designing postMessage for wholly other
> reasons. I would imagine those reasons will hole true no matter what.
>

Agreed.  That's a good point.  In that case, I was concerned about stack
depth.  The same issue might apply here.  Hmm...


...snip...

>
>>> >> >> Not quite sure I follow your proposal. How would you for example
>>> >> >> increase the value of a property by one without risking race
>>> >> >> conditions? Or keep two values in different properties in sync?
>>> I.e.
>>> >> >> so that if you update one always update the other, so that they
>>> never
>>> >> >> have different values.
>>> >> >>
>>> >> >> / Jonas
>>> >> >
>>> >> >
>>> >> > Easy.  Just like with database, the transaction is the storage lock.
>>> >> >  Any
>>> >> > storage
>>> >> > operation performed on that transaction are done atomically.
>>>  However,
>>> >> > all
>>> >> > storage
>>> >> > operations are asynchronous.  You basically string together
>>> asynchronous
>>> >> > storage
>>> >> > operations by using the same transaction for each.
>>> >> > We could add methods to get/set multiple items at once to simplify
>>> life
>>> >> > for
>>> >> > the coder.
>>> >>
>>> >> I think I still don't understand your proposal, could you give some
>>> >> code examples?
>>> >>
>>> >
>>> >
>>> > ripping off database:
>>> > interface ValueStorage {
>>> >   void transaction(in DOMString namespace, in
>>> > ValueStorageTransactionCallback callback);
>>> > };
>>> > interface ValueStorageTransactionCallback {
>>> >   void handleEvent(in ValueStorageTransaction transaction);
>>> > };
>>> > interface ValueStorageTransaction {
>>> >   void readValue(in DOMString name, in ValueStorageReadCallback
>>> callback);
>>> >   void writeValue(in DOMString name, in DOMString value);
>>> > };
>>> > interface ValueStorageReadCallback {
>>> >   void handleEvent(in ValueStorageTransaction transaction, in DOMString
>>> > value);
>>> > };
>>> > then, to use these interfaces, you could implement thread-safe
>>> increment:
>>> > window.localStorage.transaction("slice", function(transaction) {
>>> >   transaction.readValue("foo", function(transaction, fooValue) {
>>> >     transaction.writeValue("foo", ++fooValue);
>>> >   })
>>> > })
>>> > to fetch multiple values, you could do this:
>>> > var values = [];
>>> > var numValues = 10;
>>> > function readNextValue(transaction) {
>>> >   if (values.length == numValues)
>>> >    return;  // done!
>>> >   var index = values.length;
>>> >   transaction.readValue("value" + index, function(transaction, value) {
>>> >     values.push(value);
>>> >     readNextValue(transaction);
>>> >   })
>>> > }
>>> > window.localStorage.transaction("slice", readNextValue);
>>> > This has the property that all IO is non-blocking and the "lock" is
>>> held
>>> > only
>>> > for a very limited scope.  The programmer is however free to extend the
>>> > life of the lock as needed.
>>>
>>> What do you mean by that the "lock" is held for only a very limited
>>> scope? You still want to prevent modifications for as long as the
>>> transaction is being used right? I.e. no modifications can happen
>>> between the read and the write in the first example, and between the
>>> different reads in the second.
>>>
>>
>> Yes.  I only meant that the programmer doesn't have to call a special
>> function to close the transaction.  It closes by virtue of the last
>> handleEvent
>> call referring to the transaction returning.
>>
>
> So wouldn't you implement this transaction using a lock? To prevent other
> pages from accessing the localStorage?
>
>
Yes, but it wouldn't need to be a normal mutex if that's what you mean.  You
could just defer callbacks until the transaction completes.  It is purely
asynchronous locking.

-Darin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20091002/704e585f/attachment-0002.htm>