<div class="gmail_quote">On Thu, Sep 24, 2009 at 10:40 AM, Jonas Sicking <span dir="ltr"><jonas@sicking.cc></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div><div></div><div class="h5">On Thu, Sep 24, 2009 at 1:17 AM, Darin Fisher <<a href="mailto:darin@chromium.org">darin@chromium.org</a>> wrote:<br>
> On Thu, Sep 24, 2009 at 12:20 AM, Jonas Sicking <jonas@sicking.cc> wrote:<br>
>><br>
>> On Wed, Sep 23, 2009 at 10:19 PM, Darin Fisher <<a href="mailto:darin@chromium.org">darin@chromium.org</a>> wrote:<br>
>> ><br>
>> ><br>
>> > On Wed, Sep 23, 2009 at 8:10 PM, Jonas Sicking <jonas@sicking.cc> wrote:<br>
>> >><br>
>> >> On Wed, Sep 23, 2009 at 3:29 PM, Jeremy Orlow <<a href="mailto:jorlow@chromium.org">jorlow@chromium.org</a>><br>
>> >> wrote:<br>
>> >> > On Wed, Sep 23, 2009 at 3:15 PM, Jonas Sicking <jonas@sicking.cc><br>
>> >> > wrote:<br>
>> >> >><br>
>> >> >> On Wed, Sep 23, 2009 at 2:53 PM, Brett Cannon <<a href="mailto:brett@python.org">brett@python.org</a>><br>
>> >> >> wrote:<br>
>> >> >> > On Wed, Sep 23, 2009 at 13:35, Jeremy Orlow <<a href="mailto:jorlow@chromium.org">jorlow@chromium.org</a>><br>
>> >> >> > wrote:<br>
>> >> >> >> What are the use cases for wanting to store data beyond strings<br>
>> >> >> >> (and<br>
>> >> >> >> what<br>
>> >> >> >> can be serialized into strings) in LocalStorage? I can't think<br>
>> >> >> >> of<br>
>> >> >> >> any<br>
>> >> >> >> that<br>
>> >> >> >> outweigh the negatives:<br>
>> >> >> >> 1) From previous threads, I think it's fair to say that we can<br>
>> >> >> >> all<br>
>> >> >> >> agreed<br>
>> >> >> >> that LocalStorage is a regrettable API (mainly due to its<br>
>> >> >> >> synchronous<br>
>> >> >> >> nature). If so, it seems that making it more powerful and thus<br>
>> >> >> >> more<br>
>> >> >> >> attractive to developers is just asking for trouble. After all,<br>
>> >> >> >> the<br>
>> >> >> >> more<br>
>> >> >> >> people use it, the more lock contention there'll be, and the more<br>
>> >> >> >> browser UI<br>
>> >> >> >> jank users will be sure to experience. This will also be worse<br>
>> >> >> >> because<br>
>> >> >> >> it'll be easier for developers to store large objects in<br>
>> >> >> >> LoaclStorage.<br>
>> >> >> >> 2) As far as I can tell, there's no where else in the spec where<br>
>> >> >> >> you<br>
>> >> >> >> have<br>
>> >> >> >> to serialize structured clone(able) data to disk. Given that<br>
>> >> >> >> LocalStorage<br>
>> >> >> >> is supposed to throw an exception if any ImageData is contained<br>
>> >> >> >> and<br>
>> >> >> >> since<br>
>> >> >> >> File and FileData objects are legal, it seems as though making<br>
>> >> >> >> LocalStorage<br>
>> >> >> >> handle structured clone data has a fairly high cost to<br>
>> >> >> >> implementors.<br>
>> >> >> >> Not to<br>
>> >> >> >> mention that disallowing ImageData in only this one case is not<br>
>> >> >> >> intuitive.<br>
>> >> >> >> I think allowing structured clone(able) data in LocalStorage is a<br>
>> >> >> >> big<br>
>> >> >> >> mistake. Enough so that, if SessionStorage and LocalStorage<br>
>> >> >> >> can't<br>
>> >> >> >> diverge<br>
>> >> >> >> on this issue, it'd be worth taking the power away from<br>
>> >> >> >> SessionStorage.<br>
>> >> >> >> J<br>
>> >> >> ><br>
>> >> >> > Speaking from experience, I have been using localStorage in my PhD<br>
>> >> >> > thesis work w/o any real need for structured clones (I would have<br>
>> >> >> > used<br>
>> >> >> > Web Database but it isn't widely used yet and I was not sure if it<br>
>> >> >> > was<br>
>> >> >> > going to make the cut in the end). All it took to come close to<br>
>> >> >> > simulating structured clones now was to develop my own<br>
>> >> >> > compatibility<br>
>> >> >> > wrapper for localStorage (<a href="http://realstorage.googlecode.com" target="_blank">http://realstorage.googlecode.com</a> for<br>
>> >> >> > those<br>
>> >> >> > who care) and add setJSONObject() and getJSONObject() methods on<br>
>> >> >> > the<br>
>> >> >> > wrapper. Works w/o issue.<br>
>> >> >><br>
>> >> >> Actually, this seems like a prime reason *to* add structured storage<br>
>> >> >> support. Obviously string data wasn't enough for you so you had to<br>
>> >> >> write extra code in order to work around that. If structured clones<br>
>> >> >> had been natively supported you both would have had to write less<br>
>> >> >> code, and the resulting algorithms would have been faster. Faster<br>
>> >> >> since the browser can serialize/parser to/from a binary internal<br>
>> >> >> format faster than to/from JSON through the JSON serializer/parser.<br>
>> >> ><br>
>> >> > Yes, but since LocalStorage is already widely deployed, authors are<br>
>> >> > stuck<br>
>> >> > with the the structured clone-less version of LocalStorage for a very<br>
>> >> > long<br>
>> >> > time. So the only way an app can store anything that can't be<br>
>> >> > JSONified<br>
>> >> > is<br>
>> >> > to break backwards compatibility.<br>
>> >> ><br>
>> >> ><br>
>> >> ><br>
>> >> > On Wed, Sep 23, 2009 at 3:11 PM, Jonas<br>
>> >> > Sicking <jonas@sicking.cc> wrote:<br>
>> >> >><br>
>> >> >> On Wed, Sep 23, 2009 at 1:35 PM, Jeremy Orlow <<a href="mailto:jorlow@chromium.org">jorlow@chromium.org</a>><br>
>> >> >> wrote:<br>
>> >> >> > What are the use cases for wanting to store data beyond strings<br>
>> >> >> > (and<br>
>> >> >> > what<br>
>> >> >> > can be serialized into strings) in LocalStorage? I can't think of<br>
>> >> >> > any<br>
>> >> >> > that<br>
>> >> >> > outweigh the negatives:<br>
>> >> >> > 1) From previous threads, I think it's fair to say that we can<br>
>> >> >> > all<br>
>> >> >> > agreed<br>
>> >> >> > that LocalStorage is a regrettable API (mainly due to its<br>
>> >> >> > synchronous<br>
>> >> >> > nature). If so, it seems that making it more powerful and thus<br>
>> >> >> > more<br>
>> >> >> > attractive to developers is just asking for trouble. After all,<br>
>> >> >> > the<br>
>> >> >> > more<br>
>> >> >> > people use it, the more lock contention there'll be, and the more<br>
>> >> >> > browser UI<br>
>> >> >> > jank users will be sure to experience. This will also be worse<br>
>> >> >> > because<br>
>> >> >> > it'll be easier for developers to store large objects in<br>
>> >> >> > LoaclStorage.<br>
>> >> >> > 2) As far as I can tell, there's no where else in the spec where<br>
>> >> >> > you<br>
>> >> >> > have<br>
>> >> >> > to serialize structured clone(able) data to disk. Given that<br>
>> >> >> > LocalStorage<br>
>> >> >> > is supposed to throw an exception if any ImageData is contained<br>
>> >> >> > and<br>
>> >> >> > since<br>
>> >> >> > File and FileData objects are legal, it seems as though making<br>
>> >> >> > LocalStorage<br>
>> >> >> > handle structured clone data has a fairly high cost to<br>
>> >> >> > implementors.<br>
>> >> >> > Not to<br>
>> >> >> > mention that disallowing ImageData in only this one case is not<br>
>> >> >> > intuitive.<br>
>> >> >> > I think allowing structured clone(able) data in LocalStorage is a<br>
>> >> >> > big<br>
>> >> >> > mistake. Enough so that, if SessionStorage and LocalStorage can't<br>
>> >> >> > diverge<br>
>> >> >> > on this issue, it'd be worth taking the power away from<br>
>> >> >> > SessionStorage.<br>
>> >> >><br>
>> >> >> Despite localStorage unfortunate locking contention problem, it's<br>
>> >> >> become quite a popular API. It's also very successful in terms of<br>
>> >> >> browser deployment since it's available in at least latest versions<br>
>> >> >> of<br>
>> >> >> IE, Safari, Firefox, and Chrome. Don't know about support in Opera?<br>
>> >> ><br>
>> >> > The more popular it becomes, the more it's going to hurt UA<br>
>> >> > developers,<br>
>> >> > web<br>
>> >> > developers, and users. I don't see why this is an argument for<br>
>> >> > making<br>
>> >> > it<br>
>> >> > more powerful.<br>
>> >><br>
>> >> How will it hurt UA developers? I think we're stuck forever to<br>
>> >> implement the locking mechanism. Adding more datatypes to the API<br>
>> >> doesn't mean that we'll have to implement it more.<br>
>> ><br>
>> ><br>
>> > multi-core is the future. what's the opposite of fine-grained locking?<br>
>> > it's not good ;-)<br>
>> > the implicit locking mechanism as spec'd is super lame. implicitly<br>
>> > unlocking under<br>
>> > mysterious-to-the-developer circumstances! how can that be a good<br>
>> > thing?<br>
>> > storage.setItem("y",<br>
>> > function_involving_implicit_unlocking(storage.getItem("x")));<br>
>><br>
>> I totally agree on all points. The current API has big imperfections.<br>
>> However I haven't seen any workable counter proposals so far, and I<br>
>> honestly don't believe there are any as long as our goals are:<br>
>><br>
>> * Don't break existing users of the current implementations.<br>
>> * Don't expose race conditions to the web.<br>
>> * Don't rely on authors getting explicit locking mechanisms right.<br>
>><br>
><br>
> The current API exposes race conditions to the web. The implicit<br>
> dropping of the storage lock is that. In Chrome, we'll have to drop<br>
> an existing lock whenever a new lock is acquired. That can happen<br>
> due to a variety of really odd cases (usually related to nested loops<br>
> or nested JS execution), which will be difficult for developers to<br>
> predict, especially if they are relying on third-party JS libraries.<br>
> This issue seems to be discounted for reasons I do not understand.<br>
<br>
</div></div>I don't believe we've heard about this before, so that would be the<br>
reason it hasn't been taken into account.<br>
<br>
So you're saying that chrome would be unable implement the current<br>
storage mutex as specified in spec? I.e. one that is only released at<br>
the explicit points that the spec defines? That seems like a huge<br>
problem.<br></blockquote><div><br></div><div>No, no... my point is that to the application developer, those "explicit"</div><div>points will appear quite implicit and mysterious. This is why I called</div><div>
out third-party JS libraries. One day, a function that you are using</div><div>might transition to scripting a plugin, which might cause a nested</div><div>loop, which could then force the lock to be released. As a programmer,</div>
<div>the unlocking is not explicit or predictable.</div><div><br></div><div>Moreover, there are other examples which have been discussed on the</div><div>list. There are some DOM operations that can result in a frame receiving</div>
<div>a DOM event synchronously. That can result in a nesting of storage locks,</div><div>which can force us to have to implicitly unlock the outermost lock to avoid</div><div>deadlocks. Again, the programmer will have very poor visibility into when</div>
<div>these things can happen.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div><div></div><div class="h5"><br>
>> But, as imperfect as the current API is, I think the following is a<br>
>> decent way forward:<br>
>><br>
>> * Allow pages that want the convenience of localStorage to use it. For<br>
>> multi-process browsers this will mean poor UI *for pages that use<br>
>> localStorage*. Especially when said pages hold on to localStorage for<br>
>> a long time.<br>
>> * Add alternative APIs that don't suffer from the same problems. More<br>
>> below.<br>
>><br>
>> >> > In addition, this argument assumes that Microsoft (and other UAs)<br>
>> >> > will<br>
>> >> > implement the structured clone version of LocalStorage. Has anyone<br>
>> >> > (or<br>
>> >> > can<br>
>> >> > anyone) from Microsoft comment on this?<br>
>> >><br>
>> >> Given that I've never heard microsoft commit to a webstandard, ever, I<br>
>> >> doubt that we'll hear anything here. Or that the lack of hearing<br>
>> >> anything means we can draw any conclusions.<br>
>> >><br>
>> >> > This is not a small feature to add. Yes, it's smaller than creating<br>
>> >> > a<br>
>> >> > new<br>
>> >> > storage mechanism (that everyone is willing to adopt), but I still<br>
>> >> > think<br>
>> >> > that's what we should be looking at. Rather than polishing a turd.<br>
>> >><br>
>> >> I do think that localStorage is a decent API that developers will want<br>
>> >> to, and should, use. I think looking into adding a async accessor to<br>
>> >> get a storage object so that people can use an localStorage-like API<br>
>> >> while avoiding risks of blocking. This would also allow sharing data<br>
>> >> between worker threads and the main window.<br>
>> ><br>
>> > i think the async callback to get a storage object is an improvement,<br>
>> > but<br>
>> > i'm not sure that it addresses all of the problems. for example, if a<br>
>> > worker<br>
>> > wants to read values from storage, compute, and then put a value into<br>
>> > storage, it would probably do all of this from the storage callback.<br>
>> > that<br>
>> > would result in holding the lock for a long time, which would lock out<br>
>> > any<br>
>> > other threads, including non-worker threads.<br>
>> > the problem here is that localStorage is a pile of global variables. we<br>
>> > are<br>
>> > trying to give people global variables without giving them tools to<br>
>> > synchronize<br>
>> > access to them. the claim i've heard is that developers are not savy<br>
>> > enough<br>
>> > to use those tools properly. i agree that developers tend to use tools<br>
>> > without<br>
>> > fully understanding them. ok, but then why are we giving them global<br>
>> > variables?<br>
>> > there has to be a better answer.<br>
>><br>
>> I actually described an potential solution in the thread on worker<br>
>> storage.<br>
>><br>
>> The problem you describe is a worker holding on the the storage for an<br>
>> very long (indefinite) time, thereby locking out other threads/windows<br>
>> from accessing the same storage area. This seems inevitable if we want<br>
>> to prevent race conditions while at the same time not forcing the<br>
>> complexities of locks onto web developers. The WebDatabase API suffers<br>
>> from exactly the same problem.<br>
><br>
> Hmm... are you saying that from the SQLStatementCallback used to read<br>
> some data out of the database, you might compute on that data, and then<br>
> issue an executeSql call to write a computed result, and that in this<br>
> scenario,<br>
> the fact that it is the same transaction means that other threads are locked<br>
> out of accessing the same database? I hadn't considered chaining executeSql<br>
> calls like this to keep the transaction alive. Hmm...<br>
<br>
</div></div>Indeed.<br>
<div><div></div><div class="h5"><br>
>> However, we can lessen the problem. By adding multiple storage areas,<br>
>> we can allow a worker to use one storage area, while allowing other<br>
>> parties to simultaneously use other storage areas. This way, if a<br>
>> worker and a window aren't sharing data at all, they never get in the<br>
>> way of each other.<br>
>><br>
>> So a very simplistic design would be something like the following:<br>
>><br>
>> getStorageArea(name, callback)<br>
>><br>
>> when called will asynchronously call the callback parameter once the<br>
>> storage area named by the first parameter becomes available. The<br>
>> callback receives the storage area as an argument. We would also have<br>
>> the function<br>
>><br>
>> getMultipleStorageAreas(names, callback)<br>
>><br>
>> Same as above, but names is an array of strings indicating multiple<br>
>> storage areas that need to be acquired before the callback is called.<br>
>> The callback receives all the areas in an array as an argument. This<br>
>> function allows transferring data between multiple storage areas<br>
>> without risking racing.<br>
>><br>
>> There's several problems with this, such as the names are sort of<br>
>> crappy, and that getting storage areas an array isn't very friendly.<br>
>> However you get the basic idea.<br>
>><br>
>> We don't even need to use Storage objects for this. In fact, I hope<br>
>> mozilla will in a not too distant future come up with an alternative<br>
>> proposal to the WebDatabase SQL API. Something like this might fit<br>
>> into such a proposal as I think that'll have multiple separate storage<br>
>> areas anyway.<br>
>><br>
>> / Jonas<br>
><br>
><br>
> Maybe we should just invent a similar transaction method for name/value<br>
> storage? Wouldn't that be better than inventing a new idiom? Ideally,<br>
> we'd also make reads and writes on storage be asynchronous. The<br>
> transaction would then be usable to hold the lock across multiple<br>
> asynchronous reads and writes. Since local storage is backed by disk,<br>
> it seems like a more ideal local storage API would not require synchronous<br>
> filesystem access.<br>
<br>
</div></div>Not quite following what you're suggesting, but there's lots of ways<br>
to design this. The critical part is to allow grabbing (with<br>
associated locking) of just a subset of the available storage space.<br>
<font color="#888888"><br>
/ Jonas<br>
</font></blockquote></div><br><div><br></div><div>I was suggesting that we only provide asynchronous getItem / setItem calls, where</div><div>each call is parameterized by a transaction. This is how database works.</div>
<div><br></div><div>-Darin</div>