[whatwg] Cross-domain databases; was: "File package protocol and manifest support?"

[Brett Zamir]

With IndexedDB apparently gaining traction, I'd like to reiterate my 
proposal for cross-domain shared databases. Though I'll admit I'm not 
sure I understand all of the concerns raised earlier, I do want to offer 
my own rebuttals to these counter-points as I understand them (and 
subsequently highlight some additional advantages):

*Compared to using cross-domain shared workers*

While cross-domain shared workers might be more ideal for sites which 
wanted more granular control of how their shared data was accessed, and 
no doubt a positive idea for sites wishing to go to the trouble to set 
it up, my strong feeling is that making one's data shared should also be 
possible in an easy a manner as setting up the database itself. 
Requiring the developer to create their own API for others to access 
their database would no doubt A) prevent some sites which might be open 
to sharing their data from doing so, and B) Add barriers for those 
wishing to mash-up different sites' data, as it would require study of 
not only the site's unique data store, but also its means of allowing 
access to it.

If not from the start, ideally the API could eventually offer more 
fine-grained access, such as to request read-only privileges or the like 
from the user, but these features would ideally work for any site 
offering a shared database, with no need to learn a site-specific API.

*Compared to being able to copy a remote database*

Besides avoiding wasted space in duplicating another database, the 
alternative of being allowed to make a simple copy of a remote database, 
is inferior in that one would not be able to operate off of the same 
data, especially if changes continue to be made by the user within the 
application and for which they do not wish independent copies of data. 
If one site offers a note-taking application, there are strong benefits 
for portability as well as concurrent use of different interfaces if the 
same data can be manipulated by multiple applications (as in different 
word processors being able to access the same file). Moreover, there is 
also the assurance for an application that it is allowing its users to 
work with the latest data from their targeted site if the targeted site 
has its own means of ensuring automatic updates.


*Transactional concerns*

I would think transactional concerns could be addressed by building in 
transactions and/or good judgment by the sharing party about whether to 
share a database in the first place (if a third-party application could 
cause corruption). Surely there are no inherently insurmountable 
barriers here.
*

Higher Privileges, Iterating Databases, and Search*

I would even hope, especially given the apparently similarly 
privilege-seeking FileSystem API,  for the ability to request permission 
from the user to access any local database, whether shared by the site 
author or not, if the idea is not shot down by those not willing to 
accept that good features may still come at the cost of security risk to 
some users who do not pay attention to dialogues (these users can also 
be duped to download unsafe executables, opt in for geolocation, etc., 
so I don't think the web should be stunted in its growth by 
over-coddling all of us just because some may abuse it). It would also 
be nice to be able to iterate through the available databases in order 
to allow generic database viewing programs to be built as web apps, as 
well as perhaps eventually see a Full Text Search API for searching 
through all of one's data (and files).

The WHATWG cannot anticipate and make its own API for every possible 
shared data use case (even if some general uses like calendar access 
indeed call for their own API) nor expect that everyone willing to share 
data will wish to go through numerous hoops to do so. I think the full 
power of Shared Data and Open Data remains to be seen if the user were 
able to download data once, while having the ability to query that data 
as they wish from any number of their own chosen applications, and if 
shared content providers could be free to lighten their server loads and 
alleviate security concerns for enabling unrestricted searches.

Thanks,
Brett

(The email below is repeated for convenience, but due to my loss of the 
original email, obtained from 
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020299.html or 
as a better organized thread: 
http://www.mail-archive.com/whatwg@lists.whatwg.org/msg15393.html )
-----------------------------------------------
On Wed, 20 May 2009, Brett Zamir wrote:
 >
 > I would like to suggest an incremental though I believe significant
 > enhancement to Offline applications/SQLite.
 >
 > That is, the ability to share a complete database among offline
 > applications according to the URL from which it was made available.
 > [...]

On Tue, 19 May 2009, Drew Wilson wrote:
 >
 > One alternate approach to providing this support would be via shared,
 > cross-domain workers (yes, workers are my hammer and now everything
 > looks like a nail :) - this seems like one of the canonical uses of
 > cross-domain workers, in fact.
 >
 > This would be potentially even more secure than a simple shared
 > database, as it would allow the application to programmatically control
 > access from other domains, synchronize updates, etc while allowing
 > better controls over access (read-only, write via specific exposed write

On Wed, 20 May 2009, Rob Kroeger wrote:
 >
 > For what it's worth, this was my immediate thought as well upon reading
 > the idea. The database is insufficiently fast on some platforms to
 > server as an IPC mechanism and there are practical limitations with
 > having too many contending transactions so my instinct would be to build
 > large integrated web apps with a shared worker routing data between
 > components.

On Thu, 28 May 2009, Michael Nordman wrote:
 >
 > I buy this thinking too as a better strategy for integrating web apps.

Based on the above comments, I haven't added the requested feature at this
time -- let's see if the existing features can be used to do it first.


On Thu, 28 May 2009, Michael Nordman wrote:
 >
 > But still, the ability to download a fully formed SQL database, and then
 > run SQL against it would be nice.
 >
 > openDatabaseFromURL(urlToDatabaseFile);
 >
 > * downloads the database file if needed (per http cache control headers)
 > * the database can reside in an appcache (in which case it would be
 > subject to appcache'ing rules instead)
 > * returns a read-only database object
 >
 > Of course, there is the issue of the SQL database format.

On Thu, 28 May 2009, Anne van Kesteren wrote:
 >
 > Would there be a lot of overhead in just doing this through 
XMLHttpRequest,
 > some processing, and the database API?

On Thu, 28 May 2009, Michael Nordman wrote:

 > Good question. I think you're suggesting...
 > * statementsToCreateAndPopulateSQLDatabase  = httpGet();
 > * foreach(statement in above) { execute(statement); }
 > * now you get to run queries of interest
 >
 > Certainly going to use more client-side CPU than downloading a fully
 > formed db file. I think the download size would greater (all of the
 > 'INSERT into' text overhead), but thats just a guess. A database
 > containing FTS tables would change things a bit too (even less
 > client-side cpu, but more download size).

On Fri, 29 May 2009, Anne van Kesteren wrote:
 >
 > There are certainly drawbacks, but given that we still haven't nailed
 > all the details of the database API proposal discussed by the WebApps WG
 > (e.g. the SQL syntax) and given that it has not been deployed widely, it
 > seems somewhat premature to start introducing convenient APIs around it
 > that introduce a significant amount of complexity themselves. Defining
 > the rules for parsing and creating a raw database file in a secure way
 > is a whole new layer of issues and the gain seems small.

On Fri, 29 May 2009, Michael Nordman wrote:
 >
 > I don't think this feature's time has come yet either. Just food for
 > thought.

I guess we'll wait on this for now.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'