[whatwg] Installed Apps

[Ian Hickson]

On Mon, 27 Jul 2009, Michael Davidson wrote:
> 
> -- Slow startup: When a user navigates to mail.google.com, multiple 
> server requests are required to render the page. The Javascript is 
> cacheable, but personal data (e.g. the list of emails to show) is not. 
> New releases of Gmail that require JS downloads are even slower to load.

The offline app cache allows you to cache the JS and have new releases be 
downloaded in the background while the user uses the old version.

Client-side databases allow data to be cached client-side.

This leaves how to get new data to the client before the client has 
loaded.


> -- Native apps like Outlook can (and do) run background processes on the 
> user's machine to make sure that data is always up-to-date.

That would be one way to update the cached data before the client runs.


> -- Notifications: Likewise, Outlook can notify users (via a background 
> process) when new mail comes in even if it's not running.

HTML5 used to have a notifications API, but it was removed due to lack of 
implementor interest.


So there seem to be two problems here:

 - how to update a client-side database in the background when the app 
   isn't loaded, such that the data is up to date when the page is opened,

 - how to pop up notifications while the page is not loaded.

It seems like a direct solution to these would be to have a way for Web 
pages to expose a service that clients could subscribe to, which could 
send up database updates and notifications. That way, no script runs on 
the client, but the server can still update the client whenever needed.

For example, we could base this on the EventSource mechanism or the 
WebSocket mechanism, and define a protocol by which the server can at any 
time send up either:

   An INSERT, UPDATE, or DELETE SQL statement
   A string and a URL to show in a notification

The client would then open a connection to this service whenever it was 
open, and the server would update the database and send notifications as 
desired. The UA could then easily show in its UI that a remote server is 
updating data or showing a notification, and make it trivial for the user 
to unsubscribe from the relevant feaure.

This seems far less problematic than background processes, which have a 
host of problems:

 - they have unfortunate implications in multiprocess UAs (see below)

 - they are extremely susceptible to being co-opted for malware, e.g. 
   using CPU resources for computation without the user's knowledge

 - the permission UI is unproven

 - it doesn't really scale when hundreds of sites want to use the feature

 - it's not a good solution for performance problems (see anecdote about 
   Netscape 6 below)


On Mon, 27 Jul 2009, David Levin wrote:
>
> It sounds like most of the concerns are about the 2nd part of this 
> proposal: allowing a background page to continue running after the 
> visible page has been closed.
> 
> However, the first part sounds like it alone would be useful to web 
> applications like GMail:
> 
> The first, which should begenerally useful, is the ability to have a 
> hidden HTML/JS page running in the background that can access the DOM of 
> visible windows. This page should be accessible from windows that the 
> user navigates to. We call this background Javascript window a "shared 
> context" or a "background page". This will enable multiple instances of 
> a web app (e.g. tearoff windows in Gmail) to cleanly access the same 
> user state no matter which windows are open.

As far as I can tell, we can't ever have a scripting context that can 
access the DOMs of all pages in the same domain, since that would force 
those pages into the same process, which is an assumption that is already 
violated by IE (and, I believe, Chrome).

Also, since anything in the same browsing context typically has to be in 
the same process for sanity (accessing the Window object hierarchy is 
synchronous), and since many sites have <iframe>s to common sites (e.g. ad 
networks) you very quickly end up dragging many sites into the same 
process if you go down this road. So I don't think that's really workable.


> + restrict things to the same security origin.
> 
> It sounds similar in concept to a share worker except that it runs in 
> the main thread and is more concerned with dom manipulation/state while 
> workers have typically been thought of as allowing background 
> processing.
> 
> It seems that the lifetime of this could be scoped, so that it dies when 
> it isn't referenced (in a similar way to how shared worker lifetime is 
> scoped).

It doesn't seem that having a lifetime-limited hidden window would address 
any of the problems listed at the top of this thread:

 - how to update a client-side database in the background when the app 
   isn't loaded, such that the data is up to date when the page is opened,

 - how to pop up notifications while the page is not loaded.


On Tue, 28 Jul 2009, Michael Nordman wrote:
> 
> What if a sharedContext isn't gauranteed to be a singleton in the 
> browser. A browser can provide best effort at co-locating pages and 
> sharedContexts, but it can't gaurantee that, and the spec respects that.
> 
> The lesser gaurantee is that all directly scriptable pages (those from a 
> given set of related browsing contexts) WILL share the same 
> sharedContext (should the refer to it).

That sounds like shared workers, at that point. Why does this need to be a 
browsing context? If you can't necessarily get to the DOMs of all the 
actual windows, then it doesn't seem like it's worth the pain to make it 
possible to get to the DOMs of any of the windows.


On Tue, 28 Jul 2009, Drew Wilson wrote:
>
> Anyhow, addressing some of the other comments - I don't think that 
> there's necessarily a problem with the async worker APIs as they stand, 
> and I don't think we can easily retrofit synchronous APIs on top of 
> their current execution model. The issue is that the core problem that 
> Workers solve (parallel execution in a separate context from the page) 
> is different than the problem that large web apps are trying to address 
> (reduced latency).

I don't understand why workers can't reduce latency. What is the latency 
that must be reduced which workers can't help with?


> SharedWorkers are overloaded to provide a way for pages under the same 
> domain to share state, but this seems like an orthogonal goal to 
> "parallel execution" and I suspect that we may have ended up with a 
> cleaner solution had we decided to address the "shared state" issue via 
> a separate mechanism.

Shared state is addressed by the Web Storage and Web Database features; 
the shared workers are intended to provide shared computation.


> So (and forgive me for restating), it seems like hidden page addresses the
> following problems that gmail and other large web apps are having:
> 
> 1) Loading large amounts of Javascript is slow, even from cache.

The solution is to make loading the JS faster, not to load it and use the 
user's resources regardless.

Consider Netscape 6. It briefly had a "solution" to its slow load times: 
keep the browser always loaded in the background. The users *hated* this 
solution, because it's no solution at all: it just wastes the user's 
resources when he doesn't need them. What's more, the preloaded code ends 
up paging to disk anyway, and then when you need to load the app, it takes 
just as long as it would have to cold-start anyway.

Users don't appreciate software that tries to improve its performance by 
using the user's resources when it's not active.


> 2) Loading application state from the database is slow.

I don't see how a hidden window can solve this. Could you elaborate?


> 3) Sharing between pages requires going through the database or shared
> worker - you can't just party on a big shared datastructure.

This would only be solved by a hidden window if the window was accessible 
from all processes, which won't work.


> 4) There's no way to do things like new mail notifications, calendar 
> notifications, local updates of your email inbox, etc when the browser 
> is not open.

It seems a system dedicated to this could solve this problem in a much 
simpler and more efficient way than running background windows.


On Tue, 28 Jul 2009, Michael Davidson wrote:
> 
> These are true, but leave out the part that rewriting large apps to the 
> worker API is nontrivial. A major advantage of a hidden page (as you 
> mention below) is that the programming model is well known, and easy for 
> web developers to adapt to.

I'm not going to design HTML5 around what is easier for GMail to do. If it 
costs Google more to update our apps to use the new features, then so be 
it -- we can afford it. It's more important that the technologies we 
develop here be efficient for new sites going forward than for existing 
sites today, since there will always be more new sites than there have 
been old sites that need updating (unless the Web has peaked!).


> - As for persistence beyond browser lifetime, I understand the 
> reticence. However, similar problems have been solved in the past. Flash 
> asks the user for access to hardware like cameras. Surely being able to 
> take pictures of users is as scary as running code after the browser has 
> closed.

It's a lot easier to explain to the user "do you want this Web site to see 
that you are naked" than it is to explain to the user "do you want this 
Web site to be able to do background computation that might be used by 
hostile sites to perform nefarious deeds".


On Wed, 29 Jul 2009, Michael Kozakewich wrote:
> 
> Pop-up notifications would be a great thing to do, where someone can use 
> JavaScript similar to an alert() box that pops up in a corner for a few 
> seconds and displays a message. I think I'll call this one out a few 
> times, so someone sees it, because it really would be a worthwhile thing 
> to add to a spec.

It was in the spec for a while, but I couldn't get traction on getting 
browsers to implement it. Since WebKit might be adding such a feature 
after all, I may have to bring the feature back.


On Wed, 29 Jul 2009, Michael Davidson wrote:
>
> Here's something that hidden pages can help with that this solution 
> can't:
> 
> Let's say you're watching ten stocks on Google Finance, each in their 
> own window.
> 
> Right now, each page has to have its own connection to the server. Since 
> these are polling connections, the experience is going to be very bad.
> 
> With a hidden page that's accessible to all Google Finance visible 
> pages, they could share a connection to the server. Even if the hidden 
> page is closed when the last Google Finance page is closed, this is a 
> better situation than we currently have.

Shared workers are able to solve the same problem.


On Wed, 29 Jul 2009, Michael Davidson wrote:
> 
> It's my contention that frequently large web apps are doing more than 
> just polling for data. They're trying to maintain complex data 
> structures that they pass up to the UI. The programming model of 
> SharedWorkers makes this difficult. Take the chat client in Gmail, for 
> example. It's much more complex than passing stock quotes from a worker 
> to the UI.

Given that everything that goes over the wire is serialised, I don't see 
why everything that goes from the worker to the UI layer can't be 
serialised also.


On Wed, 29 Jul 2009, Jeremy Orlow wrote:
> 
> This idea of a hidden page just seems like a big hack to support today's 
> applications.  If it were adapted into the spec, I think 5 years from 
> now we'd be very sorry that it was.

Indeed.


On Wed, 29 Jul 2009, Michael Nordman wrote:
> 
> HTML parsing, layout, and rendering is very optimized, much more so than 
> programmatic HTML DOM manipulation. The incorporation of stateful script 
> contexts within each page considerably slows page loading. As 
> navigations occur, that statefullness has to be reconstructed from 
> scratch. In addition to creating and populating a new script context 
> with the <script> itself, this often involves reconstructing data the 
> script operates on. So additional server roundtrips and desrializing 
> into js objects. With the introduction of local storeage and databases, 
> augment the server roundtrips with the reading of those local 
> repositories and deserializing into js structures.
> 
> The sharedContext / backgroundPage provides a means to cut out 
> completely the need to reconstruct that JS world on each page 
> navigation.

Except it doesn't, because we can't guarantee a shared context between all 
relevant pages, since they might be in other processes.


On Thu, 30 Jul 2009, Robert O'Callahan wrote:
>
> What happened to my idea for browsers to have a special window 
> containing tabs for "background apps", which save screen real estate by 
> just showing an icon and title (and a URL or domain?) and no actual tab 
> content? You might modify the UI so that quitting the normal browser 
> leaves this window open, possibly as a separate OS app. Seems to me that 
> this would provide almost exactly the desired functionality but without 
> introducing new security concerns and without requiring a trust 
> decision.

This seems like a reasonable idea (it's very similar to ideas that have 
been floated around for persisted workers, too). It doesn't seem to 
require any help from the specs; the user can just drag a tab to such a 
"background tab tray".


On Wed, 29 Jul 2009, Michael Kozakewich wrote:
> 
> How many applications do we expect any one user to have open? I would 
> imagine one would do fine on the Taskbar or in the Notification Area, 
> like other programs, but a manager would be good if a user had a great 
> deal of applications running at once.

Realistically speaking, users will soon have many applications open if we 
make this available: Flickr, Mail, Calendar, Facebook, Reddit, Twitter, 
BBC News, their Bank, Google Voice, ... I would expect hundreds of sites 
to make use of such a feature.


On Wed, 29 Jul 2009, Maciej Stachowiak wrote:
> 
> Often, web applications would like to give users the option to subscribe 
> to notifications that occur at specific times or in response to 
> server-side events, and for the user to get these UI notifications 
> without a prerequisite that the web app is open or that the browser is 
> running. There may be a desire to do client-side computation as well, 
> but often just the ability to give the user a notification solves the 
> basic user interaction problem.
> 
> One possible way to address this kind of use case is to let users 
> subscribe to a "feed" of notifications. This feed could use standard 
> syndication formats, such as RSS or Atom. But instead of being displayed 
> in a traditional feed reader, it's displayed in the form of transient 
> notifications (along the lines of Growl on Mac OS X) which are posted 
> for each new event. To allow some pre-scheduling of events, each item 
> can have a date and won't be displayed until that date - this way a 
> calendar can give you your feed of upcoming events and you can still get 
> notifications when offline. In the case of something like email or 
> Twitter, obviously there's no sensible way to get notifications when 
> offline since they depend on unpredeictable server-side activity. There 
> could even be a client-side API that lets a Web app schedule items on a 
> subscribed notification feed from script, to enable scheduling calendar 
> events offline. Each notification would have the option to unsubscribe 
> from the notification feed, to reduce spam potential.
> 
> Notice that this opens up a lot less attack surface. The user has to 
> actively opt in to subscribing to the notification feed, just as for an 
> RSS feed. This makes it much less likely they end up with a subscription 
> to a shady site. And the notifications are passive data items (probably 
> no script should be allowed in a notification, if the format is HTML and 
> not just plain text), so they open up a lot less security risk. 
> Obviously this is less powerful than the ability to run arbitrary code 
> in the background. But it could address a large chunk of the use cases 
> with much less security risk.

This seems like a good alternative for the notifications feature, at least 
for scheduled notifications (it wouldn't work so well for new mail 
notifications).


On Thu, 30 Jul 2009, Michael Davidson wrote:
>
> Here's a rough sketch of a more far-out idea: What if all browsers were 
> XMPP clients and stanzas could be sent to display notifications? The 
> attack surface would still be low, but you'd get realtime updates. 
> Instead of subscribing to a feed of notifications, the user accepts what 
> is essentially a chat invitation from the site. Like normal XMPP 
> invitations, this would be revocable at any time.

I'm not sure XMPP is the protocol I would jump to, but it's certainly an 
option, yes.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'