[whatwg] register*Handler and Web Intents

[Ian Hickson]

Having carefully studied the Mozilla Web Activities proposal, the Web 
Intents draft, the register*Handler APIs, and to a lesser extent the 
dispatch mechanisms in existing operating systems (desktop and mobile) and 
the piles of advocacy on teh subject on the Web, I've tried to come up 
with a concrete proposal for merging all of them into a coherent whole 
that addresses the limited use cases that are of most interest.

Overall, this proposal mainly draws from the following sources:

 - For the handling mechanimsm, it's basically directly based on the 
   Mozilla "System Message Handler" mechanism.

 - For the dispatch mechanism, it is based mainly on the Web Intents 
   draft.

 - For the registration mechanism, it's based on extending the 
   register*Handler APIs consistently.

However, I have attempted to simplify the proposal from its sources of 
inspiration, where that seemed possibly without sacrificing key use cases.

The overall design is that there are URLs that can handle particular 
messages, herein called intents (mostly because that's what Web developers 
seem to have adopted as the term already).

These "intents" are indications that the user, or the system on behalf of 
the user, wants the application to do something or other.

   For example: "please open this PDF", "please share these four images",
   "please be aware that it is now the time you asked to be awoken at", 
   "please allow the user to compose an e-mail to this address".

We want pages to be able to handle intents promptly upon being opened. 
This puts certain constraints on the design that are explained in detail 
in the Mozilla "System Intents" thread:

   https://groups.google.com/forum/?fromgroups#!topic/mozilla.dev.webapi/o8bkwx0EtmM

This leads therefore to the first part of the concrete proposal:

  partial interface Window {
    void setIntentHandler(DOMString action, IntentCallback callback);
    boolean hasPendingIntent(DOMString action);
  }

  callback IntentCallback = void (Intent intent);

When a page is loaded, it registers its intent handler callback using 
setIntentHandler(), and as soon as that happens, the UA asynchronously 
calls the handler.

The Intent object itself "just" needs to describe what the page is being 
asked to do. For this, I borrowed freely from all the proposals mentioned 
above, and ended up with:

  interface Intent {
    readonly attribute DOMString action,
    readonly attribute DOMString type,
    readonly attribute any data,
    void success(any data, optional sequence<Transferable> transfer);
    void failure(DOMString message);
  }

This is basically the data that needs to be sent from the application that 
wants to initiate an intent, too:

  dictionary IntentSettings {
    DOMString action,
    DOMString type,
    any data, // structured-cloned
    sequence<Transferable> transfer,
  };

So how do we start an intent? Well, for intent handlers that are just 
registered to handle URLs being loaded using certain schemes or that have 
certain MIME types, you just use a link:

  <a href="web+customscheme:foo">...</a>
  <a href="file.customtype">...</a>

But more interestingly, to start one programmatically you would use a 
startIntent() method:

  partial interface Window {
    void startIntent(IntentSettings intent,
                     optional AnyCallback success,
                     optional DOMStringCallback failure);
  }

  callback AnyCallback = void (any data);
  callback DOMStringCallback = void (DOMString message);

The callbacks given in the method, if provided, are invoked asynchronously 
in reaction to the handler calling success() or failure() on the Intent 
object. We would just allow one success or failure message per Intent, 
for sanity's sake.

There is another way of starting an intent that I alluded to earlier: the 
system can decide to dispatch one to a specific application. For example, 
say an application has told the system (using some other API) that it 
wants to be woken up at 07:00 to start playing a song (the user's alarm). 
The system could dispatch an intent to the application using the mechanism 
described above -- a system intent. To distinguish these from intents sent 
by applications like "airplay media", "share picture", or "open mailto:", 
one would use intent names prefixed by the string "system-", for example, 
"system-alarm" or "system-nfc". The startIntent() method described above 
therefore would, in this proposal, fail if the intent name started with 
the substring "system-".

All of this leads to the final piece of the puzzle, namely how to offer to 
the user the ability to use a page as an intent handler. This is where the 
existing register*Handler() methods come in. Taking a cue from Mozilla's 
Web Activities proposal, I propose that we just use introduce methods for 
the more generic "intents" system:

  partial interface Navigator {
    void registerProtocolHandler(DOMString scheme,
      DOMString url, DOMString title, optional HandlerDisposition disposition);
    void registerContentHandler(DOMString mimeType,
      DOMString url, DOMString title, optional HandlerDisposition disposition);
    void registerIntentHandler(DOMString action, DOMString? mimeType,
      DOMString url, DOMString title, optional HandlerDisposition disposition);
    DOMString isProtocolHandlerRegistered(DOMString scheme, DOMString url);
    DOMString isContentHandlerRegistered(DOMString mimeType, DOMString url);
    DOMString isIntentHandlerRegistered(DOMString action, DOMString? mimeType, DOMString url);
    void unregisterProtocolHandler(DOMString scheme, DOMString url);
    void unregisterContentHandler(DOMString mimeType, DOMString url);
    void unregisterIntentHandler(DOMString action, DOMString? mimeType, DOMString url);
  }

  enum HandlerDisposition {
     "replace", // means that you use the browsing context of the page 
     // that invoked the intent, just like clicking a link normally
     "new", // means you must open a new top-level browsing context 
     // (window, tab) to handle the intent
     "overlay", // means you must open an auxiliary browsing context 
     // (popup dialog, sidebar) to handle the intent;
     "reuse", // means the same as "new" unless the page is already open, 
     // in which case it just switches focus to that one and sends a new message
  };

A strong argument has been made, however, that we should offer a 
declarative way of registering these: it would dramatically help with 
discovery (e.g. by allowing Web applications to be indexed by search 
engines). Thus, I propose a parallel mechanism in the form of an empty 
element that goes in the <head>:

  <intent
    action="edit"     intent action, e.g. open or edit, default "share"
    type="image/png"  MIME type filter, default omitted, required if scheme omitted
    scheme="mailto"   Scheme filter, default omitted, required if type omitted
    href=""           Handler URL, default "" (current page)
    title="Foo"       Handler user-visible name, required attribute
    disposition=""    HandlerDisposition values, default "overlay"
  >

I welcome feedback on this concrete, though only briefly described, 
proposal. 


On Tue, 17 Apr 2012, Mounir Lamouri wrote:
> > 
> > Trying to fit the registration components listed above into <meta> 
> > really doesn't work all that well, IMHO.
> 
> <meta name="viewport"> does that and is widely used AFAICT.

Oh I agree that it's widely used, but I don't think <meta name=viewport> 
is counter-evidence to the claim that it "really doesn't work all that 
well". :-)


> > I don't think wildcard matching really makes sense. In particular, I'm 
> > not aware of any service that can honestly say it supports image/*, or 
> > indeed any other topleveltype/*.
> 
> Doesn't @accept on <input> allows "image/*", "video/*", "audio/*"? We 
> should at least accept those values. But maybe it would be safe to allow 
> foo/* if a service and client would be connected only if they both 
> marked they "foo/*" as a type?

See discussion below.


> Given that Web Intents would deprecate/supersede RPH and RCH, I wonder 
> why we should bother incorporating them.

I don't think this mechanism supersedes either. We still need a way, for 
example, to implement Web-based mailto: handlers.


On Fri, 20 Apr 2012, Greg Billock wrote:
> 
> For RPH, agreed that passing the URL is pretty much the only possible 
> approach. For RCH, web intents allows us to do better than this: we can 
> pass the content directly, in a Blob, rather than passing the URL, thus 
> decoupling the service from the (possibly sensitive) URL from which the 
> intent was triggered. That isn't always the right plan -- for feed URLs, 
> passing the URL is an important feature enabler for a feed reader to 
> deal with the content. Anyway, mostly pointing out that considering 
> these together is a vehicle for more fine-grained control of the 
> coupling. If anything, this intertwines them more closely.

Providing the data does seem to make sense, yes.


> Our remaining discomfort here is with isIntentHandlerRegistered(), and 
> for similar reasons to the fingerprinting qualities of 
> isProtocolHandlerRegistered(). That is, we'd prefer that web content 
> simply call registerProtocolHandler blindly, similar to what a 
> declarative registration would do, and let the UA sort out whether the 
> user ought to be shown any kind of registration UI.

These is*Registered() methods don't expose anything that isn't already 
possible with cookies, so I propose that we just disable them if cookies 
are disabled, and say that clearing cookies should recommend clearing the 
registrations too.


> This does, however, impose some burden on clients to these systems. 
> They'd prefer to know that at least something is on the other side of a 
> content load or protocol link or web intents invocation. For web 
> intents, we're proposing that a set of suggested services could be 
> attached to the intent invocation, so if developers are worried about 
> this, they can attach suggestions which the UA may then regard as hints 
> if no qualifying filter matches the intent invocation.

I haven't added this to the proposal, but it does seem like a good idea. 
What do people think? Is it worth it? Traditionally, browsers and 
operating systems have handled the lack of a handler by offering known 
handlers, rather than relying on the caller to provide some. Given the 
ideal of declarative <intent> registration, one can imagine the browser 
redirecting to a search engine's list of handlers, for example.


> Putting this in the HTML spec sounds like a great plan to us.

Roger.


> [wildcard handlers]
> One exception would be for "save" type intents, where pretty much any 
> type can be dealt with. Another is where the handler is using say <img> 
> or <canvas>, and would like to specify accepted types in an open-ended 
> way.

Interesting.

For a "save"-type intent, which is really oblivious to the type, I guess 
we'd just allow the type to be null, meaning "any type".

For features that are entirely dependent on client-side technologies, we 
could by convention say that a registration for image/* means "all the 
image types supported in <img>, drawImage(), background-image, etc", and 
that video/* means "all the types supported in <video>" and audio/* means 
"all the types supported in audio/*". (We should probably say that that's 
what the keywords mean for type=file, too.)


> At the DAP meeting, we agreed to extend this system to include 
> string-literal types, which provides a way to do good integration with 
> microdata types. There we expect to do exact string matching, and it is 
> true the eliminating wildcard types would bring these two type 
> namespaces a bit closer. MIME is complex enough that it would still have 
> to be treated separately, however. (Parameters and all that.)

I'm not really sure I follow here. Could you elaborate?


On Thu, 24 May 2012, Greg Billock wrote:
> 
> Some more concordance moves to make them more like different aspects of 
> the same feature:
> 
> * In registerContentHandler, "t" can be a space-separated list of MIME 
> types for registerContentHandler.

Do we need to support a space-separated list at all? When will the list be 
so long that it really makes sense to add that feature rather than just 
having three registrations back-to-back?


(I have omittd a number of e-mails from this reply that gave suggestions 
that seemed reasonable and that have therefore been incorporated into the 
proposal above.)


If nobody finds any egregious problems with the proposal above, I'll start 
speccing it as part of a rewrite of the register*Handler() section. (That 
section needs a rewrite into more imperative language anyway.)

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'