[whatwg] several messages about <event-source> and related subjects

[Ian Hickson]

Sorry about the three-way cross-post, but this message contains replies to 
e-mails from those three mailing lists. Please select just the one mailing 
list when replying, and trim the quotes below to include only what you are 
discussing. Thanks.

Executive summary:

 * Made it possible to adjust the reconnection timeout.

 * Made it so that the client tells the server what the last message it 
   received was when it reconnects, so as to allow seamless continuation. 

 * Rewrote the implementation processing requirements.

 * Simplified the event stream format:
    - removed the concept of 'commands'
    - changed the comment syntax from starting with a ';' to
      starting with a ':'

 * Removed the possibility to control anything of the event other than the 
   'data' and 'type' of the event.

 * Restricted the events that can be fired to all use MessageEvent.

 * Made 'message' events not bubble.

 * Made the MIME type of the new event stream format be
   text/event-stream; removed mention of the old format's type 
   (application/x-dom-event-stream).


In 2006, Anne wrote, in an e-mail I apparently lost, but which he
later reminded me of:
> 
> * The event-source element has no DOM interface while it has some 
> attributes that probably warrant one.

This got fixed at some point.


> * The specification doesn't define when the onevent event handler is 
> invoked nor when the event event is dispatched. They are only defined. 
> It's also unclear which interface they implement, et cetera.

This should be fixed now.


> * A problem with the event-source element is that the resource is loaded 
> before you can attach event listeners to the document. Perhaps the 
> loading should start after the load event is dispatched? Unless the 
> element was inserted into the document of course (that's actually also a 
> bit unclear).

If you use src="", just use onmessage="" to receive the messages (and 
only use 'message' events). Alternatively, do it dynamically.


> * Since event sources can be attached using other ways than using the 
> event-source element the Target field should be amended to take that 
> into account. (Some sentences there don't make sense for an event source 
> attached to an object that is not an event-source element.)

Target is gone.


> * Regarding that, I'd be interested in hearing the use case for allowing 
> any EventTarget to be a source for server-sent events.

This was requested by Opera... I'll happily drop it if it's no longer 
required. I'd like to only make it <event-source> elements.


> * Event namespaces throughout should be changed to match DOM Level 3 
> Events. That basically means that http://www.w3.org/2001/xml-events is 
> gone.

No mention of namespaces is left.


> * Perhaps RemoteEvent should be replaced with a reference to CustomEvent
>   from DOM Level 3 Events which offers the same type of functionality?

RemoteEvent is now MessageEvent and this no longer applies.


> * What happens when the event given in the Event field doesn't match the 
> NCName production as required by DOM Level 3 Events such as in the 
> example in section 7.1.7? (It uses the event "stock change".)

Nothing.


> * At the moment the BNF does need error handling because you could have 
> a ";" at the start of a line without any data following (or a new line 
> for that matter).

That would have been a comment. Now it's an ignored fied.


> * It might be better to replace the BNF with something similar as the 
> HTML parsing specification currently has. That provides a much more 
> clear processing model.

Both are present.


> * What happens for other line feed characters? Are they treated as 
> fields? Won't that give lots of problems for authors coding in non-Unix 
> formats? HTTP for example allows both.

CRLF, CR, and LF, are all allowed now.


> * "For each non-blank, non-comment line, the field name is first 
> taken[...]" doesn't cover what happens to command lines.

Removed command lines.


> * "The ctrlKey field would be ignored[...]" should probably say
> "keyIdentifier" as that's what's used in the example.

That section is gone now.


On Thu, 14 Jun 2007, reed wrote:
> 
> I was wondering how you mitigate the persistent connection limitations 
> described in RFCs 2068 and 2616 vs server-sent events. It seems the 
> former limits the laters usability.

This seems like an HTTP issue. I'm not sure what we can do about it.


On Thu, 21 Jun 2007, Anne van Kesteren wrote:
>
> If want to send XML or HTML documents from the server using server-sent 
> events you have to escape them in some way and then unescape them again 
> on the client side. For sending documents it would be more useful if you 
> could specify something like the length of the chunk of data and a MIME 
> type followed by the actual data and that on the client side you just 
> have an attribute on the event that gives you the DOM representation of 
> the document similar to what XMLHttpRequest already has. Specifying a 
> new format besides application/x-dom-event-stream is probably 
> sufficient. Something like application/x-dom-document-event-stream...

"Escaping" just consists of splitting on newlines and prefixing each 
line with 'data: ', which is trivial. You get back a string which you can 
process any way you like (document.innerHTML = s, parseJSON(s), whatever).


On Fri, 3 Aug 2007, Henry Mason wrote:
> 
> I've been working on an implementation of Server-Sent DOM Events. I ran 
> into a small inconsistency in the specifications as they stand. Under 
> section 6.2.4's "Class field" part:
> 
>     If the Namespace is null and the Event field is message (including
>     if it was not specified explicitly), then the MessageEvent interface
>     must be used.
>     Otherwise, the Event interface must be used.
> 
> I feel this should be changed and simplified to to:
> 
>     Otherwise, the MessageEvent interface must be used.

This is now moot as only MessageEvent can be used.


On Thu, 16 Aug 2007, Henry Mason wrote (privately):
> 
> Firstly, I'm not totally clear on the proper way to handle boolean event 
> attributes.

These are now all gone.


> It's also not entirely sure clear to handle attributes of events which
> reference other nodes.

These are also all gone now.


> There is also a particular oddity in the KeyboardEvent interface. It has 
> 4 boolean attributes (ctrlKey, shiftKey, altKey, metaKey) for key 
> modifiers, but its initializer (initKeyboardEvent) instead uses a 
> modifiersList argument, which is a "white space separated list of 
> modifier key identifiers to be activated on this object." So these two 
> events evaluate to the same thing?

This is moot as you can no longer create such events now.


On Thu, 16 Aug 2007, Henry Mason wrote (still privately):
> 
> That sounds fine but why not use the same for Cancelable and Bubbles 
> while we're at it?

These are gone too.


On Wed, 10 Oct 2007, Mihai Sucan wrote:
> 
> 1. Regarding section 6.2.2. "Connecting to an event stream" [2] it's 
> defined that:
> 
> "If such a resource completes loading (i.e. the entire HTTP response 
> body is received or the connection itself closes), the user agent should 
> request the event source resource again after a delay of approximately 
> five seconds."
> 
> The event source resource polling interval cannot be defined by Web 
> authors. Shouldn't this be configurable somehow? The 5 seconds delays 
> seems completely arbitrary.

This is now possible by including a field like this:

   retry: 5000

...which gives the retry interval in milliseconds.


> 2. In section 6.2.4. "Interpreting an event stream" [3], there's a typo 
> in the definition of the Class field, first phrase:
> 
> "This field gives *is* the interface used for the event, for instance 
> Event, UIEvent, MutationEvent, KeyboardEvent, etc. "
> 
> Correction: remove "is".

That section is gone now.


> 3. Why does the specification define "comments" in the stream?

For three reasons: one to allow static streams to be commented, two, to 
allow keepalive data to be sent without affecting anything else, and 
three, because otherwise we'd have to define lines starting with a colon 
in some other way, e.g. as an error, and that seemed no more useful.


> Given it does, why must the UA fire the message event when a comment is 
> sent? There's no use to do so, other than having an event listner which 
> could figure out "hey, something is coming from the server, maybe they 
> are comments".

Fixed. This is no longer required.


> 4. The specification does not make any statements about the security 
> implications of SSE.
> 
> There are possibilities to flood the UA with events.

This is already the case with any number of mechanisms (e.g. JS timeouts 
and DOM Events), and I wouldn't consider that a new threat.


> There's also the possibility to exploit security holes

This is the case with anything.


> or simply to crash the UA when sending specific binary data as 
> application/x-dom-event-stream.

The spec clearly indicates how all bytes are to be interpreted -- binary 
data that doesn't fit UTF-8 will be treated as U+FFFD. However, that's 
actually somewhat irrelevant -- the user agent should never crash. It's 
not a security concern in the spec if the user agent crashes, since the 
spec already requires that it not crash.


> The connection should be closed/forcefully terminated, in some cases.

Which cases?


> Regarding security, I would suggest to restrict the allowed chars in 
> field names to a-z, A-Z, 0-9, _ and -. This would allow the 
> implementation to terminate parsing the entire line earlier in cases 
> when garbage data is sent. If that's too limited, allow only certain 
> Unicode blocks (maybe from 0000 to 024F ?). [4]

Why not U+0000 to U+10FFFF?


> For now, the spec requires UAs to support the usage of binary-named 
> fields and values, which is rather overkill, I'd say.

I don't understand your usage of the term "binary" here.


> I haven't tested yet, but I'm quite sure existing implementations (such 
> as Opera) do not allow *any* field name, which would be valid within the 
> current spec rules.

This is now in fact required. :-)



On Thu, 11 Oct 2007, Kornel Lesinski wrote:
> 
> If an application requires to get each and every event only once -- for 
> example it's streaming log from the server, line after line, and should 
> not display duplicate lines and should not skip any -- in case of 
> network failure and automatic reconnection, it's tricky to resume events 
> at the right point.
> 
> Reliable resuming of event streams requires server to keeping track of 
> messages sent to each client, which is costly and only tells how many 
> were sent, not how many were sucessfully received. Alternatively server 
> could re-send number of messages, but that is wasteful and requires 
> client to filter out duplicates.
> 
> A solution to this would be for browser to trigger an event when error 
> occurs or before it's about to reconnect. This would allow script to 
> modify URL of <event-source> to include identifier/timestamp of last 
> message it has got, so the server could easily resume the stream from 
> the right point.

Something similar is now in the spec. Basically each event can include an 
"id" field, and if a reconnection attempt is made, the last "id" field 
received is sent back to the server along with the connection attempt.


> How about adding XMLHttpRequest-alike interface to <event-stream>? It 
> could use readystatechange event to signal connection progress, 
> reconnections and network errors. Since it is possible to implement 
> <event-stream> using XMLHttpRequest, it might be easy for 
> implementations to expose it.

This seems excessively complex. What's the use case?


On Fri, 12 Oct 2007, Mihai Sucan wrote:
> 
> There should be a way to track down if server-sent events are sent from 
> a remote source or not. I would say to require that UAs send all the 
> events from remote sources with an additional property: remote, string, 
> set to the event source URI/IRI.

The event now contains the origin of the event-source.


> This might seem an irrelevant, use-less property. However, this allows 
> authors to code Web applications which do not allow certain sensitive 
> events from being triggered by, possibly, malicious remote event 
> sources. This is just like the UA can tell if some events (like click) 
> were generated synthetically by scripts, or they are the direct result 
> of user interaction, for the purpose of allowing/disallowing popups.

Only synthetic events (MessageEvent events) can be created now.


> (somewhat off-topic idea) Thinking along the same lines, it would be 
> interesting to be able to tell synthetic events apart from events 
> triggered by user interaction, such as clicks, just like UAs do. Should 
> there be something like a boolean property, 'synthetic'? This would 
> always be true for remote events, obviously.

XBL2 suggests a "secure" property, IIRC, but this is a topic for the DOM 
Events spec, not HTML5.


On Sat, 5 Jan 2008, Henry Mason wrote:
>
> There's recently been some talk about completely removing HTML 5 section 
> 6.2, "Server-sent DOM events". I propose that rather than remove, we 
> revise.

Done. :-)

> The major concerns I've heard about section 6.2 include:
> 
> - Unnecessary dependency on DOM Events

This is reduced somewhat now, though DOM Events are still used.


> - Redundancy with already existing techniques, especially XMLHttpRequest

Consensus seems to be that this is not similar enough to warrant removal.


> - Complicated parsing of event fields

This is now simplified; there are only four types of field, and they are 
all either arbitrary strings or integers.


> - Inability to support cross-domain events (without the as-of-yet 
> unimplemented and untested Access-Control HTTP header mechanism)

This still relies on Access-Control, but that spec is becoming more mature 
so that shouldn't be a problem.


> - Continued problems of the 2 connection limit on HTTP server 
> scalability

I'm not sure how to fix that.


> I propose that we remove support for non-message events; that is, allow only
> events with MessageEvent interface. This will make implementations easier, as
> UAs will only need to parse the "Bubbles", "Cancelable", and "data" fields.

Done, except I've even removed Bubbles and Cancelable. (I have, however, 
allowed events to be named if desired, though they all use MessageEvent.)


> The critically cool part, however, is that since MessageEvents store 
> their domain and URI origin, it will be safe to allow for cross-domain 
> messaging through this server-sent events.

The danger with cross-origin event sources isn't hostile event sources, 
it's hostile Web pages sourcing victim event sources from origins that do 
not expect to be exposed, e.g. intranet sites. That's what Access-Control 
is protecting.


On Mon, 7 Jan 2008, Anne van Kesteren wrote:
>
> Opera also supports the "Event" and "Target" fields. I think we'd be 
> fine if the latter is removed, but the former is really useful as you 
> can then have separate handlers to deal with the incoming data. (That 
> they all implement the same MessageEvent interface is fine.)

Kept Event (renamed 'event') and dropped "Target".


On Mon, 7 Jan 2008, Dan Mosedale wrote:
> Anne van Kesteren wrote:
> >
> > Is there any realistic solution to this other than to use separate 
> > domains and have cross-domain working?
>
> Simply get rid of, or significantly raise, the limit?  Standards work 
> related to this probably belongs (and conceivably is already happening) 
> in an HTTP protocol forum.  I did some poking around recently, and 
> browser behavior related to this limit is fairly varied, depending on 
> context.

Any luck with this?


On Tue, 8 Jan 2008, Kornel Lesinski wrote:
> 
> Sharing of event-source connection between many pages/windows would help 
> tremendously.
> 
> Currently each <event-source> on every page gets its own connection. 
> This makes use of it as standard element of page's design (e.g. live 
> stock ticker in a sidebar on financial portal) impractical - for every 
> page in every window browser would open separate connection, easily 
> reaching the limit.
> 
> This could be improved, if connection to event source was handled 
> outside of page's context, in a global pool of stream connections. 
> Browser would never use more than one connection to the same source and 
> would use one connection to trigger events on all pages that subscribe 
> to the given source. As a bonus, browser could also keep connection open 
> while user navigates between pages instead of closing and re-opening 
> stream connections each time.
> 
> The downside is that applications couldn't rely on seeing every event in 
> the source, so they couldn't rely on incremental updates. To remedy 
> this, an additional feature, like checkpoints in the stream and 
> replaying of past events would be neccessary.

I think we want to keep shared resources very localised, and we would 
probably localise that to simply a shared worker environment, as e.g. 
proposed in:

   http://hixie.ch/specs/dom/messages/0.9
   http://hixie.ch/specs/dom/workers/0.9

We could just expose RemoteEventTarget to workers.


On Mon, 7 Jan 2008, Stewart Brodie wrote:
> 
> I have an implementation - it does precisely that, as I mention above.

Cool! What do you think of the changes here? Are they acceptable?


> Removing the requirement to support anything other than MessageEvent 
> class of events would certainly be a tremendous simplification.  I'm not 
> sure whether or not it is a good idea - it would leave us needing to 
> perform all sorts of string parsing in our JS if we wanted to issue 
> other types of event.  In fact, if this simplification were to be made, 
> I'd probably have to retain this ability for compatibility with our 
> existing applications.

I've changed the MIME type to allow easy migration in existing 
implementations -- just support both types independently. (Other types can 
also be supported, e.g. for experimentation of new ideas.)


On Thu, 21 Feb 2008, Alex Russell wrote:
> 
> Perhaps, but server-sent events are *still* silent on the most important 
> points:
> 
> 	* how do you detect connection close?

You don't, it reconnects automatically. Do you think we should expose more 
state here? How should we do that, given that you can register multiple 
sources per element? I'm certainly open to suggestions. :-)

Cheers,
-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'