[whatwg] Workers feedback

[Ian Hickson]

Summary:

 * added window.location.resolveURL(). I haven't added it to the Location 
   in Workers -- should I?

 * no other normative changes.


On Thu, 13 Nov 2008, Jonas Sicking wrote:
>
> Actually, i think we should remove the location accessor as well. I 
> can't think of a common enough use case that warrants an explicit API. 
> You can always transfer the data through postMessage.
> 
> What are the use cases? Also note that we can't use it with shared 
> workers since they can be connected to several pages from different 
> uris.

On Fri, 14 Nov 2008, Aaron Boodman wrote:
> 
> In Gears, authors asked us for a location object because they had 
> applications that could be served from different origins. Also in Gears, 
> workers can be accessed across origins, so incoming messages need to be 
> validated that they are from the correct origin. It's more convenient 
> for a worker to access its own origin through an API the UA provides 
> than to bake it into the script or send it in the first postMessage().
> 
> Although HTML5 workers don't have the ability to be accessed across 
> origins, you do have the ability to receive ports that are connected to 
> other origins, right? So this might still be an issue.
> 
> Other than that, I can't think of any specific use cases. It seemed like 
> a generally useful API to have access to.

It seems somewhat useful to me too. Is the implementation cost high?


On Thu, 13 Nov 2008, Jonas Sicking wrote:
> >
> > It returns the script's URL, not the page's.
> 
> Oh?! Then I understand even less what the use case is. This is something 
> that doesn't exist for <script> and i've never heard anyone ask for it 
> (granted, that is not proof that no one wants it).

On Fri, 14 Nov 2008, Alexey Proskuryakov wrote:
> 
> Actually, this exists for <script> is you say that it returns the URL 
> that the script execution context uses for resolving relative URLs. It 
> just so happens that for <script>, it's document URL, and for workers, 
> it's script URL.

On Fri, 14 Nov 2008, Jonas Sicking wrote:
> > 
> > Actually, this exists for <script> is you say that it returns the URL 
> > that the script execution context uses for resolving relative URLs. It 
> > just so happens that for <script>, it's document URL, and for workers, 
> > it's script URL.
> 
> That works the same in Workers iirc, without the need for a .location 
> API.

How is a script supposed to know what base URL is used to resolve URLs if 
we remove .location?


On Fri, 14 Nov 2008, Jonas Sicking wrote:
>
> Actually, come to think of it, what is the BaseURI for workers? I.e. 
> what URI is importScripts and XHR resolved against?

   http://www.whatwg.org/specs/web-workers/current-work/#base-urls


On Fri, 14 Nov 2008, Jonas Sicking wrote:
> > 
> > "The base URL of a URL passed to an API in a worker is the absolute 
> > URL given that the worker's location attribute represents. Both the 
> > origin and effective script origin of scripts running in workers are 
> > the origin of the absolute URL given that the worker's 
> > locationattribute represents."
> 
> Hmm.. this makes a lot of sense for importScripts, but for XHR you 
> probably want the baseURI to be that of the opening page, since it's 
> quite likely that the opening page gave you a URI to open and process.
> 
> Of course that would be quite confusing (different baseURIs for 
> different APIs), as well as impossible for shared workers as they don't 
> have a single document as opener.
> 
> What we need is an API for resolving relative URIs, that way scripts can 
> at least do the resolving manually. We could also add an API for getting 
> the baseURI of the document on the other side of a port (should possibly 
> live on the message event).

On Fri, 14 Nov 2008, Aaron Boodman wrote:
> 
> My expectation was that the base URI would always be the URI of the 
> worker. I think of opening a worker a lot like starting a new process, 
> or opening a new window. I would expect that the new process has its own 
> base URI which is the same URI as the script it is running.

On Fri, 14 Nov 2008, Michael Nordman wrote:
> 
> My mental model for workers is a GUI less document almost. A dedicated 
> worker is akin to a nested iframe. A shared worker is akin to a top 
> window or tab. In that model, base is src of the resource loaded into 
> the context.

On Fri, 14 Nov 2008, Jonas Sicking wrote:
> 
> I think in many cases for XHR that is probably not what is the most 
> useful. Consider for example:
> 
> example.html:
> 
> w = new Worker("http://docs.google.com/MSWordParser.js");
> w.onmessage = function (e) {
>   doneLoadingDoc(JSON.parse(e.data));
> }
> w.postMessage("/documents/report.doc");
> 
> 
> MSWordParser.js:
> 
> onmessage = function(e) {
>   xhr = new XMLHttpRequest();
>   xhr.open("GET", e.data, false);
>   xhr.send();
>   res = mainParse(xhr.responseText);
>   postMessage(JSON.stringify(res));
> }
> 
> 
> The above won't work since the URL is relative to the worker JS file, 
> not relative to example.html. So an absolute URI always needs to be used 
> which sort of sucks as is unintuitive from the point of view of most DOM 
> APIs today.
> 
> I don't really have a good solution to propose though. We might need to 
> add an API for resolving relative URIs and require that scripts use that 
> manually.

I've added window.location.resolveURL for this case.


On Fri, 14 Nov 2008, Alexey Proskuryakov wrote:
> Nov 14, 2008, × 2:30 AM, Ian Hickson ÎÁÐÉÓÁÌ(Á):
> > 
> > I believe that the idea that the API for shared and dedicated workers 
> > should be the same is misguided. The spec used to make the two cases 
> > identical. The result was confusion, and the dedicated case was much 
> > more complex than necessary.
> 
> This is the strongest argument for having separate interfaces of all 
> presented, but in my opinion, it misses some important data points:
>
> - Was introducing a new interface the only way to resolve the confusion?

I'm sure it it wasn't.


> Maybe renaming a method or two would have had a positive effect?

Maybe.


> - Did it help resolve the confusion?

I believe so; it made dedicated workers much simpler.


> - What kind of danger the confusion was? Would authors write under- 
> performing and unreliable code, or refuse to use such a cumbersome API 
> at all? Or was is just a momentary delay for some, resolved quickly and 
> harmlessly?

A mixture. Mostly though it would be matter of making people use more 
boiler plate code instead of understanding what they were doing.


> I don't remember any of these discussed here.

It was discussed here and in person, I believe.


> > Shared workers and dedicated workers are fundamentally different and 
> > have different needs, and we should expose these needs in ways 
> > optimised for the two cases.
> > 
> > The basic need is that dedicated workers be able to have a two-way 
> > communication channel with their creators, and shared workers be able 
> > to have a two-way communication with each user of the worker.
> 
> I think that this argument is false. It is normal for a single API to 
> support multiple use cases.
> 
> Replace Worker with XMLHttpRequest, and we end up with separate 
> interfaces for GET and POST.

I think the difference between dedicated and shared workers is greater 
than the difference between GET and POST. It's more like the difference 
between audio and video -- and we do have HTMLAudioElement and 
HTMLVideoElement, both inheriting fomr a common base, just like workers.


> > For dedicated workers though that's way more complexity than we want 
> > to require of authors -- why do they have to listen for a port when 
> > there will always be exactly one? So it makes sense to use
> 
> It is not true that there will always be one - additional ports can be 
> passed in via postMessage().

Sure, but that's true regardless of dedicated or shared workers. I'm 
talking about the ports set up by the API itself.


> > Now, we've at this point made the two different already, so as to 
> > simplify the dedicated worker, so we could (and the spec does) make 
> > the dedicated worker even simpler while we're at it.
> 
> Returning to the XMLHttpRequest example, we really can combine open() 
> and send() for XMLHttpRequestGET, but not for XMLHttpRequestPOST. 
> Generally, the interfaces can be very different if we try to.

I wouldn't hold up XMLHttpRequest as an example of good API design. :-)


> > So, we end up with what the spec has now.
> > 
> > I think what we have now is better than making dedicated and shared 
> > workers superficially the same (as the spec used to be, and as the 
> > people involved in this thread argued was bad) is more confusing for 
> > authors.
> 
> I'd be more that happy with a separate interface if the objects actually 
> behaved differently. One example of a good reason to have separate 
> interfaces was recently proposed here: shared workers should outlive 
> their creators. This is the sort of difference that would make having a 
> separate API reasonable, in my opinion.

You don't think that the way that a handle to a shared worker can be 
obtained dynamically without contact with the original creator is enough 
of a difference? It's a pretty big functional difference IMHO!


> > At this point, if the only arguments for changing the API are "it's 
> > confusing for authors", then I'd rather not change the API. We got to 
> > where we are today by carefully considering what would be better for 
> > authors. We could continue going back-and-forth and reverting earlier 
> > decisions until the cows come home, but I see no benefit to doing so.
> 
> I don't think it's inappropriate to continue back-and-forth until there 
> is at least one reasonably complete implementation validating the spec. 

I'm happy to change the spec based on good reasons, but I don't want to 
change the spec because "it's confusing" when both alternatives have 
people claiming that they are confusing.


On Fri, 14 Nov 2008, Aaron Boodman wrote:
> 
> Thanks for taking the time to read and understand all the feedback.

My pleasure.


> Although this is not my most preferred design for the API, I can live 
> with it. I'm happy that we removed startConversation(). I think that was 
> just extra complexity on top of an already large API.

Right-o.


> As for putting forward contradictory suggestions, I apologize for that. 
> In the future, I will try to form final opinions before opining.

No worries, I was just amused. :-)

It does make my job harder if the same person puts forward to opposing 
viewpoints though. :-)


On Sun, 16 Nov 2008, Shannon wrote:
> >
> > The "user-agent specified amount of time" delay is implemented by 
> > every major browser for every script they run today.
> >
> > How can people write clean code when they have no consistency in how 
> > long their scripts will run (or if they will run at all)?
> >
> > Why is this any different?
>
> Why does that matter? I think you're asking the wrong question. As 
> designers of a new spec the question should be "how can we fix this?". 
> If the answer is to include a mandatory cleanup window for ALL scripts 
> then that should be considered (even if that window is 0 seconds).

Defining a timeout in seconds makes no sense, since it would mean 
different amounts of code on different hardware and with different 
interpreters.


> > You need to do that anyway to handle powerouts and crashes.
>
> That was the point of my concern. Given that the only 100% reliable 
> cleanup window is 0 seconds it would be more consistent (and honest) to 
> make that the spec.

It may be possible to deal with no cleanup, but that doesn't mean it's 
optimal. For example, the server might be keeping some resources around 
for each client, with a timeout that expires after an hour of inactivity. 
But that doesn't mean that it's not better to tell the server "by the way, 
I'm closing now, so you can release those resources".


> Offering a cleanup window of uncertain length is somewhat pointless and 
> bound to cause incompatibilities across UAs.

This argument is, as noted, demonstrably false, because we already have 
script running under a timeout window everywhere today, and it doesn't 
cause great problems.


> Is there a strong argument against making 0 seconds mandatory, given 
> that anything else is inconsistent across UA, architecture and 
> circumstance?

Yes, as noted above.


On Thu, 13 Nov 2008, Jonas Sicking wrote:
> > 
> > The above would also mean that the timeout would fire even after the 
> > document has been closed, which is not what we want.
> 
> True, you also need to define that when a document is closed pending 
> timers and in-progress XHR requests, are canceled.

How is this different from what the spec says now?


On Fri, 14 Nov 2008, Alexey Proskuryakov wrote:
> 
> Oh, that's my mistake - I totally didn't expect that it could have such 
> side effect. It seems weird that addEventListener("message", ...) does 
> not have such effect, does it?

It doesn't, no. Indeed start() is there mainly for that case (so that 
you can register multiple listeners and then start it).


> > > In an async processing model, there is simply no way for the 
> > > receiver to have a list of all objects that were posted to it - it's 
> > > exactly the reason for the existence of the queue that events are 
> > > delivered asynchronously and cannot be peeked before being 
> > > delivered. For example, in a multi-process implementation, these 
> > > events may still be across process boundary.
> > 
> > It actually doesn't really matter if there is something that has been 
> > posted but not yet received, because that is indistinguishable (as far 
> > as I can tell) from the case of the worker having shut down a split 
> > second before that object was posted.
> 
> I'm not sure what state you mean by "shut down" here - the spec does not 
> define this, and shutting down a side of an async communication channel 
> is complicated (see e.g. a TCP/IP state diagram). Anyway, the contents 
> of "the worker's ports" is used for defining "active needed worker" and 
> "suspendable worker" further on, which are concepts that are very 
> important for worker lifetime definition. If the ports in event queue 
> are not important, then the spec should not say that they are included 
> in "the worker's ports". This would resolve the concurrency problem, but 
> I don't think that the resulting behavior would be desirable.

If we remove the requirement altogether then, as you say, the behaviour 
would be undesireable (most notably, a worker would instantly garbage 
collect when created).

However, I don't see the problem with the requirement. If an event is 
still not across the process boundary, then the worker will still be kept 
alive by the port over which the new port is being sent. The only way a 
port can be added to the list when the list would otherwise be empty is 
during the creation of a shared worker, but that is all done with a lot 
more marshalling so that if the worker exists, it will be used (and won't 
GC), and if it doesn't, a new one will be created, without a race 
condition.

I guess I could stop mentioning that queued ports are part of the list. It 
might remove the confusion. I've commented it out for now. I don't think 
this makes a practical difference.


> > > It is not possible to have a symmetric relationship in an 
> > > asynchronous messaging model - we need a multi-step 
> > > entagling/unentangling protocol, so the relationship is necessarily 
> > > asymmetric. One can't freeze another process (or really, even 
> > > another thread) to change something in it synchronously.
> > 
> > The above is not a requirement, it's just a description of the 
> > concept. I don't think anything actually depends on it being 
> > symmetric; all the parts that actually entangle ports have (or, are 
> > intended to have, maybe I missed some) pretty well-defined 
> > synchronisation points.
> 
> OK, say there is a pair of entangled ports in different 
> threads/processes, portA and portB. We concurrently post both with 
> postMessage, which causes the ports to be cloned. From the point of view 
> of first thread, PortA is now unentangled, and portA' is entangled with 
> portB. From the point of view of second thread, PortB is unentangled, 
> and portB' is entangled with portA.
> 
> Next, threads send asynchronous notifications to each other, asking to 
> update entangling information. First thread's notification asks portB to 
> become entangled with portA'. So, portB will need to forward this 
> notification to portB' (and possibly further, because portB' may have 
> been posted and cloned again). This already is unduly complicated.

The complication here seems to be in the way you are implementing this. 
Port entangling should be atomic across threads -- when you are sending a 
port over another channel, you should block both threads, create the new 
object, update the information, shunt all pending messages over, and then 
resume the threads. If you implement the actual IPC using, say, a Unix 
socket, then you can just pass the actual socket along and do the same 
thing without blocking. Either way, the point is that as soon as the port 
is passed to another thread, the original thread (and MessagePort object) 
are dead, and would never receive messages.


> Now consider that all these ports need to have destroyed sooner or 
> later, but not too soon. This basically means that we now have a 
> many-to-many distributed GC system. It was bad enough when we had to 
> garbage protect ports between threads, because this required 
> modification of the JavaScript interpreter to support a certain case of 
> distributed GC. But this example basically shows that we need a 
> full-blown distributed GC system in order to implement port cloning.

I agree that if you implement this by having the threads themselves act as 
proxies to the ports that you end up with all kinds of crazy problems.


> > For example, any method that entangles two ports blocks until both 
> > threads are synchronised and entangled.
> 
> This will cause deadlocks - if portB' is sent to the first thread as 
> portB'' in the above scheme, the lock will not let synchronization ever 
> finish.

Could you elaborate on this? I'm not sure I follow what you mean. If you 
mean that two ports in two threads are posted to each other's threads at 
the same time, then deadlock would only happen in a naive implementation 
that didn't keep pumping its message queue while waiting for a response 
from the other thread. Instead what you would want to do is to ask for a 
semaphore to communicate with the other thread, and if you don't get it, 
see if anyone wants to communicate with you, and if they do, let them do 
whatever they want, and then try again.

Sure, it's not simple. But it's a whole lot better than proxying messages 
around using an "address change" mechanism like the post office, where one 
could easily imagine a message chasing the port through multiple threads 
desperately trying to catch up with the port so it could actually be 
delivered!

(Personally I would prefer passing sockets around.)


> > (The spec is somewhat implicit about this, but the intent is that 
> > workers really be implemented either as two system threads, one doing 
> > communication and one running the JS, or by one system thread that 
> > runs the JS in an interruptible fashion. In particular, doing 
> > something that synchronises with a worker isn't expected to have to 
> > wait for that worker to finish running its current JS.)
> 
> The JS thread will need to be interrupted in any case - we certainly 
> don't want it to read a half-written pointer from memory or something. 
> Adding memory barriers around access to data that can be modified 
> externally is not sufficient, because MessagePort algorithms are not 
> designed in a lock-free fashion (lock-free algorithms that only rely on 
> read/write atomicity do exist, but these aren't such). Locking around 
> all MessagePort functions will cause deadlocks, as demonstrated above, 
> and is generally against best practices. A middle ground may exist, but 
> it may not, and it's definitely hard to find.

I'm certainly open to changing the algorithms around if a better solution 
exists in a manner that gets the same behavior. I'm certainly no expert on 
the topic (as I'm sure the above responses have shown).


> I don't think that pursuing a design that relies on locking is 
> particularly promising - for the same reason that workers do not expose 
> shared data to JS programmers, it is highly desirable to not rely on 
> shared data in implementations, too (except for a few well understood 
> constructs, such as an event queue). So, I think that the specs (Web 
> Workers and HTML5 channel messaging) should be cleaned up from anything 
> that mentions synchronous access to entangled port's data structures to 
> really be verified for correctness. This is not straightforward, and may 
> seriously affect the API - e.g., I doubt that passing MessagePorts 
> around is implementable with reasonable complexity, and there is not a 
> lot of use in MessagePorts if they cannot be passed around.

I agree with the latter. I'm not sure how to do the former.


On Fri, 14 Nov 2008, Aaron Boodman wrote:
>
> Jonas asked me about this in IRC, and specifically why I suggested 
> appCodeName (since everyone claims they are Mozilla). My thinking was:
> 
> a) Browser detection is a mess, scripts use everything on the
> navigator object (often incorrectly) to try and figure out which
> browser is which
> b) HTML5 may eventually define something better
> 
> Therefore, I could see two rational things for workers to expose: all of 
> whatever window.navigator has today, or whatever the new better API is 
> that HTML5 defines. I guess doing some subset of what window.navigator 
> does today could also be rational, but I don't know how to pick the 
> subset.

I did a lot of research when speccing Navigator, and ended up with:

  readonly attribute DOMString appName;
  readonly attribute DOMString appVersion;
  readonly attribute DOMString platform;
  readonly attribute DOMString userAgent;

This excludes anything that is not implemented in three or more browsers, 
and excludes anything that always returns a constant regardless of 
browser. I may add more as people find reasons to add the others.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'