[whatwg] WebWorkers and images

[Jorge]

On 13/01/2011, at 15:41, Boris Zbarsky wrote:
> On 1/13/11 7:24 AM, Jorge wrote:
>> Not too long ago, the browsers did allow timeouts of less than 10ms.
> 
> Uh, no.  Not too long ago browsers did not allow timeouts of less than 10ms, ever.

Last time I checked that in 2008, most Mac browsers had a +new Date() and a SetTimeout resolution of ~1ms down to ~ 0 ms, unlike Windows browsers (except Safari PC) which had a minimum T of ~ 16ms IIRC.

There's still and "old" mac browser you can check today ( iCab 3.5 ) whose setTimeout(f,0) still has a period of < 1 ms.

Current Mac browsers behave ~ as you say, except Opera, that converts 0ms timeouts to 10ms, but for anything >= 1ms honors the ms parameter. Tested with http://jorgechamorro.com/cljs/097/

> At this point the only browser that does for repeated timeouts is Chrome, which allows 4ms.  Various browsers allow timeouts up to some small level of nesting to run arbitrarily quickly; that started because it happens to make JSBench run faster due to the broken way that benchmark is constructed.
> 
> In any case, to implement the 4ms thing on Windows Chrome has to have a dedicated thread polling the multimedia timer, which is a huge PITA in terms of things like wakeups and battery life... And for what it's worth on my Mac they don't manage to hit the 4ms thing either; they get closer to 4.7ms on average, though I haven't looked at the distribution.  And if I lower Gecko's clamp to 4ms, I get a similar 4.7ms average on Mac, so that might just be an OS limitation too.

On a Mac I get :

SetTimeout(f,0) :
Safari (nightly) : 4.3ms
FF4: 10.4ms
Opera 11: 10.3ms
Chrome 9ß: 4.3ms

SetTimeout(f,1) :
Safari (nightly) : 4.3ms
FF4: 10.4ms
Opera 11: 1.2ms <--- **** ( ! ) ****
Chrome 9ß: 4.3ms

> It looks like other browsers will be following suit on the 4ms thing eventually for two reasons: the HTML5 spec specs it and there are lots of bogus performance "benchmarks" that measure JavaScript execution speed across timeouts.  Now I suspect browsers won't actually _deliver_ decent 4ms timers on Windows unless they jump through the hoops that Chrome does with a polling thread; Windows just doesn't give you a sane way to deal with times less than a tick (10ms on single-processor systems, 15ms on multiprocessor ones).  But they might be able to deliver something that fires every 4ms on average (firing immediately sometimes and after 10-15ms other times).

If the goal is to protect the users from (badly written) programs that may drain the mobile's or laptop's battery, it's a bit sad to have to go down this route. 

>> Why was the>= 10ms minimum timer duration spec'ed this way ?
> 
> The current spec draft says the floor is 4ms.

That's much better than 10.

>>> Note that if your computational work is entirely working with
>>> ImageData, you can send the ImageData to a thread.  It's limiting (you
>>> can't blit images to the canvas that way, since you don't have the
>>> Canvas interface), but it may be enough for your case.
>> 
>> I've tried once to improve a full-screen animation like that, and found that the cost of passing the data back and forth to the worker is so high
> 
> A full-screen on my screen would mean a 9.2MB imagedata.  So depending on how often you're trying to pass it back and forth, just creating the copies could be pretty expensive, yes.  If it's being round-tripped through a string or something it would be even worse.
> 
> It would be interesting to have an API that allows passing an imagedata object (not a copy) to a worker.  Such an API would have to make the data disappear on the caller's side.  That could be implemented reasonably efficiently using shared memory (either directly with threads or using shared memory segments with processes).
> 
> That said, I would be interested in seeing your testcase for this; in my experience bottlenecks in situations like this can be in ... surprising places.

I'm quite confident, the time was being spent serializing/deserializing in order to pass the objects via .postMessage( as text ), because there was no other way to pass them.

>> That was passing the objects serialized as text messages.
> 
> And serializing in JS?  Yeah, that would be pretty slow!
> 
>> Perhaps with structured clones, the situation may have been improved a bit.
> 
> It should have, even with the copying that probably happens now.  Worth retesting.

Do current browsers implement the structured clones already ?

>> But I think that the workers desperately need a mechanism that permitted to pass objects *quickly*, and *quickly* most likely means by reference not by copy.
> 
> Then it needs to be something that passes the object and _forgets_ it on the caller side.

Exactly. I think that's the key, forgets or throws on access, but must be unreachable once passed.

>> To preserve shared-nothingness, the passed object (and the object's children) could be made unreachable (somehow, don't ask me)
> 
> This is actually not all that bad for imagedata: just deallocate its storage on the caller and make any access to the buffer throw.  The key is that you don't care that you have to copy things like the width/height; you just don't want to copy the giant byte array.  So you move the byte array, and deny all access to its elements after that. Since the elements are never pointed to by reference from JS, this works.
> 
> For arbitrary objects this is harder, but could be done, actually. Gecko already does something similar for Window objects when their origin changes: you might still have a reference to the original object, but you can no longer actually touch any of it.  Under the hood, this is implemented in a way that could be used for sending objects to a worker too, I think.

Good. So you think it might be feasible ?

I think so too for objects composed only of data properties, but what about methods ? getters ? setters ? and prototypes ?

Wrt these, perhaps there's some compromises to take:

- functions can't be passed because they carry context with them: thus no methods allowed.
- prototypes complicate things, and there's always the Object.prototype at the end of the chain, with its methods.

So these objects, for this purpose, should be "simple", without any methods, and with an empty (.__proto__ = null) prototype chain.

Still, the ability to pass between threads *true* modifiable objects of *any*size*, by reference-> in a fraction of a nanosecond, even with these tiny limitations, would still be a great, a huge huge gain.
-- 
Jorge.