[whatwg] WebSocket bufferedAmount includes overhead or not

[Ian Hickson]

(If you haven't posted on this thread before, please see the bottom of 
this e-mail -- I'm still looking for more opinions.)

On Thu, 25 Mar 2010, Niklas Beischer wrote:
> 
> What about having bufferedAmount represent the number of bytes 
> (including overhead) buffered by the WebSocket, for flow control 
> purposes, and adding a new indicator (bufferedMessages) representing the 
> number of messages that are not fully pushed to the network? Since the 
> API is message based there is, besides flow control, little reason to 
> specify how much of a particular message has been sent, right?

What would the use case be for the second one? As far as I'm aware there's 
only one use case here: making it possible to saturate the network but not 
over-saturate it (i.e. to send data at the exact rate that the network can 
take it, without getting behind by sending more data than the network can 
handle, and without sending so little that the network is ever idle).


On Thu, 25 Mar 2010, Olli Pettay wrote:
>
> The problem here is that how can the API describe what the 
> bufferedAmount actually is. And since the underlying protocol may 
> change, the values in bufferedAmount may change.

Why would the underlying protocol change without the API changing?


> Hmm, would it be enough to have just bufferedMessages, and remove 
> bufferedAmount.

That wouldn't address the use case when the rate of message sending is low 
but the size of messages is high (i.e. when you want to only have an 
average of one message queued at a time).


On Thu, 25 Mar 2010, Erik Möller wrote:
> 
> Just to clarify then, the two use cases we're trying to accommodate are:
> a) The client wants to be able to limit the data sent over the wire to X kb/s.
> b) The client wants to make sure some earlier message(s) has been sent before
> queuing a new one.
> 
> Is that correct, or are there any other use cases anyone had in mind?

I wasn't aware of (b). What's the case where you would want to know when a 
particular message has hit hte network layer, without needing to know that 
it hit the wire, or, more importantly, hit the server? I'd have thought 
that cases like that would want to confirm receipt, rather than just 
confirm send, but I'm not clear on what the exact case is.


On Thu, 25 Mar 2010, Olli Pettay wrote:
>
> What I'd expect the API to tell me is, for example, that if I call 
> ws.send(foo), and nothing is yet sent over the network, what is the 
> exact value of bufferedAmount.

I don't see a problem with defining this. I agree that if we include 
overhead that it should be defined, but just saying that it's "the number 
of bytes to be sent that have not yet been sent to the network" does 
define it, as far as I can tell.


> Again, I'd really wish to keep the API to work reasonable similar way as 
> XHR+progress events where protocol overhead isn't reported.

The use case for progress events is radically different -- there you want 
to show UI. This isn't about UI. I think it makes sense that such 
different use cases would have different behaviours.


On Thu, 25 Mar 2010, Perry Smith wrote:
> 
> I do kernel level networking support as my day job.  Trying to get 
> javascript to not saturate the network is not going to work.  There are 
> vast amounts of technology that govern how data flows in a network and a 
> primitive javascript app is woefully under equipped.

It's certainly not well-equiped, I agree. But the "saturate" examples 
here:

   http://damowmow.com/playground/demos/websocket/

...show that it is indeed possible to do it pretty well in at least some 
relatively normal network conditions with just the current API.


> I like the idea of bufferedAmount but it could almost be "arbitrary 
> units" -- just something that the app can determine "Hey!  I'm not 
> making any progress" or perhaps give the javascript a way to keep the 
> users updated as to the progress.  But making this into a way to not 
> saturate the network is not going to work.

Why not?


> As far as adding in any protocol overhead, there is no way an 
> application is going to know what that is unless you split the protocol 
> stack at some point. I don't see why the application level would want to 
> know anything about the lower level protocol.  At the same time, if an 
> implementation wants to add in some of the overhead that it sees, that 
> is still going to give the application all the tools it needs to 
> implement whatever it can.

Whether it includes the overhead or not is mostly academic for authors, I 
think. It's only relative numbers that actually matter.


> I would focus on words like "monotonically decreasing" after a send.  
> And "eventually ends up at zero".  Those two things are what I'd like to 
> be sure are true.

It's not clear to me how the current text would fail to match those 
conditions.


On Thu, 25 Mar 2010, Olli Pettay wrote:
> 
> bufferedAmount is not only about "not saturating".

What else is it about?


> from API point of view, why should bufferedAmount contain some bytes 
> which aren't specified in the API spec?

I think viewing the API spec and the protocol spec as separate is a 
mistake. They are one document:

   http://www.whatwg.org/specs/web-apps/current-work/complete.html#network

It just happens that different parts are published in other places and 
that we're working with other working groups on those parts.


It seems that the consensus is now leaning towards changing the spec again 
to include the overhead, but I haven't changed it yet because I don't want 
to be flip-flopping back and forth -- if we change this, I don't want to 
change it back. I think the use cases I know of are addressed whether we 
include overhead or not, so from an objective perspective I don't think it 
really matters, which makes this more of an opinion thing. I'd encourage 
anyone else with an opinion on this to make their opinion known. (Yes, I'm 
actually _asking_ for people to suggest colours for this bikeshed!)

Cheers,
-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'