[whatwg] WebSocket bufferedAmount includes overhead or not

[Olli Pettay]

On 3/25/10 11:50 PM, Ian Hickson wrote:
>
> (If you haven't posted on this thread before, please see the bottom of
> this e-mail -- I'm still looking for more opinions.)
>
> On Thu, 25 Mar 2010, Niklas Beischer wrote:
>>
>> What about having bufferedAmount represent the number of bytes
>> (including overhead) buffered by the WebSocket, for flow control
>> purposes, and adding a new indicator (bufferedMessages) representing the
>> number of messages that are not fully pushed to the network? Since the
>> API is message based there is, besides flow control, little reason to
>> specify how much of a particular message has been sent, right?
>
> What would the use case be for the second one? As far as I'm aware there's
> only one use case here: making it possible to saturate the network but not
> over-saturate it (i.e. to send data at the exact rate that the network can
> take it, without getting behind by sending more data than the network can
> handle, and without sending so little that the network is ever idle).
>
>
> On Thu, 25 Mar 2010, Olli Pettay wrote:
>>
>> The problem here is that how can the API describe what the
>> bufferedAmount actually is. And since the underlying protocol may
>> change, the values in bufferedAmount may change.
>
> Why would the underlying protocol change without the API changing?
>
>
>> Hmm, would it be enough to have just bufferedMessages, and remove
>> bufferedAmount.
>
> That wouldn't address the use case when the rate of message sending is low
> but the size of messages is high (i.e. when you want to only have an
> average of one message queued at a time).
>
>
> On Thu, 25 Mar 2010, Erik M�ller wrote:
>>
>> Just to clarify then, the two use cases we're trying to accommodate are:
>> a) The client wants to be able to limit the data sent over the wire to X kb/s.
>> b) The client wants to make sure some earlier message(s) has been sent before
>> queuing a new one.
>>
>> Is that correct, or are there any other use cases anyone had in mind?
>
> I wasn't aware of (b). What's the case where you would want to know when a
> particular message has hit hte network layer, without needing to know that
> it hit the wire, or, more importantly, hit the server? I'd have thought
> that cases like that would want to confirm receipt, rather than just
> confirm send, but I'm not clear on what the exact case is.
>
>
> On Thu, 25 Mar 2010, Olli Pettay wrote:
>>
>> What I'd expect the API to tell me is, for example, that if I call
>> ws.send(foo), and nothing is yet sent over the network, what is the
>> exact value of bufferedAmount.
>
> I don't see a problem with defining this. I agree that if we include
> overhead that it should be defined, but just saying that it's "the number
> of bytes to be sent that have not yet been sent to the network" does
> define it, as far as I can tell.
>
>
>> Again, I'd really wish to keep the API to work reasonable similar way as
>> XHR+progress events where protocol overhead isn't reported.
>
> The use case for progress events is radically different -- there you want
> to show UI. This isn't about UI. I think it makes sense that such
> different use cases would have different behaviours.
>
>
> On Thu, 25 Mar 2010, Perry Smith wrote:
>>
>> I do kernel level networking support as my day job.  Trying to get
>> javascript to not saturate the network is not going to work.  There are
>> vast amounts of technology that govern how data flows in a network and a
>> primitive javascript app is woefully under equipped.
>
> It's certainly not well-equiped, I agree. But the "saturate" examples
> here:
>
>     http://damowmow.com/playground/demos/websocket/
>
> ...show that it is indeed possible to do it pretty well in at least some
> relatively normal network conditions with just the current API.
>
>
>> I like the idea of bufferedAmount but it could almost be "arbitrary
>> units" -- just something that the app can determine "Hey!  I'm not
>> making any progress" or perhaps give the javascript a way to keep the
>> users updated as to the progress.  But making this into a way to not
>> saturate the network is not going to work.
>
> Why not?
>
>
>> As far as adding in any protocol overhead, there is no way an
>> application is going to know what that is unless you split the protocol
>> stack at some point. I don't see why the application level would want to
>> know anything about the lower level protocol.  At the same time, if an
>> implementation wants to add in some of the overhead that it sees, that
>> is still going to give the application all the tools it needs to
>> implement whatever it can.
>
> Whether it includes the overhead or not is mostly academic for authors, I
> think. It's only relative numbers that actually matter.
>
>
>> I would focus on words like "monotonically decreasing" after a send.
>> And "eventually ends up at zero".  Those two things are what I'd like to
>> be sure are true.
>
> It's not clear to me how the current text would fail to match those
> conditions.
>
>
> On Thu, 25 Mar 2010, Olli Pettay wrote:
>>
>> bufferedAmount is not only about "not saturating".
>
> What else is it about?
>
>
>> from API point of view, why should bufferedAmount contain some bytes
>> which aren't specified in the API spec?
>
> I think viewing the API spec and the protocol spec as separate is a
> mistake. They are one document:
>
>     http://www.whatwg.org/specs/web-apps/current-work/complete.html#network
>
> It just happens that different parts are published in other places and
> that we're working with other working groups on those parts.
>
>
> It seems that the consensus is now leaning towards changing the spec again
> to include the overhead,
Well, what if the overhead isn't buffered? What if the implementation
just buffers the bytes from send(), and writes the websocket protocol 
specific bytes to the socket when needed?

And if bufferedAmount includes the overhead, it needs to be specified
what bufferedAmount is during handshake.

-Olli


> but I haven't changed it yet because I don't want
> to be flip-flopping back and forth -- if we change this, I don't want to
> change it back. I think the use cases I know of are addressed whether we
> include overhead or not, so from an objective perspective I don't think it
> really matters, which makes this more of an opinion thing. I'd encourage
> anyone else with an opinion on this to make their opinion known. (Yes, I'm
> actually _asking_ for people to suggest colours for this bikeshed!)
>
> Cheers,