[whatwg] API for encoding/decoding ArrayBuffers into text

[Anne van Kesteren]

On Thu, 22 Mar 2012 14:47:05 +0100, Glenn Maynard <glenn at zewt.org> wrote:
> A problem with this is that the bugs resulting from not calling eof() are
> subtle.  The only thing eof() would ever do, I think, is return U+FFFD
> characters if there are leftover characters in the internal buffer; if  
> you
> never call eof(), you'll never get incorrect results unless you test with
> invalid inputs.
>
> It's minor, as subtle-edge-cases-that-people-won't-get-right go, but it's
> at least worth a mention.  Maybe people who would use this API instead of
> the simpler non-streaming version (which could be a thin wrapper on this)
> in the first place are also more likely to get this right.
>
> I'm guessing a common, incorrect pattern would be:
>
> string = new Encoder("euc-kr").encode(bytes);
>
> which would *not* be equivalent to bytes.encode("euc-kr").

Another way would be to have a second optional argument that indicates  
whether more bytes are coming (defaults to false), but I'm not sure of the  
chances that would be used correctly. The reasons you outline are probably  
why many browser implementations deal with EOF poorly too.


-- 
Anne van Kesteren
http://annevankesteren.nl/