[whatwg] [URL] Starting work on a URL spec

[Brett Zamir]

  On 7/24/2010 2:02 PM, Boris Zbarsky wrote:
> On 7/24/10 1:50 AM, Brett Zamir wrote:
>>> I would be particularly interested in data on this last, across
>>> different browsers, operating systems, and locales... There seem to be
>>> servers out there expecting their URIs in UTF-8 and others expecting
>>> them in ISO-8859-1, and it's not clear to me how to make things work
>>> with them all.
>>
>> Seems to me that if they are not in UTF-8, they should be treated as
>> bugs, even if that is not a de jure standard.
>
> Treated as bugs by whom?
>
By the servers/scripting languages. While it is great that the browsers 
are involved in the process, I think it would be reasonable to invite 
the other stake-holders to join the discussions.
> The scenario is that a user types some non-ASCII text in the url bar. 
> This needs to be url-encoded to actually go on the wire, which raises 
> the question of what encoding.  If the user is using IRIs, the answer 
> is UTF-8.  A number of servers barf if you do this, especially because 
> some server-side scripting languages (PHP, e.g., last I checked) 
> default to URI-unescaping via something other than UTF-8.
>
Hopefully to be fixed in PHP6 with its promise of full Unicode support...

Though per http://www.slideshare.net/kfish/unicode-php6-presentation :

*Slide 34: *Conversions & Encoding “HTTP Input Encoding”
With Unicode semantics switch enabled, we need to convert HTTP input to 
Unicode
GET requests have no encoding at all and POST ones rarely come marked 
with the encoding
Encoding detection is not reliable
*Correctly decoding HTTP input is somewhat of an unsolved problem*

*Slide 35: *Conversions & Encoding “HTTP Input Encoding”
PHP will perform lazy decoding
Delays decoding data in $_GET, $_POST, and $_REQUEST until the first time 
you access them
Allows user to set expected encoding or just rely on a default one
Allows decoding errors to be handled by the same mechanism
Applications should also use filter extension to filter incoming data

> So some browser encode the non-query part of the URI as UTF-8 and the 
> query part as ... something (user's default filesystem encoding, say, 
> for lack of a better guess).  Others always use UTF-8 (and end up with 
> some servers not usable).  Others... I have no idea.  That's why I 
> want data.  ;)  In particular, while the "just use UTF-8, and if the 
> user can't access the site sucks to be the user" approach has a 
> certain theoretical-purity appeal, it doesn't seem like something I 
> want to do to my friends and family (always a good criterion for 
> things you'd like to do to users).
>
What I meant is to try to get the server systems on board to fix the 
issue, including in the long-term. I appreciate you all being admirably 
practical champions of present-day compatibility, though I'd hope there 
is a vision to make things work better for the future, even if there 
will be some inevitable growing pains for a subset of users (as the lack 
of standardization no doubt creates pains for another subset as it is).

Brett

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20100724/87d807a5/attachment-0001.htm>