[whatwg] Guessing the fallback encoding from the top-level domain name before trying to guess from the browser localization

[Henri Sivonen]

On Sat, Feb 8, 2014 at 12:37 AM, Ian Hickson <ian at hixie.ch> wrote:
> The correlation should be at least as high, as far as I can tell.

Logically, yes, for most parts of the world.

> Or maybe a 50%/50% experiment
> with that as the first 50% and the default coming from the TLD instead of
> the UI locale in the second 50%, with the corresponding instrumentation,
> to see how the results compare.

Mozilla doesn't have a proper A/B testing infrastructure yet. I expect
the A to be Firefox 29 on the release channel and B to be Firefox 30
on the release channel. So unless this gets backed out, I expect to
have data around the time of Firefox 31 going to release.

> Have you tried deploying this?

It is on Firefox trunk now. However,  not all country TLDs  are
participating. I figured it is better to leave unsure cases the way
they were. It doesn't make sense to put a lot of effort into
researching those before seeing if the general approach works for the
case that it was designed for, specifically Traditional Chinese. The
success metric I expect to be looking at is if the usage of the
character encoding menu in the Traditional Chinese localization of
Firefox falls to the same level as in other Firefox localizations in
general.

If this change turns out to be successful for Traditional Chinese,
then I think  it will be worthwhile to research the unobvious cases.

The TDLs listed in
https://mxr.mozilla.org/mozilla-central/source/dom/encoding/nonparticipatingdomains.properties
do not participate at present (i.e. get a browser UI
localization-based guess like before). The TLDs listed in
https://mxr.mozilla.org/mozilla-central/source/dom/encoding/domainsfallbacks.properties
get the fallbacks listed in that file. All other TLDs map to
windows-1252.

> What have you learnt so far?

It hasn't been an obvious and immediate disaster.

-- 
Henri Sivonen
hsivonen at hsivonen.fi
https://hsivonen.fi/