Bug 176650 - KDE doesn’t respect country chosen in user’s locale
Summary: KDE doesn’t respect country chosen in user’s locale
Status: REOPENED
Alias: None
Product: i18n
Classification: Translations
Component: general (show other bugs)
Version: unspecified
Platform: Compiled Sources Unspecified
: NOR normal
Target Milestone: ---
Assignee: John Layt
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-12-01 21:37 UTC by Karl Ove Hufthammer
Modified: 2024-07-09 19:09 UTC (History)
14 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
screenshot (83.46 KB, image/png)
2015-05-19 10:13 UTC, Olli
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Karl Ove Hufthammer 2008-12-01 21:37:04 UTC
Version:            (using Devel)
Installed from:    Compiled sources

KDE doesn’t respect the country chosen in the user’s locale. It seems to use the language (from the LANG variable?) for translations, but not the country, so dates, currency etc. are incorrectly displayed by default.

This was originally reported as a bug in Mandriva Linux
https://qa.mandriva.com/show_bug.cgi?id=46085
where one developer said:
‘KDE should really honor environment variables regarding l10n...’
I tend to agree.

Here’s the text from the Mandriva bug report, which should explain the problem in greater detail:

Description of problem:
When creating a new profile, the country (and language) is not automatically set in the 'Country/region and language' settings in KDE. This means that the country-specific locale settings (such as dates and currency) are not used in KDE applications (these are taken from a country settings file in KDE and not from the system locale), so these items are wrongly formatted.

Even though the rest of the KDE interface may be shown in the correct language (when no language is set in this dialogue, KDE by default use the LANG variable), numbers, dates, currency etc. are not shown in the correct language.

Note that KDE use advanced handling of formatting of such items, which depends on *both* the (top) language and the country chosen in this dialogue. So if you for example set the country to Norway, the way the (e.g.) dates and currency is formatted depends on the chosen language. It will be different for Norwegian Nynorsk/Bokmål, Northern Sami or Esperanto, for instance.

How reproducible:
Always (though only tested when Norwegian Nynorsk and Norway is chosen as the system language). I have also tested this with the KDE 4.1.3 packages from the 'testing' repository.


Steps to Reproduce:
1. Create a new user (using userdrake).
2. Log in in KDE as the new user.
3. Start 'systemsettings'
4. Go to 'Country/region and Language'
5. Note that 'Country or region' is set to 'Not set (Generic English)' and the language list is empty.

Running localedrake for the current user also doesn't help.
Comment 1 Anne-Marie Mahfouf 2008-12-05 16:12:39 UTC
Karl: what env var should be picked to set the country please? 

I also do not understand why the list is not populated by languages and what the Add Language combobox is for.

This module needs some work on design and needs fixing.
Comment 2 Karl Ove Hufthammer 2008-12-05 16:40:36 UTC
I’m not 100% certain, but I belive it is 1) LC_ALL if it exists, and 2) LANG if it does not. So if you have

LC_ALL=nn_NO.UTF-8
LANG=da_DK.UTF-8

the country should be NO (Norway). If you only have

LANG=da_DK.UTF-8

or

LANG=da_DK

the country should be DK (Denmark).

If the variables are set to ‘C’ or ‘POSIX’ the generic (C) ‘country’ should be used.
Comment 3 Karl Ove Hufthammer 2008-12-05 16:45:28 UTC
About the language list: It is a list of languages in preferred order. A translation is taken from the first language in the list that has a translation of the string.

The explanation you get when you hover over the list is thus misleading.
Comment 4 Karl Ove Hufthammer 2009-04-29 11:00:59 UTC
According to this blog entry
http://jtechinda.blogspot.com/2009/04/on-brighter-side-of-things-kubuntu-904.html
this bug is fixed in Kubuntu.

But from what I gather from the blog, the Kubuntu developers did not contribute the fix upstream, so this bug is still present in KDE.
Comment 5 Nicolas L. 2009-04-29 12:23:37 UTC
i looked and think that the ubuntu patch that have to be reviewed for inclusion is this one : 

http://patches.ubuntu.com/k/kdebase-workspace/extracted/kubuntu_13_startkde_set_country.diff

i just added it  on mandriva to see if it works ok for us too
Comment 6 Karl Ove Hufthammer 2009-04-29 12:40:58 UTC
It doesn’t look like the patch take the user-specific LC_MESSAGES or LC_ALL into consideration, only LANG and the system defaults. LC_ALL overrides LC_MESSAGES, and both override LANG. The correct way to find the correct country is thus to look at both

LC_ALL
LC_MESSAGES
LANG
system defaults

and use the *first one* of these that contain the country.
Comment 7 Chusslove Illich 2009-05-02 23:47:10 UTC
I'd just like to clarify why this bug isn't fixed in KDE, upstream.

For language, detection based on environment can work cleanly: in KDE
settings, the default language is "System Default", in which case
environment is examined; and user can at any time select specific language,
or return to system default.

So, pretty much as soon Karl Ove opened the bug, I made a patch to do
similar autodetection for locale too. However, possible issues started to
crop up.

First, there is no obvious link between system (as in libc) locale name and
KDE's locale name. Sure, with common locale strings (contains country code,
not an alias) and common countries (de, fr...) just taking country from the
environment variables will work; but in general not.

Then, even when autodetection manages to hit proper KDE locale name, once a
hapless user selects a locale in KDE settings, there is no way to go back to
autodetection of system locale (unlike with language). And this is not a
simple issue of locale control module interface, because...

If the user tweaks some locale settings, then if autodetection is in effect,
the manually tweaked settings would not change as system locale changes,
while others would. And without any indicator to the user whatsoever which
one has and which one has not been manually tweaked. This is both hard to
fix under the hood, and to design a proper interface to present it.

The central problem is that KDE and "system" (i.e. libc) locale systems are
completely separate, with different interfaces and sets of features, and it
is nowhere near easy to design and implement a sane bridge between them.

(Note that above patch in Kubuntu will work only once; if the user later
changes system locale, this will not reflect upon KDE, even if in the
meantime the user didn't even touch KDE settings. In fact, given the above
ugly effects of real autodetection, this kind of one-shot autodetection may
be for the best. But since it's just replacing one bug with another, albeit
perhaps more amiable, I would let it remain upon distributions whether to do
it. One extra suggestion would be to, instead of manually rumaging through
kdeglobals, use:

  kwriteconfig --group Locale --key Country $country

where $country is KDE locale name guessed from environment.)
Comment 8 Karl Ove Hufthammer 2009-05-17 19:47:30 UTC
> First, there is no obvious link between system (as in libc) locale name and
> KDE's locale name. Sure, with common locale strings (contains country code,
> not an alias) and common countries (de, fr...) just taking country from the
> environment variables will work; but in general not.

But won’t we be 99% there by doing this? This will be infinitely better than the current situation, where we are 0% there, and always falls back to POSIX defaults, which is only suitable for US English.

> Then, even when autodetection manages to hit proper KDE locale name, once a
> hapless user selects a locale in KDE settings, there is no way to go back to
> autodetection of system locale (unlike with language).

Why not just have a ‘(System default)’ option for both language and country?

> If the user tweaks some locale settings, then if autodetection is in effect,
> the manually tweaked settings would not change as system locale changes,
> while others would.

One solution would be to always let the autodetection override the changes. You could for example grey out all options when the ‘(System default)’ Country and Language is selected.

> (Note that above patch in Kubuntu will work only once; if the user later
> changes system locale, this will not reflect upon KDE, even if in the
> meantime the user didn't even touch KDE settings.

Yes, this is very unfortunate.
Comment 9 Martin Schlander 2009-05-18 11:15:47 UTC
Like Karl Ove I don't see how anything could be worse than, or even remotely as bad as, the current state.

This bug causes some nasty issues, like for example whenever any Danish user will reply to an e-mail with KMail, the date format will be completely wrong - and anyone receiving such an e-mail will see that Linux/KDE sucks.

Unless the user thinks to go to systemsettings -> Regional and language and set the locale manually... which ain't gonna happen 99.5% of the time.
Comment 10 Karl Ove Hufthammer 2009-07-09 12:32:51 UTC
Would it be possible to have a special string in translation files, so that languages can specify a *default* country? I.e., setting the language (perhaps automatically via the LANG variable) will implicitly select a country. For instance, setting the language to Danish would select Denmark, and setting the language to British English would select the UK?

This won’t be a perfect solution, as some languages are spoken in more than one country, but will for the vast majority of languages be a great improvement (and the country settings for a country that share the same language will probably be a much better match than the settings for the ‘C‘ country is).
Comment 11 John Layt 2010-08-07 20:20:43 UTC
Assign to me as I'm currently trying to sort a solution for this in KLocale :-)  

Step 1 is at http://reviewboard.kde.org/r/4915/ which is code to guess a host system country on Win/Mac/Linux.  We may require better heuristics, but it's a framework to do so.

Step 2 is improving the kcm to better show which settings are defaults, which are a users choices, and to toggle between them.  That's a WIP.

See also threads on the k-c-d mailing list.
Comment 12 John Layt 2010-08-23 00:48:10 UTC
SVN commit 1166797 by jlayt:

Try to choose a default locale Country if the user hasn't set one yet.

See http://reviewboard.kde.org/r/4915/

Under Linux this will use QLocale to try obtain a default country.

Under Windows and Mac this will fake locale support by loading the KDE settings
for the system country, proper support using real host settings will follow
later.

Fix my stupidity with not understanding how virtual call work in constructors.

Add validation checking to country code values, and improve documentation.

Compiled and tested under Linux and Windows, I am unable to build a Mac test
environment atm, please patch or beat me up if it fails under Mac.

CCMAIL: kde-windows@kde.org
CCMAIL: kde-mac@kde.org
CCBUG: 176650



 M  +7 -3      klocale.cpp  
 M  +14 -2     klocale.h  
 M  +84 -26    klocale_kde.cpp   [POSSIBLY UNSAFE: system]
 M  +15 -0     klocale_mac.cpp  
 M  +13 -0     klocale_mac_p.h  
 M  +31 -1     klocale_p.h   [POSSIBLY UNSAFE: system]
 M  +23 -2     klocale_win.cpp  
 M  +12 -0     klocale_win_p.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1166797
Comment 13 John Layt 2010-10-23 14:57:28 UTC
Just to explain what the new code does from KDE 4.5.2 onwards.

If the user has never set a country in the Locale KCM, then:
 * KDE will make a guess every time you log in.
 * On Windows and Mac this will be the system country.
 * On Linux this will use QLocale to make the guess, which involves lots of magic around env vars and CLDR.
 * If the user changes the system locale, at next start-up KDE will pick up the change and possibly guess a different country.
 * If the user goes into the KDE Locale control module and overrides any settings other than the country, then these settings will persist even if the user changes the system locale.
 * If KDE guesses the wrong country the user can override this in the Locale control module and this setting will persist.
 * In 4.6, a new control module will show the user that KDE is using the system country, and provide easy ways to identify where they have overridden the default settings and reset individual settings to the default. 

If the user has already explicitly set their country in the control module then:
 * This setting will always be applied and the system locale ignored.
 * In 4.6 the control module will allow the user to change to use the system derived country instead.

If anyone spots weird behaviour please let me know.  In particular, I can add a whitelist of system locales that Qt returns the wrong result for a particular system locale.

You can test this by going into your .kde4/share/config/kdeglobals file and remove from the [Locale] section any line with "Country=xx"
Comment 14 Gökçen Eraslan 2010-10-23 15:10:48 UTC
(In reply to comment #13)
> Just to explain what the new code does from KDE 4.5.2 onwards.

So will you backport this change to 4.5? I couldn't see in KDE websvn of 4.5 branch. 

Anyway it would be awesome.
Comment 15 John Layt 2010-10-23 16:54:39 UTC
Whoops!  I did backport it, but forgot to commit.  Sorry, that will now be in 4.5.3.
Comment 16 Gökçen Eraslan 2010-10-24 19:54:07 UTC
(In reply to comment #15)
> Whoops!  I did backport it, but forgot to commit.  Sorry, that will now be in
> 4.5.3.

Ignore that if you've committed but I still couldn't find the commit in 4.5 branch.
Comment 17 John Layt 2010-11-02 00:54:11 UTC
SVN commit 1192086 by jlayt:

Backport to 4.5 KLocale fix for guessing the users country if not manually set.

Unlike the 4.6 version, the 4.5 version uses QLocale for all platforms.

Qt uses some heuristics to make good guesses, but may still get it wrong, but
anything has to be better than defaulting to C.

CCBUG: 176650



 M  +25 -2     klocale.cpp   [POSSIBLY UNSAFE: system]


WebSVN link: http://websvn.kde.org/?view=rev&revision=1192086
Comment 18 michaelaquilina@gmail.com 2013-07-18 07:23:14 UTC
I can confirm this problem is present in Kubuntu 13.04 as well. if i set country to say American English in the language settings control panel, the 'locale' command will still items such as LC_TIME and LC_NUMERIC to my 'system detected language' Maltese, i.e. mt_MT.UTF-8. 

This means that all my desktop is shown in English, but other things like date and time are shown in Maltese (which is very inconsistent and annoying). 

The only way i've found to solve this is to edit the 'setlocale.sh' file that is run at startup. If you add 'export LC_ALL=en_US.UTF-8' to the end of the file then everything will be shown in english.
Comment 19 Graeme Hewson 2013-08-06 10:00:44 UTC
(In reply to comment #18)
> I can confirm this problem is present in Kubuntu 13.04 as well.

> The only way i've found to solve this is to edit the 'setlocale.sh' file
> that is run at startup.

setlocale.sh is specific to Kubuntu, and isn't in the general KDE distribution.

See https://bugs.kde.org/show_bug.cgi?id=322688 and https://bugs.launchpad.net/ubuntu/+source/kde-workspace/+bug/1204182
Comment 20 Olli 2015-05-19 10:09:05 UTC
This is still valid for 5.3
linux-0gts:~> locale
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=
linux-0gts:~> env | grep LANG
LANG=en_US.utf8
LANGUAGE=en_US

The date is still represented in Finnish
Comment 21 Olli 2015-05-19 10:13:37 UTC
Created attachment 92703 [details]
screenshot
Comment 22 Pekka Helenius 2017-03-26 12:36:42 UTC
KDE/Plasma 5. Framework 5.32, KDE 5.9.4.

Is there a specific reason the system gives me odd locales like "en_FI" or similar which are not recognized by the rest of my system programs and shouldn't exist at all. KDE understands these strange locales fine but by using them cause easily problems with programs such as perl.

Basically, I'm trying to use locale "fi_FI" for my time/date but KDE locale settings doesn't give that choice at all. Why?

I'm currently forced to use "en_US" locale setting which is not I'm looking for since I'm not used to read calendar like US people.

For the rest of locale settings, I use "en_US".
Comment 23 Bug Janitor Service 2020-06-20 08:22:09 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/plasma-workspace/-/merge_requests/100
Comment 24 Bug Janitor Service 2020-06-20 08:22:11 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/plasma-workspace/-/merge_requests/100
Comment 25 Alexander Lohnau 2020-06-25 13:19:36 UTC
Git commit edc64d04a1e569d7032c41e6ee0ebf59833c26f2 by Alexander Lohnau.
Committed on 25/06/2020 at 13:18.
Pushed by alex into branch 'Plasma/5.19'.

Fix broken ENV variables for detailed settings
Related: bug 417070, bug 327757

M  +21   -10   startkde/startplasma.cpp

https://invent.kde.org/plasma/plasma-workspace/commit/edc64d04a1e569d7032c41e6ee0ebf59833c26f2
Comment 26 Alexander Lohnau 2020-06-25 13:19:36 UTC
Git commit edc64d04a1e569d7032c41e6ee0ebf59833c26f2 by Alexander Lohnau.
Committed on 25/06/2020 at 13:18.
Pushed by alex into branch 'Plasma/5.19'.

Fix broken ENV variables for detailed settings
Related: bug 417070, bug 327757

M  +21   -10   startkde/startplasma.cpp

https://invent.kde.org/plasma/plasma-workspace/commit/edc64d04a1e569d7032c41e6ee0ebf59833c26f2
Comment 27 Nate Graham 2020-07-15 19:33:52 UTC
We had to revert the commit that fixed this as it caused Bug 423995. We'll work on finding another way.
Comment 28 soredake 2021-01-05 16:01:07 UTC
any progress on this?