Bug 403580

Summary: KCM shows invalid locales and allows them to be chosen, which can break various parts of the system
Product: [Applications] systemsettings Reporter: Andreas Schneider <asn>
Component: kcm_formatsAssignee: Sebastian Kügler <sebas>
Status: RESOLVED FIXED    
Severity: major CC: alexander.lohnau, asn, autipial, david.cortes.rivera, edarocha1324, eragon92, gaetan.gilbert, hanyoung, karl, katyaberezyaka, koshka, liubomirwm, medhefgo, mfabian, nate, postix, prettyvanilla, rantanplan-web, rdieter, rick, simonisfrederic, sitter, sparr0, spleefer90, till2.schaefer, xand.net, yule2000
Priority: HI    
Version: 5.19.3   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In: 5.26
Sentry Crash Report:

Description Andreas Schneider 2019-01-25 09:16:15 UTC
SUMMARY

My system is set to en_US.UTF-8. However I would like to use change Region, Numbers, Time, Currency etc. to a different locale. In my case it is German. So if I search the list for Germany, it only offers me en_DE for Germany and that locale doesn't exist. If you look through the list not only the locale for German is completly bogus also for others.

Compare it using 'locale -a' on a console which gives you a list of supported locales.

Other examples: ksh_DE doesn't exist. So I don't know what this is even coming from. However if I select en_DE then LC_* is set to it and all tools run in a terminal will give you a lot of warnings that invalid locales are used.

I'm on Fedora 29, plasma is 5.14.4
Comment 1 Andreas Schneider 2019-01-25 09:20:40 UTC
It looks like that Qt uses cldr, however glibc doesn't support all locales of cldr. Also it is strange that it doesn't offer de_DE which is the only supported locale by glibc.
Comment 2 Andreas Schneider 2019-01-25 09:23:01 UTC
The issue probably is that KDE sets LANG or LC_* variables to locales which don't exist. So all terminal tools like 'ls', 'git' etc. are printing warnings that I use locales which don't exist.
Comment 3 Mike FABIAN 2019-02-13 15:58:07 UTC
(In reply to Andreas Schneider from comment #1)
> It looks like that Qt uses cldr, however glibc doesn't support all locales
> of cldr. Also it is strange that it doesn't offer de_DE which is the only
> supported locale by glibc.

Yes, de_DE is also a valid locale for CLDR/libicu.  The list KDE
offers is very incomplete.
Comment 4 Mike FABIAN 2019-02-13 16:00:58 UTC
(In reply to Andreas Schneider from comment #2)
> The issue probably is that KDE sets LANG or LC_* variables to locales which
> don't exist. So all terminal tools like 'ls', 'git' etc. are printing
> warnings that I use locales which don't exist.

KDE/Qt5 apparently uses the locale support in libicu. The list of locales
libicu supports is different from the locales glibc supports, there is considerable overlap between the list of locales supported by libicu and glibc, but the lists are not equal.

If the user chooses a locale in KDE which does not exist in glibc, it makes not sense setting the LANG or LC_* variables to such a locale.
Comment 5 Till Schäfer 2020-07-08 14:16:51 UTC
In combination with Bug 423995 this really becomes a showstopper, since systemwide locale settings are ignored, and the kcm settings are always faulty. 


This problem is reproducible on multiple machines here and another friend of mine contacted me recently because of this localization problem. Thus, it seems to be a quite common issue.
Comment 6 Till Schäfer 2020-07-08 14:20:04 UTC
*** Bug 404151 has been marked as a duplicate of this bug. ***
Comment 7 Till Schäfer 2020-07-08 14:20:17 UTC
*** Bug 352030 has been marked as a duplicate of this bug. ***
Comment 8 Alexander Lohnau 2020-07-09 07:47:33 UTC
The offers for the KCM are generated using:
>QLocale::matchingLocales(QLocale::AnyLanguage, QLocale::AnyScript, QLocale::AnyCountry);
(plasma-desktop/kcms/formats/kcmformats.cpp 75)

So the installed languages aren't considered, just all possibilities are displayed.
Comment 9 Alexander Lohnau 2020-07-09 18:34:43 UTC
But the title of this bug report is wrong, because there are not only invalid locales shown.

@nate Could we maybe fetch the available/installed languages by running the "locale" process (Just a thought, or is there another way to do this. The translations model uses KLocalizedString::availableDomainTranslations, but this seems like a different usecase). But then we should make it easy and intuitive for the user to install additional language packs.

Also I remember having some kind of notifications of incomplete language support when I first tried out KDE. See https://unix.stackexchange.com/questions/421066/popup-language-support-is-incomplete-what-packages-does-it-want-to-install. Maybe this idea can be of use in this case.
Comment 10 Harald Sitter 2021-06-25 12:27:26 UTC
Some musings: https://bugs.kde.org/show_bug.cgi?id=431292#c12
Comment 11 Nate Graham 2021-09-30 17:51:11 UTC
*** Bug 441010 has been marked as a duplicate of this bug. ***
Comment 12 Nate Graham 2021-09-30 17:51:15 UTC
*** Bug 441468 has been marked as a duplicate of this bug. ***
Comment 13 Nate Graham 2021-09-30 17:52:22 UTC
*** Bug 384451 has been marked as a duplicate of this bug. ***
Comment 14 hanyoung 2021-10-02 12:45:18 UTC
The problem is Qt uses icu but GNU utilities use glibc. The current behavior is correct for icu, because you don't need to "generate" any locales in icu, they just work. However with glibc locale, we need to generate them. But then again, valid icu locale may not be valid glibc locale, vice versa.
Comment 15 Nate Graham 2021-10-05 17:13:52 UTC
*** Bug 416103 has been marked as a duplicate of this bug. ***
Comment 16 Nate Graham 2021-12-08 21:02:44 UTC
*** Bug 446684 has been marked as a duplicate of this bug. ***
Comment 17 Nate Graham 2022-01-12 00:35:05 UTC
*** Bug 447626 has been marked as a duplicate of this bug. ***
Comment 18 Nate Graham 2022-01-18 23:33:09 UTC
This is in progress with https://invent.kde.org/plasma/plasma-workspace/-/merge_requests/1147.
Comment 19 Nate Graham 2022-01-25 21:18:04 UTC
*** Bug 449128 has been marked as a duplicate of this bug. ***
Comment 20 Nate Graham 2022-03-26 03:58:10 UTC
*** Bug 451685 has been marked as a duplicate of this bug. ***
Comment 21 Nate Graham 2022-03-28 22:30:58 UTC
*** Bug 451920 has been marked as a duplicate of this bug. ***
Comment 22 hanyoung 2022-07-02 09:31:07 UTC
Git commit 134e2d5c989c36ac0e985ee0ae382996c6b7b56e by Han Young.
Committed on 02/07/2022 at 09:29.
Pushed by hanyoung into branch 'master'.

merge Language and Formats
Related: bug 192019, bug 341235, bug 344588, bug 394477, bug 397974, bug 397975, bug 417564, bug 420268, bug 429474, bug 431292, bug 444772, bug 446785, bug 447787, bug 448324, bug 448355, bug 451919, bug 451944, bug 454991

closes https://invent.kde.org/plasma/plasma-workspace/-/issues/23

M  +1    -0    .kde-ci.yml
M  +52   -8    CMakeLists.txt
M  +2    -1    config-workspace.h.cmake
D  +0    -2    doc/kcontrol/formats/CMakeLists.txt
D  +0    -63   doc/kcontrol/formats/index.docbook
R  +1    -1    doc/kcontrol/region_language/CMakeLists.txt [from: doc/kcontrol/translations/CMakeLists.txt - 063% similarity]
R  +-    --    doc/kcontrol/region_language/go-top.png [from: doc/kcontrol/translations/go-top.png - 100% similarity]
R  +36   -23   doc/kcontrol/region_language/index.docbook [from: doc/kcontrol/translations/index.docbook - 050% similarity]
A  +-    --    doc/kcontrol/region_language/list-remove.png
D  +-    --    doc/kcontrol/translations/list-remove.png
M  +1    -2    kcms/CMakeLists.txt
D  +0    -30   kcms/formats/CMakeLists.txt
D  +0    -5    kcms/formats/Messages.sh
D  +0    -61   kcms/formats/formatssettings.kcfg
D  +0    -80   kcms/formats/kcmformats.cpp
D  +0    -34   kcms/formats/kcmformats.h
D  +0    -183  kcms/formats/localelistmodel.cpp
D  +0    -140  kcms/formats/optionsmodel.cpp
D  +0    -126  kcms/formats/package/contents/ui/main.qml
A  +75   -0    kcms/region_language/CMakeLists.txt
A  +8    -0    kcms/region_language/Messages.sh
R  +2    -27   kcms/region_language/exampleutility.cpp [from: kcms/formats/exampleutility.cpp - 052% similarity]
A  +22   -0    kcms/region_language/exampleutility.h     [License: GPL(v2.0+)]
A  +82   -0    kcms/region_language/kcm_regionandlang.desktop [TRAILING SPACE] ** [TRAILING SPACE] **
R  +4    -4    kcms/region_language/kcm_regionandlang.json [from: kcms/formats/kcm_formats.json - 098% similarity]
A  +250  -0    kcms/region_language/kcmregionandlang.cpp     [License: GPL(v2.0+)]
A  +58   -0    kcms/region_language/kcmregionandlang.h     [License: GPL(v2.0+)]
A  +372  -0    kcms/region_language/languagelistmodel.cpp     [License: GPL(v2.0+)]
A  +100  -0    kcms/region_language/languagelistmodel.h     [License: GPL(v2.0+)]
A  +30   -0    kcms/region_language/localegenerator.cpp     [License: LGPL(v2.0+)]
A  +17   -0    kcms/region_language/localegenerator.h     [License: LGPL(v2.0+)]
A  +19   -0    kcms/region_language/localegeneratorbase.cpp     [License: GPL(v2.0+)]
A  +23   -0    kcms/region_language/localegeneratorbase.h     [License: GPL(v2.0+)]
A  +32   -0    kcms/region_language/localegeneratorglibc.cpp     [License: GPL(v2.0+)]
A  +24   -0    kcms/region_language/localegeneratorglibc.h     [License: GPL(v2.0+)]
A  +101  -0    kcms/region_language/localegeneratorubuntu.cpp     [License: GPL(v2.0+)]
A  +27   -0    kcms/region_language/localegeneratorubuntu.h     [License: GPL(v2.0+)]
A  +36   -0    kcms/region_language/localegenhelper/CMakeLists.txt
A  +187  -0    kcms/region_language/localegenhelper/localegenhelper.cpp     [License: GPL(v2.0+)]
A  +46   -0    kcms/region_language/localegenhelper/localegenhelper.h     [License: GPL(v2.0+)]
A  +20   -0    kcms/region_language/localegenhelper/org.kde.localegenhelper.conf
A  +21   -0    kcms/region_language/localegenhelper/org.kde.localegenhelper.policy
A  +8    -0    kcms/region_language/localegenhelper/org.kde.localegenhelper.service.in
A  +158  -0    kcms/region_language/localelistmodel.cpp     [License: GPL (v2+)]
R  +22   -22   kcms/region_language/localelistmodel.h [from: kcms/formats/localelistmodel.h - 060% similarity]
A  +197  -0    kcms/region_language/optionsmodel.cpp     [License: GPL(v2.0+)]
R  +21   -11   kcms/region_language/optionsmodel.h [from: kcms/formats/optionsmodel.h - 054% similarity]
A  +204  -0    kcms/region_language/package/contents/ui/AdvancedLanguageSelectPage.qml     [License: LGPL(v3.0+)]
A  +238  -0    kcms/region_language/package/contents/ui/main.qml     [License: LGPL(v3.0+)]
A  +64   -0    kcms/region_language/regionandlangsettings.cpp     [License: GPL(v2.0+)]
A  +21   -0    kcms/region_language/regionandlangsettings.h     [License: GPL(v2.0+)]
A  +37   -0    kcms/region_language/regionandlangsettingsbase.kcfg
R  +2    -2    kcms/region_language/regionandlangsettingsbase.kcfgc [from: kcms/formats/formatssettings.kcfgc - 055% similarity]
A  +17   -0    kcms/region_language/settingtype.h     [License: GPL(v2.0+)]
D  +0    -49   kcms/translations/CMakeLists.txt
D  +0    -2    kcms/translations/Messages.sh
D  +0    -191  kcms/translations/language.cpp
D  +0    -32   kcms/translations/language.h
D  +0    -312  kcms/translations/package/contents/ui/main.qml
D  +0    -86   kcms/translations/translations.cpp
D  +0    -53   kcms/translations/translations.h
D  +0    -193  kcms/translations/translationsmodel.cpp
D  +0    -72   kcms/translations/translationsmodel.h
D  +0    -27   kcms/translations/translationssettings.cpp
D  +0    -24   kcms/translations/translationssettings.h
D  +0    -28   kcms/translations/translationssettingsbase.kcfg
D  +0    -6    kcms/translations/translationssettingsbase.kcfgc

The files marked with ** at the end have a problem. Either the file contains a trailing space or the file contains a call to potentially dangerous code. Please read: https://community.kde.org/Sysadmin/CommitHooks#Email_notifications for further information. Please either fix the trailing space or review the dangerous code.


https://invent.kde.org/plasma/plasma-workspace/commit/134e2d5c989c36ac0e985ee0ae382996c6b7b56e
Comment 23 Lyubomir 2023-03-07 12:53:40 UTC
Shouldn't the status of this bug be "FIxed for 5.26"? Judging by https://invent.kde.org/plasma/plasma-workspace/-/merge_requests/1147