Bug 454991

Summary: Language KCM sets wrong language variable for European Portuguese (pt_PT)
Product: [Applications] systemsettings Reporter: Eduardo Correia <eduardosareias>
Component: kcm_regionandlangAssignee: Eike Hein <hein>
Status: RESOLVED FIXED    
Severity: normal CC: hanyoung, luigi.toscano, nate, plasma-bugs
Priority: HI Keywords: accessibility, usability
Version: 5.26.4   
Target Milestone: ---   
Platform: Kubuntu   
OS: Linux   
Latest Commit: Version Fixed In: 5.27

Description Eduardo Correia 2022-06-07 18:53:37 UTC
SUMMARY
When we add "European Portuguese" language to KDE language settings, it correctly adds it but sets it wrong in the LANGUAGE varibles. After running "locale" in the terminal, the "LANGUAGE" variable is set to "pt" instead of the correct "pt_PT". Having just "pt" makes all apps that follow the system language to ignore that setting and fall back to the next configured language, or Brazilian Portuguese (it's VERY different from European Portuguese in many places and words, so please don't mix these two). This is because some apps read "pt" as being equal to "pt_BR". But all apps use European Portuguese correctly when "pt_PT" is used instead.


STEPS TO REPRODUCE
1. Open System Settings, go to Language, add "European Portuguese" as the main language, apply.
2. Apps that follow the system's language, like any Chromium-based browser, are displayed in Brazilian Portuguese instead.
3. As an example, do "sudo nano ~/.config/plasma-localerc" and you can confirm LANGUAGE under "Translations" is set to "pt" instead of "pt_PT".
4. Change this file manually to "pt_PT", save it, reload plasma (logout and log back in)
5. All apps are correctly shown in European Portuguese.

6. Bonus, small related bug: After setting up "pt_PT" manually in this file, Language KCM complaints about having an invalid language configured and completely replaces our custom configuration if we click "Apply". It rewrites that config file with "pt" again.

OBSERVED RESULT
Apps are shown in Brazilian Portuguese (pt OR pt_BR)

EXPECTED RESULT
Apps should be shown in European Portuguese (pt_PT)

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Kubuntu 22.04
(available in About System)
KDE Plasma Version: 5.24.5
KDE Frameworks Version: 5.94.0
Qt Version: 5.15.3

ADDITIONAL INFORMATION
This bug was present for many years now, but I never really understood why (never cared too much about it), until now. Happens on any distro, but only with KDE.
Comment 1 Nate Graham 2022-06-08 19:24:24 UTC
This might be fixed with https://invent.kde.org/plasma/plasma-workspace/-/merge_requests/1147.
Comment 2 hanyoung 2022-07-02 09:32:03 UTC
Git commit 134e2d5c989c36ac0e985ee0ae382996c6b7b56e by Han Young.
Committed on 02/07/2022 at 09:29.
Pushed by hanyoung into branch 'master'.

merge Language and Formats
Related: bug 192019, bug 341235, bug 344588, bug 394477, bug 397974, bug 397975, bug 403580, bug 417564, bug 420268, bug 429474, bug 431292, bug 444772, bug 446785, bug 447787, bug 448324, bug 448355, bug 451919, bug 451944

closes https://invent.kde.org/plasma/plasma-workspace/-/issues/23

M  +1    -0    .kde-ci.yml
M  +52   -8    CMakeLists.txt
M  +2    -1    config-workspace.h.cmake
D  +0    -2    doc/kcontrol/formats/CMakeLists.txt
D  +0    -63   doc/kcontrol/formats/index.docbook
R  +1    -1    doc/kcontrol/region_language/CMakeLists.txt [from: doc/kcontrol/translations/CMakeLists.txt - 063% similarity]
R  +-    --    doc/kcontrol/region_language/go-top.png [from: doc/kcontrol/translations/go-top.png - 100% similarity]
R  +36   -23   doc/kcontrol/region_language/index.docbook [from: doc/kcontrol/translations/index.docbook - 050% similarity]
A  +-    --    doc/kcontrol/region_language/list-remove.png
D  +-    --    doc/kcontrol/translations/list-remove.png
M  +1    -2    kcms/CMakeLists.txt
D  +0    -30   kcms/formats/CMakeLists.txt
D  +0    -5    kcms/formats/Messages.sh
D  +0    -61   kcms/formats/formatssettings.kcfg
D  +0    -80   kcms/formats/kcmformats.cpp
D  +0    -34   kcms/formats/kcmformats.h
D  +0    -183  kcms/formats/localelistmodel.cpp
D  +0    -140  kcms/formats/optionsmodel.cpp
D  +0    -126  kcms/formats/package/contents/ui/main.qml
A  +75   -0    kcms/region_language/CMakeLists.txt
A  +8    -0    kcms/region_language/Messages.sh
R  +2    -27   kcms/region_language/exampleutility.cpp [from: kcms/formats/exampleutility.cpp - 052% similarity]
A  +22   -0    kcms/region_language/exampleutility.h     [License: GPL(v2.0+)]
A  +82   -0    kcms/region_language/kcm_regionandlang.desktop [TRAILING SPACE] ** [TRAILING SPACE] **
R  +4    -4    kcms/region_language/kcm_regionandlang.json [from: kcms/formats/kcm_formats.json - 098% similarity]
A  +250  -0    kcms/region_language/kcmregionandlang.cpp     [License: GPL(v2.0+)]
A  +58   -0    kcms/region_language/kcmregionandlang.h     [License: GPL(v2.0+)]
A  +372  -0    kcms/region_language/languagelistmodel.cpp     [License: GPL(v2.0+)]
A  +100  -0    kcms/region_language/languagelistmodel.h     [License: GPL(v2.0+)]
A  +30   -0    kcms/region_language/localegenerator.cpp     [License: LGPL(v2.0+)]
A  +17   -0    kcms/region_language/localegenerator.h     [License: LGPL(v2.0+)]
A  +19   -0    kcms/region_language/localegeneratorbase.cpp     [License: GPL(v2.0+)]
A  +23   -0    kcms/region_language/localegeneratorbase.h     [License: GPL(v2.0+)]
A  +32   -0    kcms/region_language/localegeneratorglibc.cpp     [License: GPL(v2.0+)]
A  +24   -0    kcms/region_language/localegeneratorglibc.h     [License: GPL(v2.0+)]
A  +101  -0    kcms/region_language/localegeneratorubuntu.cpp     [License: GPL(v2.0+)]
A  +27   -0    kcms/region_language/localegeneratorubuntu.h     [License: GPL(v2.0+)]
A  +36   -0    kcms/region_language/localegenhelper/CMakeLists.txt
A  +187  -0    kcms/region_language/localegenhelper/localegenhelper.cpp     [License: GPL(v2.0+)]
A  +46   -0    kcms/region_language/localegenhelper/localegenhelper.h     [License: GPL(v2.0+)]
A  +20   -0    kcms/region_language/localegenhelper/org.kde.localegenhelper.conf
A  +21   -0    kcms/region_language/localegenhelper/org.kde.localegenhelper.policy
A  +8    -0    kcms/region_language/localegenhelper/org.kde.localegenhelper.service.in
A  +158  -0    kcms/region_language/localelistmodel.cpp     [License: GPL (v2+)]
R  +22   -22   kcms/region_language/localelistmodel.h [from: kcms/formats/localelistmodel.h - 060% similarity]
A  +197  -0    kcms/region_language/optionsmodel.cpp     [License: GPL(v2.0+)]
R  +21   -11   kcms/region_language/optionsmodel.h [from: kcms/formats/optionsmodel.h - 054% similarity]
A  +204  -0    kcms/region_language/package/contents/ui/AdvancedLanguageSelectPage.qml     [License: LGPL(v3.0+)]
A  +238  -0    kcms/region_language/package/contents/ui/main.qml     [License: LGPL(v3.0+)]
A  +64   -0    kcms/region_language/regionandlangsettings.cpp     [License: GPL(v2.0+)]
A  +21   -0    kcms/region_language/regionandlangsettings.h     [License: GPL(v2.0+)]
A  +37   -0    kcms/region_language/regionandlangsettingsbase.kcfg
R  +2    -2    kcms/region_language/regionandlangsettingsbase.kcfgc [from: kcms/formats/formatssettings.kcfgc - 055% similarity]
A  +17   -0    kcms/region_language/settingtype.h     [License: GPL(v2.0+)]
D  +0    -49   kcms/translations/CMakeLists.txt
D  +0    -2    kcms/translations/Messages.sh
D  +0    -191  kcms/translations/language.cpp
D  +0    -32   kcms/translations/language.h
D  +0    -312  kcms/translations/package/contents/ui/main.qml
D  +0    -86   kcms/translations/translations.cpp
D  +0    -53   kcms/translations/translations.h
D  +0    -193  kcms/translations/translationsmodel.cpp
D  +0    -72   kcms/translations/translationsmodel.h
D  +0    -27   kcms/translations/translationssettings.cpp
D  +0    -24   kcms/translations/translationssettings.h
D  +0    -28   kcms/translations/translationssettingsbase.kcfg
D  +0    -6    kcms/translations/translationssettingsbase.kcfgc

The files marked with ** at the end have a problem. Either the file contains a trailing space or the file contains a call to potentially dangerous code. Please read: https://community.kde.org/Sysadmin/CommitHooks#Email_notifications for further information. Please either fix the trailing space or review the dangerous code.


https://invent.kde.org/plasma/plasma-workspace/commit/134e2d5c989c36ac0e985ee0ae382996c6b7b56e
Comment 3 Eduardo Correia 2022-12-25 18:59:37 UTC
This bug is still present, if not somewhat worse after the languages KCM rework.
I also suggest many visual improvements while we are at it.
There is a quick list with everything at the end. Lost many hours testing and repeating all this so I hope I am giving as much details as possible for an easier debug.

KDE still insists in using [pt] in all config files when european portuguese is selected. However, 99% of apps read [pt] as being the same as [pt_BR] which is Brazilian Portuguese, a completely different language in many ways, especially grammar.

The new languages KCM rework was great but didn't solve this bug and even made it worse: When we try to switch language or add more, we completely lost the "third" option of "European Portuguese" (called "Português (Portugal)" or "português europeu", can't remember) that we had before 5.26 . The only two options we have are now "português" and "português (brasil)". Choosing "português" makes it add "pt" in plasma-localerc in the [Translations] part. If we then add the other option, "português (brasil)", that config file gets changed to "pt:pt_BR". The "pt_BR" part is correct, but the "pt" is not. And while theoretically using "pt" to refer to European Portuguese is not wrong, almost all third party apps on all OSs and all DEs even many KDE own apps consider "pt" to be the same as "pt_BR". So having "pt" in plasma-localerc makes all apps display their content in Brazilian. This together with this bug makes it so effectively both language options in the language picker are, in the end, actually both brazilian.

If we manually edit plasma-localerc [Translations] from "pt" to "pt_PT" and re-login, the languages KCM will show "português europeu" (European Portuguese) in the title of the currently added languages, even though that option doesn't exist when we try to add it via the languages picker. Changing it manually to pt_PT also makes all apps to be correctly shown in European Portuguese including all third party ones, be it native, flatpaks, snaps, everything. If we try to change it in the KCM and choose the "português" option, it reverts back to "pt" in plasma-localerc.

My suggestion would be to completely remove the "português" option from the add/edit languages KCM and instead add the "Português (Portugal)" option and make it link to "pt_PT", so we get the only two options that actually matter: "Português (Portugal)" [pt_PT] and "Português (Brasil)" [pt_BR]. Having the other "português" [pt] doesn't make much sense since all apps consider it as pt_BR anyway and will only create confusion for the final user in the UI.

Also related, all of these languages should be correctly written to look more professional, and correctly capitalized. "português (brasil)" should be "Português (Brasil)" for example. All languages suffer from this (see the language picker), some are correct, others are not, it's very inconsistent. When not in the picker, the list that shows currently added languages should reflect this same improvement as it suffers from the same thing. "português europeu" as it reads when we manually edit the plasma-localerc to be pt_PT, should instead read "Português (Portugal)" as should the languages picker. Languages should all also follow a more professional naming scheme: "American English" should instead be "English (US)". British English should be English (UK), and so on and so forth.

The "home page" of this "languages and region" KCM (I don't know their real names) also shows different things in the "Language" part: when we have "pt_PT" manually added in plasma-localerc, this reads "português europeu" (should read Português (Portugal) too), and when we select "português (brasil)" this reads "português" only. However the "português" option in the language picker actually is the one that means "pt" (the bug) and when this one is selected the home page also reads "português". So the home page says "português" no matter which of the two availabme options in the picker is selected. When we click "modify" to enter into the edit language settings, it says different things again. I know this is all very confusing, it also is for me as it took me a lot of time to figure out what the hell was happening here. Can't imagine this being any easier for less techy users.

Another related random bug I encontered when playing with this, is that some specific languages, for example and for coincidence the "português (brasil)" being one of them, it makes plasma-localerc completely lose the [Formats] part of it. Only the [Translations] stays. Funnily enough, choosing "português" makes the [Formats] reappear and actually say "pt_PT.UTF-8" which is absolutely correct. The [Translations] bellow it is the one where the "pt" bug happens.

So to sum up all this, this is what I report and suggest:
1 - "português" in the new languages KCM still links to [pt] formats which is wrong.
2 - The "português" option [pt] in the add/edit languages picker should be removed as it does not reflect the European Portuguese locale, since all apps think [pt] is the same as [pt_BR].
3 - "Portuguese (Portugal)" should be added instead, with the correct [pt_PT] format.
4 - All language names/text there should be correctly capitalized, for example "português (brasil)" should be "Português (Brasil)". Many languages suffer from this, see the language picker. Maybe also make them all follow a consistent and professional naming scheme like "English (US)" instead of the current "American English".
5 - The page that shows all currently added languages should also reflect the point 4.
6 - The home page of this region and languages KCM is not showing the language names correctly too, refer to point 4 again.
7 - There is a small bug that makes [Formats] in plasma-localerc disappear in some languages, for example "português (brasil)". Only [Translations] is there. Choosing another language (one that is not affected by this) brings back [Formats].
Comment 4 Nate Graham 2023-01-02 22:10:15 UTC
CCing the author of those changes.
Comment 5 Bug Janitor Service 2023-01-04 13:55:32 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/plasma-workspace/-/merge_requests/2478
Comment 6 hanyoung 2023-01-08 06:37:04 UTC
In the short term the Region and Language KCM will automatic set "pt" to "pt_PT". In the long term, we'll rename the 'pt' directory in i10n svn to 'pt_PT'. See the discussion on https://discuss.kde.org/t/rename-pt-in-i10n-svn-to-pt-pt/118
Comment 7 Luigi Toscano 2023-01-08 13:20:19 UTC
Let's not promise a solution which hasn't been agreed upon. Other relevant software in the free software world based on gettext (I've checked GNU, GNOME and LibreOffice) uses pt. Does it mean they are broken too? If there is an issue, maybe it should be solved in gettext.
Comment 8 Nate Graham 2023-01-10 15:41:52 UTC
Git commit efbef4f66365f16f545cad6d2cce460fcb212697 by Nate Graham, on behalf of Han Young.
Committed on 10/01/2023 at 15:41.
Pushed by ngraham into branch 'master'.

kcms/regionanglang: explicitly set pt to pt_PT

Explicitly set pt to pt_PT as a workaround for GNU Gettext and CLDR
treating the default dialect of 'pt' differently.

See the discussion on: https://mail.kde.org/pipermail/kde-i18n-doc/2023-January/001340.html
FIXED-IN: 5.27

M  +13   -1    kcms/region_language/languagelistmodel.cpp

https://invent.kde.org/plasma/plasma-workspace/commit/efbef4f66365f16f545cad6d2cce460fcb212697