Bug 429408

Summary: kde-open5 changes case of argument
Product: [Plasma] kde-cli-tools Reporter: Thomas Borowski <mail+kdebugs>
Component: generalAssignee: Aleix Pol <aleixpol>
Status: CONFIRMED ---    
Severity: normal CC: adeptsmail, ayrton, beppe85, boschi, casenet.us, egon.willighagen, ender8282, equal.bowl2131, eric1, francescortiz, gloriouseggroll, hpj, iamgrief, janl, jchevarley, jeremy.william.murphy, kahennig, kde, kde, kde, kustodian, leledumbo_cool, levon.avagyan, lf22, me, me, me, nico.kruber, nigels, noga.dany, ntropia, oded, pia, rafaeln.dev, rocketraman, rodrigogarciama, root, segiralph, sokann, sven, tennov+kde
Priority: NOR Keywords: accessibility
Version: 5.24.5   
Target Milestone: ---   
Platform: Arch Linux   
OS: Linux   
URL: https://casenet.us/
Latest Commit: Version Fixed In:

Description Thomas Borowski 2020-11-20 18:00:08 UTC
SUMMARY

Calling

  kde-open5 'thunderlink://messageid=e4b0c66f-07c8-453d-abd7-97e416ec7b56@Spark' 

results in the error

  Couldn't find an email message for ThunderLink
  thunderlink://messageid=e4b0c66f-07c8-453d-abd7-97e416ec7b56@spark

Note that the 'S' in 'Spark' at the end of the argument is lowercase, not uppercase as it was passed. Apparently kde-open5 converts the argument to lowercase, resulting in the message ID passed to Thunderlink not being valid.

Passing an all-lowercase argument works as expected (Thunderbird opens the corresponding message).

kde-open5 should not change the case of the argument which it was passed but rather pass it on as-is.

Here's the contents of the corresponding desktop file:

[Desktop Entry]
Encoding=UTF-8
Name=Thunderlink Opener
Comment=Open specific Message-ID Thunderbird
Exec=thunderbird -thunderlink %u
Terminal=false
X-MultipleArgs=false
Type=Application
Icon=thunderbird
Categories=Application;Network;Email;
MimeType=x-scheme-handler/thunderlink;
StartupNotify=true
NoDisplay=true



SOFTWARE/OS VERSIONS
Operating System: Kubuntu 20.04
KDE Plasma Version: 5.18.5
KDE Frameworks Version: 5.68.0
Qt Version: 5.12.8
Kernel Version: 5.4.0-54-generic
OS Type: 64-bit
Processors: 4 × Intel® Core™ i5-6300U CPU @ 2.40GHz
Memory: 15,5 GiB of RAM
Comment 1 Casenet 2021-11-11 10:52:19 UTC
*** Removed by KDE Sysadmin for violation of the KDE Community Code of Conduct ***
Comment 2 Thomas Borowski 2021-11-11 11:00:36 UTC
(In reply to Casenet from comment #1)
> Depending on the used android keyboard you get the following reactions when
> writing uppercase - https://casenet.us/
> - Gboard: The Windows system usually reacts with the typical alert-sound
> when I press the shift key on the android keyboard to write uppercase. The
> transmission of the uppercase character works so far, dispite the
> windows-alert-sound.
> - Gboard: Uppercase characters appear lowercase in a RDP session
> - SwiftKey: Works as expected when not in RDP
> - SwiftKey: Uppercase characters appear lowercase in a RDP session

I think you're talking about a completely unrelated error. This bug is regarding to the kde-open5 program under KDE Plasma and has nothing to do with Android, Gboard, Windows, RDP, etc.
Comment 3 Egon Willighagen 2022-03-05 07:48:13 UTC
I got here via this StackOverflow workaround for a problem where Slack does not open all workspaces. The SO flow suggests the same upper/lower rewrite by KDE as a suspect. Here is some data from my machine just now (for privacy I use fake codes, like WORKSPACEID, xxx, YYYY, and zzz, of course with upper/lower casing exact):

    /bin/sh /usr/bin/xdg-open slack://WORKSPACEID/magic-login/xxxx/YYYY/magic-login/zzz-zzzzz-zzzzzzzzz
    kde-open5 slack://WORKSPACEID/magic-login/xxxx/YYYY/magic-login/zzz-zzzzz-zzzzzzzzz
    /usr/lib/slack/slack --enable-crashpad slack://workspaceid/magic-login/xxxx/YYYY/magic-login/zzz-zzzzz-zzzzzzzzz

I hope you trust my translation of my observation, but I can confirm that WORKSPACEID goed into kde-open5 as upper case, but is converted to lower case when passed to the `/usr/lib/slack/slack` application. And Slack seems to be case sensitive as suggested in the SO post: if I upper case back the WORKSPACEID in the `/usr/lib/slack/slack` call, then it opens properly.

* Slack 4.23.0
* kde-open5/kioclient 5.20.5
Comment 4 Levon Avagyan 2022-06-10 07:45:45 UTC
(In reply to Egon Willighagen from comment #3)
> I got here via this StackOverflow workaround for a problem where Slack does
> not open all workspaces. The SO flow suggests the same upper/lower rewrite
> by KDE as a suspect. Here is some data from my machine just now (for privacy
> I use fake codes, like WORKSPACEID, xxx, YYYY, and zzz, of course with
> upper/lower casing exact):
> 
>     /bin/sh /usr/bin/xdg-open
> slack://WORKSPACEID/magic-login/xxxx/YYYY/magic-login/zzz-zzzzz-zzzzzzzzz
>     kde-open5
> slack://WORKSPACEID/magic-login/xxxx/YYYY/magic-login/zzz-zzzzz-zzzzzzzzz
>     /usr/lib/slack/slack --enable-crashpad
> slack://workspaceid/magic-login/xxxx/YYYY/magic-login/zzz-zzzzz-zzzzzzzzz
> 
> I hope you trust my translation of my observation, but I can confirm that
> WORKSPACEID goed into kde-open5 as upper case, but is converted to lower
> case when passed to the `/usr/lib/slack/slack` application. And Slack seems
> to be case sensitive as suggested in the SO post: if I upper case back the
> WORKSPACEID in the `/usr/lib/slack/slack` call, then it opens properly.
> 
> * Slack 4.23.0
> * kde-open5/kioclient 5.20.5
Same problem with kde-open5/kioclient 5.24.90 and slack-4.26.1-0.1.fc21.x86_64.rpm
Comment 5 Egon Willighagen 2022-06-20 09:35:59 UTC
(In reply to Egon Willighagen from comment #3)
> I got here via this StackOverflow workaround for a problem where Slack does

SO link: https://stackoverflow.com/questions/70867064/signing-into-slack-desktop-not-working-on-4-23-0-64-bit-ubuntu/71062113#71062113

The approach to get the URLs is basically:

```shell
while sleep .1; do ps aux | grep slack | grep -v grep | grep magic; done
```

Can I just say this status of this bug should not be mere NORMAL but actually is a data corruption bug? Maybe not one of the severest, but annoyingly annoying.
Comment 6 Connor Schunk 2022-06-21 20:11:54 UTC
I'm not extremely familiar with the inner workings of Qt, but encountered this problem and decided to dig into it a bit in hopes of helping this bug get resolved. From what I can tell from the code kde-open5 seems to use (https://invent.kde.org/plasma/kde-cli-tools/-/blob/master/kioclient/kioclient.cpp#L235), QURL is involved (https://invent.kde.org/plasma/kde-cli-tools/-/blob/master/kioclient/urlinfo.h).

In QUrl (https://code.woboq.org/qt5/qtbase/src/corelib/io/qurl.cpp.html), I notice "nameprepping" and RFCs related to normalization are mentioned a few times, e.g.
>  Note that the case folding rules in \l{RFC 3491}{Nameprep}, which QUrl conforms to, require host names to always be converted to lower case, regardless of the Qt::FormattingOptions used

If I had to guess (with my limited understanding), I would assume the problem lies in the call to setHost, which in turn calls qt_ACE_do (https://code.woboq.org/qt5/qtbase/src/corelib/io/qurl.cpp.html#1365) - the normalization to lowercase probably lies somewhere in that chain.
Comment 7 kde 2022-07-07 00:39:33 UTC
https://code.woboq.org/qt5/qtbase/src/corelib/io/qurlidna.cpp.html#2536

There's your root cause. Bitwise OR of an ASCII character with 0x20 results in a change to lower case. This appears to be taking place in a unicode-to-ASCII conversion, which seems to be erroneously converting to lower case as a matter of course. The previously cited RFC, 3490/3491, obsoleted by 5890/5891, does make mention of lower case conversions, but strictly in the context of generating unique DNS labels for hostnames. 

Relevant:
https://www.rfc-editor.org/rfc/rfc5890
Comment 8 Joe 2022-11-09 02:30:41 UTC
Just wanted to say I came across this the other day and as Egon so nicely put it, it was "annoyingly annoying".  I ended up just doing things in Firefox to make signing into Slack work.
Comment 9 CyberKitsune 2022-12-06 21:32:05 UTC
+1 on seeing this happen with slack, KDE 2.26.4 and Qt 5.15.7

Judging by comment #7 would this actually be a Qt bug? Should it be worked around by kioclient or would we need to report and wait on a fix from Qt?
Comment 10 Oded Arbel 2022-12-08 11:58:40 UTC
I have the same issue with Slack - using "Open in desktop app" from the website running under Chrome, results in kde5-open being called and mangling the authority section of the URL.

(In reply to kde from comment #7)
> which seems to be erroneously converting to
> lower case as a matter of course. The previously cited RFC, 3490/3491,
> obsoleted by 5890/5891, does make mention of lower case conversions, but
> strictly in the context of generating unique DNS labels for hostnames. 

I believe the correct RFC for QURL implementations - in this case - is RFC 3986: URI Generic Syntax, where section 3.2.2 says: "The host subcomponent is case-insensitive". Granted that a reading of the spec to mean "fold everything to lowercase" is reaching a bit, but as per RFC 1122, p1.2.2 - receivers should accept lowercased host names.

According to discussions with Slack support, they are aware of this issue -which depending on how you look at it, is either KDE doing unexpected things, or Slack not complying with RFC 3986 - even though there's no way to know that because they don't have a public bug tracker. The also suggested a workaround: Apparently if you launch the `slack://...` URL from Firefox, it works fine: the URL isn't mangled.

I'm not sure what Firefox uses instead of kde5-open, but looking at the code for xdg-open, I can see that it has code to open flatpak apps that uses a dbus call, and in that case no case folding is happening and the application is launched successfully - event if the app is not a flatpak app. For example - the following command launches the DEB installed Slack on my system:

gdbus call --session --dest org.freedesktop.portal.Desktop --object-path /org/freedesktop/portal/desktop --method org.freedesktop.portal.OpenURI.OpenURI "" "slack://MAGICTOKEN/magic-login/id-id-id-id-id-id-id-id-id-id-id-id-id-id-id-id-id-id-id-id" {}

This call is handled by the xdg-desktop-portal process. Also, it may only work because Firefox is running in a snap?
Comment 11 Nicolai Langfeldt 2023-01-31 08:53:37 UTC
I also had this problem.
Comment 12 Hans-Peter Jansen 2023-03-16 15:41:28 UTC
This is getting slightly off-topic now, but is still related!

Newer slack versions (using 4.29 here) seem to suffer from this differently.

On my Tumbleweed system (specs below), I've straced it, but the slack URI appeared lowercased already (electron?), and is not fed through kde-open5 anymore!

For all, that come across this AND still want to use the app, here's an antidote: https://pypi.org/project/slackfix, as well as https://github.com/frispete/slackfix. 

Operating System: openSUSE Tumbleweed 20230313
KDE Plasma Version: 5.27.2
KDE Frameworks Version: 5.104.0
Qt Version: 5.15.8
Kernel Version: 6.2.6-3-preempt (64-bit)
Graphics Platform: X11
Processors: 24 × AMD Ryzen 9 3900X 12-Core Processor
Memory: 125.4 GiB of RAM
Graphics Processor: NVIDIA GeForce RTX 3060/PCIe/SSE2
Manufacturer: ASUS
Comment 13 Nigel Stewart 2023-04-12 07:05:08 UTC
Same problem here with Slack v4.29.149 64-bit, Ubuntu 22.04, KDE Plasma 5.24.7

Would recommend prioritising a fix for this, Slack client is daily go-to and does not seem to be at all to blame.
Comment 14 Hans-Peter Jansen 2023-04-13 21:12:11 UTC
Hi Nigel,

you may want to try my slackfix hack - that provides a -v option, allowing you to prove, if it's Qt to blame here, or something else...
Comment 15 Yuriy Vidineev 2023-05-12 19:00:36 UTC
Same problem here with Slack (in snap). 
The solution from https://stackoverflow.com/questions/70867064/signing-into-slack-desktop-not-working-on-4-23-0-64-bit-ubuntu  worked for me
Comment 16 Stefano Forli 2023-06-16 21:06:09 UTC
(In reply to Hans-Peter Jansen from comment #14)
> Hi Nigel,
> 
> you may want to try my slackfix hack - that provides a -v option, allowing
> you to prove, if it's Qt to blame here, or something else...

Hi Hans,
any hints on how to use the script? I am tempted to replace the /usr/bin/slack symlink but that doesn't seem to be the right way to do it.
Comment 17 Hans-Peter Jansen 2023-06-17 08:04:55 UTC
(In reply to Stefano Forli from comment #16)
> (In reply to Hans-Peter Jansen from comment #14)
> > Hi Nigel,
> > 
> > you may want to try my slackfix hack - that provides a -v option, allowing
> > you to prove, if it's Qt to blame here, or something else...
> 
> Hi Hans,
> any hints on how to use the script? I am tempted to replace the
> /usr/bin/slack symlink but that doesn't seem to be the right way to do it.

Just run slackfix instead of slack, I've provided a .desktop file for that reason.
The script runs slack in a controlled way, and supplies the fix at the right moment.
Consequently, make sure, slack isn't running beforehand. 

Feedback is appreciated.
Comment 18 lf22 2023-07-14 09:44:04 UTC
Hi to all, i have same problem only with "short" argument.

example: kde-open5 'callto://545'

change

545 in 0.0.2.33

but if i use callto://545545545

it's all OK!! :((

This is the link of my post in other forum https://unix.stackexchange.com/questions/751435/kde-xdg-mime-and-random-number



Version: kde plasma = 5.27.6
KDE Frameworks = 5.107.0
Qt = 5.15.10 Kernel = 6.4.3-MANJARO (64 bit)
Comment 19 André Werlang 2023-12-17 03:48:05 UTC
(In reply to Connor Schunk from comment #6)
> In QUrl (https://code.woboq.org/qt5/qtbase/src/corelib/io/qurl.cpp.html), I
> notice "nameprepping" and RFCs related to normalization are mentioned a few
> times, e.g.
> >  Note that the case folding rules in \l{RFC 3491}{Nameprep}, which QUrl conforms to, require host names to always be converted to lower case, regardless of the Qt::FormattingOptions used

Except RFC 3491 doesn't make any mention to "case folding". RFC 3490 mentions

> 4) Whenever two labels are compared, they MUST be considered to match
>       if and only if they are equivalent, that is, their ASCII forms
>       (obtained by applying ToASCII) match using a case-insensitive
>       ASCII comparison.  Whenever two names are compared, they MUST be
>       considered to match if and only if their corresponding labels
>       match, regardless of whether the names use the same forms of label
>       separators.

I don't think this applies to QUrl as no comparison between labels takes place.

(In reply to Oded Arbel from comment #10)
> I believe the correct RFC for QURL implementations - in this case - is RFC
> 3986: URI Generic Syntax, where section 3.2.2 says: "The host subcomponent
> is case-insensitive". Granted that a reading of the spec to mean "fold
> everything to lowercase" is reaching a bit, but as per RFC 1122, p1.2.2 -
> receivers should accept lowercased host names.

It's not reaching a bit, RFC recommends lowercasing domain names in section 3.2.2:

> Although host is case-insensitive, producers and normalizers should use lowercase for registered names and hexadecimal addresses for the sake of uniformity, while only using uppercase letters for percent-encodings.

Keyword here is SHOULD. QUrl is not _required_ to do so.

TLDR; Thunderbird should avoid the authority component and use a single "/" character or none at all after the ":". kde-open5 probably doesn't need to normalize anything just to figure out the correct scheme handler.
Comment 20 Tom Crider 2024-01-10 19:29:48 UTC
Based on the information given I made a small workaround patch for qt5-qtbase.  I needed this functionality to work for slack. No idea if it's any kind of acceptable approach/welcome to suggestions:

```
diff --git a/src/corelib/io/qurlidna.cpp b/src/corelib/io/qurlidna.cpp
index a2f0caa6..88748cb3 100644
--- a/src/corelib/io/qurlidna.cpp
+++ b/src/corelib/io/qurlidna.cpp
@@ -2532,10 +2532,10 @@ QString qt_ACE_do(const QString &domain, AceOperation op, AceLeadingDot dot)
                 ushort uc = in->unicode();
                 if (uc > 0x7f)
                     simple = false;
-                if (uc >= 'A' && uc <= 'Z')
-                    *out = QChar(uc | 0x20);
-                else
-                    *out = *in;
+                /*if (uc >= 'A' && uc <= 'Z')
+                    *out = QChar(uc | 0x20);*/
+                /*else*/
+                *out = *in;
             }
         }
 
```
Comment 21 Tom Crider 2024-01-13 02:56:32 UTC
Alternate version for qt6-qtbase:
```
diff --git a/src/corelib/io/qurlidna.cpp b/src/corelib/io/qurlidna.cpp
index bb14e1cd..1c560975 100644
--- a/src/corelib/io/qurlidna.cpp
+++ b/src/corelib/io/qurlidna.cpp
@@ -892,8 +892,19 @@ QString qt_ACE_do(const QString &domain, AceOperation op, AceLeadingDot dot,
     if (domain.isEmpty())
         return {};
 
+    /* Check for url quirk such as slack or thunderlink.
+    We do this before mapDomainName because mapDomainName will
+    cast it to lowercase, and some urls are case sensitive.
+    If that is resolved in the future then we need to call before
+    mappedToAscii, because that strips the : from the url. */
+    QString quirkcheck = "://";
+    if (domain.toStdString().find(quirkcheck.toStdString()) >= 0){
+        return domain;
+    }
+
     bool mappedToAscii;
     const QString mapped = mapDomainName(domain, options, &mappedToAscii);
+
     const QString normalized =
             mappedToAscii ? mapped : mapped.normalized(QString::NormalizationForm_C);
 ```
Comment 22 Tom Crider 2024-01-17 02:00:18 UTC
not sure why i went around my butt to get to my elbow for those :facepalm:, could have been much simpler. assuming the issue only occurs when :// is passed and generally :// shouldn't be passed from applications -- I just ended up doing this:

qt5-qtbase:
-----------
diff --git a/src/corelib/io/qurlidna.cpp b/src/corelib/io/qurlidna.cpp
index a2f0caa6..ad18d5b6 100644
--- a/src/corelib/io/qurlidna.cpp
+++ b/src/corelib/io/qurlidna.cpp
@@ -2490,6 +2490,10 @@ QString qt_ACE_do(const QString &domain, AceOperation op, AceLeadingDot dot)
     if (domain.isEmpty())
         return domain;
 
+    if (domain.toStdString().find("://") >= 0){
+        return domain;
+    }
+
     QString result;
     result.reserve(domain.length());
 
-----------

qt6-qtbase:
-----------
diff --git a/src/corelib/io/qurlidna.cpp b/src/corelib/io/qurlidna.cpp
index bb14e1cd..204085c2 100644
--- a/src/corelib/io/qurlidna.cpp
+++ b/src/corelib/io/qurlidna.cpp
@@ -892,6 +892,10 @@ QString qt_ACE_do(const QString &domain, AceOperation op, AceLeadingDot dot,
     if (domain.isEmpty())
         return {};
 
+    if (domain.toStdString().find("://") >= 0){
+        return domain;
+    }
+
     bool mappedToAscii;
     const QString mapped = mapDomainName(domain, options, &mappedToAscii);
     const QString normalized =
-----------
Comment 23 kahennig 2024-01-17 12:08:24 UTC
I can confirm that this is still happening on kubuntu 23.10 with Plasm 5.27.10
Comment 24 Tom Crider 2024-01-20 00:56:35 UTC
I'm unable to edit my comments. anyway previously applied patches caused some encoding issues, had to limit them further:

+    if ((domain.toStdString().find("slack://") >= 0) || (domain.toStdString().find("thunderlink://") >= 0)) {
+        return domain;
+    }
Comment 25 Bug Janitor Service 2024-01-24 00:58:27 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/kde-cli-tools/-/merge_requests/99
Comment 26 Tom Crider 2024-01-24 01:50:26 UTC
(In reply to Bug Janitor Service from comment #25)
> A possibly relevant merge request was started @
> https://invent.kde.org/plasma/kde-cli-tools/-/merge_requests/99

Tested MR 99 on kde5, modified for thunderlinks and slack:

+    if ((url.startsWith(QLatin1String("thunderlink://"))) || (url.startsWith(QLatin1String("slack://")))) {
+        // :// -> :/// so QUrl doesn't break the payload's

It's working well here. Will test on kde6 shortly
Comment 27 Tom 2024-02-26 14:04:48 UTC
It's also happening to an updated vanilla Archlinux.
Comment 28 Pedro Boschi 2024-03-28 13:20:47 UTC
Issue persists in KDE 6