Bug 254198

Summary: Kwalletd locks when 2 applications try to open the same wallet, the second using the D-BUS interface
Product: [Applications] kwalletmanager Reporter: Andrei Mihaila <andrei.mihaila>
Component: generalAssignee: Valentin Rusu <valir>
Status: RESOLVED FIXED    
Severity: major CC: adaptee, christo.candiotes, christopherheiny, cognifloyd, craig.magina, daniel.pena.arteaga, dietrichmathias, dima, eArquilla, frederic.coiffier, getaceres, gree, joseph.2011, juergen.sauer, juergennagel, julian, kde, leon.maurer, lieven.vanacker, mattm3a, michaelaquilina, null, ochominutosdearco, philipp.woelfel, phlogi1, predder, rdieter, someuniquename, svenstaro, tais.hansen, thanosk, travier, valir, vasyl.demin, xenoterracide, xjakub, yulia.novozhilova
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Gentoo Packages   
OS: Linux   
See Also: https://bugs.kde.org/show_bug.cgi?id=317305
Latest Commit: Version Fixed In: 4.11.2
Sentry Crash Report:

Description Andrei Mihaila 2010-10-14 21:55:30 UTC
Version:           unspecified (using KDE 4.5.2) 
OS:                Linux

This happens most of the times at system startup when both say... KMail/Kontact and Pidgin with the KWallet plugin (http://kde-apps.org/content/show.php/Libpurple+KWallet+Plugin?content=127658 ) try to open the default wallet. If Pidgin goes first everything is ok, if KMail goes first and Pidgin second kwalletd seems to lock. In a few seconds the Pidgin plugin gets a timeout and KMail hangs. I have to restart kwalletd in order for it to work (only tried by killing it and starting it again).

Reproducible: Always

Steps to Reproduce:
I've tried to make it as easy as possible to reproduce using qdbus. Assuming there is a wallet called kdewallet the next 2 lines should make kwalletd lock.

qdbus org.kde.kwalletd /modules/kwalletd org.kde.KWallet.open kdewallet 1 'Test1' &
qdbus org.kde.kwalletd /modules/kwalletd org.kde.KWallet.open kdewallet 2 'Test2' &


Actual Results:  
The response after a few seconds is:

Error: org.freedesktop.DBus.Error.NoReply
Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Error: org.freedesktop.DBus.Error.NoReply
Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

And after that any request for kwalletd hangs if via the C++ interface or gets timed-out if via D-BUS.

Expected Results:  
The D-BUS request should be handled correctly - that is queued. After the user answers the first request the second should be shown. If the user takes too long to respond (enter password/click Allow) and you can't find another solution but to give it a timeout - that should be transparent so that the application would know it needs to try and connect again.

I had a look at the kwalletd code (it was early this spring, maybe things changed since then) and couldn't find anything obviously wrong... I'm thinking it could be the generated D-Bus wrapper.

Thanks in advance for solving this (or at least for reading my report).
Comment 1 Andrei Mihaila 2011-02-01 21:11:51 UTC
Still there in KDE versions 4.5.3 to 4.6. I wish I could find some time to hunt the problem myself ... Thanks for reading anyway.
Comment 2 Jürgen Nagel 2011-03-11 19:16:00 UTC
I stumbled over this bug, too, with Chromium and Kopete locking up kwalletd.
Comment 3 Andrei Mihaila 2011-06-16 19:04:01 UTC
Still there in KDE 4.6.4... Additionally, found that after kwalletd is locked the wallet manager shows it as being empty, although it is not (after restarting the daemon all my passwords were there).
Comment 4 Daniel 2011-07-29 13:30:23 UTC
I also found this problem with KDE SC 4.6.5 in Archlinux. For example, Akonadi IMAP resources will just hang there without being updated, no passwords shown in kwalletmanager, no way to create new wallets, chromium unable to fill password fields if just started, ... If I then kill kwalletd, suddenly the dialogs asking for all password appear, just like without using kwallet, and everything seems to be fixed until next kwallet hang.
Comment 5 Vasyl Demin 2011-09-17 21:32:06 UTC
Same problem with Kopete and Chromium, KDE 4.7.1
Comment 6 Cyrill Helg 2011-10-20 20:24:48 UTC
I can confirm this bug, also running arch linux.
Comment 7 Cyrill Helg 2011-10-20 20:25:24 UTC
*** This bug has been confirmed by popular vote. ***
Comment 8 Tais P. Hansen 2011-10-25 07:03:59 UTC
I can confirm this behavior.

I use keychain with ksshaskpass in my bashrc, so I have to -KILL kwalletd every single time I log in to KDE and want to use the console.
Comment 9 Gábor Gludovátz 2011-12-23 07:27:02 UTC
Same problem with KMess and KRDC. KDE 4.7.3.
Comment 10 Sven-Hendrik Haase 2012-04-03 13:06:07 UTC
Same problem in KDE 4.8.1.
Comment 11 Jacob Floyd 2012-07-28 04:23:58 UTC
This looks like an old bug: #123583

I have ksshaskpass and google-chrome cause some kind of race condition hanging ksshaskpass, and kwalletd. Google-chrome continues like nothing happened, except that it doesn't have any saved passwords (as expected with this issue).

I'm in gentoo with kde 4.8.3 with a 64 bit kernel.
Comment 12 Joseph Reagle 2013-01-03 12:51:26 UTC
Still present in 4.9.4 .... still using the X or gnome ssh-askpass....
Comment 13 Frédéric COIFFIER 2013-01-25 14:34:17 UTC
Still present in KDE 4.9.98 (alias 4.10 RC3).
And seems much more annoying than previously...
Comment 14 kde 2013-02-06 08:43:43 UTC
This problems hits me while starting KDE (version 4.9.5 on kubuntu 12.10) with both network manager plasma widget and chromium: kwalletd hangs and must be killed to get things working again :-(!

Any news on this annoying issues?
Comment 15 Nathan 2013-02-06 18:36:51 UTC
Is this actually being worked on? 

It's rather high impact if kwallet is being used to store wireless passwords. I was unable to get online until I figured out what was going on and disabled the wallet.

More duplicates:
https://bugs.kde.org/show_bug.cgi?id=307731
https://bugs.kde.org/show_bug.cgi?id=259229
Comment 16 Christoph Feck 2013-02-13 00:24:16 UTC
*** Bug 307731 has been marked as a duplicate of this bug. ***
Comment 17 Gael Beaudoin 2013-02-21 07:06:52 UTC
Getting the same problem upon a session restore. If I have no application asking for a password at startup, then kwalletd works fine. If anything asks for a password during startup/session restore (chrome, chokoq or amarok in my case), then they never get the password and behave abnormally.

Kubuntu 12.10 64bits, KDE 4.10.
Comment 18 Juergen Sauer 2013-03-19 12:29:14 UTC
Same problem as here. Ugly Bug.
Comment 19 Milos Jakubicek 2013-04-16 12:11:41 UTC
Same problem here, definitely not Gentoo specific (I'm using Fedora).
Comment 20 Frédéric COIFFIER 2013-04-17 07:30:16 UTC
Still present in 4.10.2
Comment 21 Frédéric COIFFIER 2013-05-20 18:04:00 UTC
In my case, the problem was due to the ownCloud client : if this one isn't restored at the session start, the problem disappears.
Comment 22 Philipp Woelfel 2013-05-20 21:30:56 UTC
Frédéric, is it possible that this is because if you don't start the client then there is no more concurrent access to kwallet by two clients?
Comment 23 Philipp Woelfel 2013-05-20 22:16:06 UTC
B.t.w. I have the same problem with openSUSE kde 4.10.2 binaries. I "fixed" it by removing the kwallet password.
Comment 24 Frédéric COIFFIER 2013-05-21 08:17:38 UTC
Philipp, in my case, at KDE session startup, many applications are restored at the same time and several of them require a kwallet access : akonadi (2 x POP3, Google resources...), konqueror, amarok, firefox, kopete.
Comment 25 michaelaquilina@gmail.com 2013-07-18 20:45:45 UTC
I can confirm this issue occurs to me too. In my case wifi gets stuck authenticating and i have to kill and restart kwalletd in order to fix the issue at each startup.
Comment 26 Christo Candiotes 2013-07-27 05:05:00 UTC
I have the same issue only at startup with possibly a race condition as mentioned earlier in this bug report. Running Manjaro 0.8.6 with KDE 4.10.5. Symptoms is that my instant messaging (KDE Telepathy), KMail etc cannot connect being unable to get the passwords from KWallet. Manually opening the wallet in KWallet Manager gives NO entries in the opened wallet. Merely closing the wallet and re-opening does not help. Only workaround for me is to go to Settings->System Settings->Account Details->KDE Wallet then untick "Enable the KDE wallet subsystem" (thereby killing the current instance of KWalletManager and then just ticking it again and Apply it. All works fine then until next reboot. Hope this can point someone in the right direction to resolve this bug.
Comment 27 Lieven Van Acker 2013-07-30 12:03:20 UTC
I can confirm the above. Also running on KDE 4.10.5. Only workaround is to set an empty password on the wallet, which completely misses the intended purpose of the application. I cannot understand the lack of attention to this bug, cause it renders the complete DE very insecure.
Comment 28 Valentin Rusu 2013-08-26 19:28:41 UTC
Found race condition in the wallet opening logic. Working on it...
Comment 29 Valentin Rusu 2013-08-27 18:02:38 UTC
The wallet gets locked because of the nested event loops and in the situation where the calling application doesn't have a main window, just like the qdbus call exposed by Andrei. It seems that the password prompt needs some window to attach to and if such a window is given, via the wId open method parameter, then the wallet won't lock. This can be demonstrated by the KWallet Manager application, wich never locks the daemon. In fact, on my system, I never encountered this lock proble.

I'll continue the work and try to solve the problem, but it's a hard one. Meanwhile, I suggest a workaround. Your application should wait for the GUI thread to start, then initiate wallet open call taking care to give the main window's ID.
Comment 30 Andrei Mihaila 2013-08-27 19:02:40 UTC
@Valentin:
First of all thank you very much for investigating this issue.

Tried the workaround, don't know if I did it right, but it didn't work - started Amarok and KCalc (could be any 2 KDE apps) then ran in a terminal:
--------------
qdbus org.kde.kwalletd /modules/kwalletd org.kde.KWallet.open kdewallet `xwininfo -int -name 'KCalc' | grep 'Window id:' | cut -d' ' -f4` 'Test KCalc' &
qdbus org.kde.kwalletd /modules/kwalletd org.kde.KWallet.open kdewallet `xwininfo -int -name 'Amarok' | grep 'Window id:' | cut -d' ' -f4` 'Test Amarok' &
--------------

Got roughly similar results to what I wrote in the initial bug report:
--------------
Error: org.freedesktop.DBus.Error.NoReply
Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
--------------

and a locked kwallet.


I wish I could help with the fix but I don't have enough QT knowledge (if at all - I was able to tell that the lock happens when the nested event loop starts but didn't know why).
When creating the fix one should bear in mind that there might be cases when a window-less client asks for access to KWallet (for example subversion or git). So for this kind of situation it may be better to have a totally independent thread/process that waits for the wallet to open and then sends the result back to the caller.
Comment 31 Valentin Rusu 2013-08-28 12:26:34 UTC
@Andrei

I suppose you didn't wait to enter the password in the open dialog popped by the first command. Isn't it?
If you enter the correct password and the wallet opens, kwalletd no longer gets stuck.

If I'm correct, then you hit another bug I already planned to fix. That one is about a rush condition when opening wallet.
Comment 32 Andrei Mihaila 2013-08-28 12:53:42 UTC
@Valentin
Yes, you are correct. That was the issue that I always had at start-up when 2 apps request access to KWallet almost at the same time (and can't make them wait for each other - in which case it would work fine).

OK,  I look forward to your solution.
Comment 33 Gael Beaudoin 2013-08-28 12:55:27 UTC
This is a very annoying bug, thanks you very very much for working on this @Valentin Rusu !
Comment 34 Valentin Rusu 2013-08-30 23:07:47 UTC
Fixed the bug in kwalletd by kdelibs/kwallet needs also fixing. Please hold on, as I need to commit both in order to prevent regressions.
Comment 35 Valentin Rusu 2013-09-01 20:29:25 UTC
Git commit b0f9053ed8abff4ef973b10842f761422ee17f41 by Valentin Rusu.
Committed on 31/08/2013 at 23:28.
Pushed by vrusu into branch 'master'.

Fix the synchronous-mode wallet open logic

The wallet synchronous open requests now use qdbus delayed replies.
The execution path now returns to the main event loop instead of
a nested event loop. The wallet opening UI logic is correctly handled
no longer leading to a frozen kwalletd.

Beware that this commit should be used along with the corresponding
fix of the kdelibs/kdeui module. Failing to updating kdelibs lead to
an aparently similar freeze condition, as kdeui also used an internal
event loop, faking synchronous operation when making async kwalletd calls.

M  +0    -1    kwalletd/CMakeLists.txt
M  +53   -21   kwalletd/kwalletd.cpp
D  +0    -46   kwalletd/kwalletopenloop.cpp
D  +0    -50   kwalletd/kwalletopenloop.h
M  +1    -0    kwalletd/main.cpp

http://commits.kde.org/kde-runtime/b0f9053ed8abff4ef973b10842f761422ee17f41
Comment 36 Valentin Rusu 2013-09-01 20:29:56 UTC
Git commit f8fea3f01c85eb0d6d479647ac27fe431846a1ae by Valentin Rusu.
Committed on 31/08/2013 at 23:16.
Pushed by vrusu into branch 'master'.

Fix the synchronous-mode wallet open logic

The wallet opening logic, for the synchronous mode, had a nested
event loops problem, leading to frozen kwalletd. That was because
kwalletd wasn't using qdbus delayed replies. kdelibs used
asynchronous open methods even for the synchronous mode, coupled
with an internal event loop to simulate synchronous mode.
This commit removes that internal event loop, as the kwalletd now
blocks on synchronous wallet open requests.

M  +8    -20   kdeui/util/kwallet.cpp

http://commits.kde.org/kdelibs/f8fea3f01c85eb0d6d479647ac27fe431846a1ae
Comment 37 Valentin Rusu 2013-09-01 20:32:50 UTC
Please note that this fix consists of two commits, one on kde-runtime and a corresponding on in kdelibs. You should absolutely update and build both modules in order to get the fix. Failing to do that, you get another case of kwallet freeze, in kdelibs this time.
Comment 38 Valentin Rusu 2013-09-01 22:18:46 UTC
Git commit aff38f3cfeca0abb93e899c54f27e382cfa427ac by Valentin Rusu.
Committed on 01/09/2013 at 22:15.
Pushed by vrusu into branch 'frameworks'.

Fix the synchronous-mode wallet open logic

The wallet opening logic, for the synchronous mode, had a nested
event loops problem, leading to frozen kwalletd. That was because
kwalletd wasn't using qdbus delayed replies. kdelibs used
asynchronous open methods even for the synchronous mode, coupled
with an internal event loop to simulate synchronous mode.
This commit removes that internal event loop, as the kwalletd now
blocks on synchronous wallet open requests.

This commit has the same effect as f8fea3f01c85eb0d on kdelibs 4.11
but cherry-picking won't work because of KF5 source splitting. I just
redone it here.

M  +9    -21   staging/kwallet/src/kwallet.cpp

http://commits.kde.org/kdelibs/aff38f3cfeca0abb93e899c54f27e382cfa427ac
Comment 39 Valentin Rusu 2013-09-02 19:36:26 UTC
*** Bug 322925 has been marked as a duplicate of this bug. ***
Comment 40 Valentin Rusu 2013-09-02 19:52:36 UTC
*** Bug 319138 has been marked as a duplicate of this bug. ***
Comment 41 Valentin Rusu 2013-09-02 20:12:46 UTC
*** Bug 317305 has been marked as a duplicate of this bug. ***
Comment 42 Valentin Rusu 2013-09-02 20:15:51 UTC
*** Bug 316624 has been marked as a duplicate of this bug. ***
Comment 43 Valentin Rusu 2013-09-02 20:30:50 UTC
*** Bug 312123 has been marked as a duplicate of this bug. ***
Comment 44 Valentin Rusu 2013-09-02 20:36:59 UTC
*** Bug 308540 has been marked as a duplicate of this bug. ***
Comment 45 Valentin Rusu 2013-09-02 21:01:41 UTC
*** Bug 300991 has been marked as a duplicate of this bug. ***
Comment 46 Caleb Cushing 2013-09-03 03:59:22 UTC
Will this be in 4.11.1?
Comment 47 Valentin Rusu 2013-09-03 19:56:38 UTC
Well, 4.11.1 has already been tagged, so no. It'll be 4.11.2
http://techbase.kde.org/Schedules/KDE4/4.11_Release_Schedule
Comment 48 Rex Dieter 2013-09-03 20:33:52 UTC
Valentin, I don't see your commits in the KDE/4.11 branch at all yet (only master, frameworks branches).  did you mean to backport (or merge) them and forget?
Comment 49 Valentin Rusu 2013-09-03 20:43:51 UTC
*** Bug 259229 has been marked as a duplicate of this bug. ***
Comment 50 Valentin Rusu 2013-09-03 21:00:54 UTC
*** Bug 245869 has been marked as a duplicate of this bug. ***
Comment 51 Valentin Rusu 2013-09-03 21:27:02 UTC
Git commit c5bdc96316e9e5f727085f65fcf8eb059ebf4357 by Valentin Rusu.
Committed on 31/08/2013 at 23:28.
Pushed by vrusu into branch 'KDE/4.11'.

Fix the synchronous-mode wallet open logic

The wallet synchronous open requests now use qdbus delayed replies.
The execution path now returns to the main event loop instead of
a nested event loop. The wallet opening UI logic is correctly handled
no longer leading to a frozen kwalletd.

Beware that this commit should be used along with the corresponding
fix of the kdelibs/kdeui module. Failing to updating kdelibs lead to
an aparently similar freeze condition, as kdeui also used an internal
event loop, faking synchronous operation when making async kwalletd calls.

M  +0    -1    kwalletd/CMakeLists.txt
M  +53   -21   kwalletd/kwalletd.cpp
D  +0    -46   kwalletd/kwalletopenloop.cpp
D  +0    -50   kwalletd/kwalletopenloop.h
M  +1    -0    kwalletd/main.cpp

http://commits.kde.org/kde-runtime/c5bdc96316e9e5f727085f65fcf8eb059ebf4357
Comment 52 Valentin Rusu 2013-09-03 21:28:33 UTC
Git commit 9027e0620d1f6bb06cbeb00db1072047ccb8ff13 by Valentin Rusu.
Committed on 31/08/2013 at 23:16.
Pushed by vrusu into branch 'KDE/4.11'.

Fix the synchronous-mode wallet open logic

The wallet opening logic, for the synchronous mode, had a nested
event loops problem, leading to frozen kwalletd. That was because
kwalletd wasn't using qdbus delayed replies. kdelibs used
asynchronous open methods even for the synchronous mode, coupled
with an internal event loop to simulate synchronous mode.
This commit removes that internal event loop, as the kwalletd now
blocks on synchronous wallet open requests.

M  +8    -20   kdeui/util/kwallet.cpp

http://commits.kde.org/kdelibs/9027e0620d1f6bb06cbeb00db1072047ccb8ff13
Comment 53 Valentin Rusu 2013-09-03 21:31:33 UTC
@Rex: thanks for pointing that out!
Comment 54 Mathias Dietrich 2013-09-04 08:45:33 UTC
Thx for fixing this bug Valentin ! 
Great work, especially as it has some many votes.
Comment 55 Valentin Rusu 2013-09-04 12:46:03 UTC
*** Bug 234939 has been marked as a duplicate of this bug. ***
Comment 56 Andrei Mihaila 2013-11-10 21:09:56 UTC
The bug is gone in my KDE 4.11.2 (tested using the 2 commands in my original report). Thank you Valentin!