Bug 317569

I can confirm that i have extremely slow KMail, and that revert resolves the problem. I strongly advise this gets reverted for 4.10.2.

Simeon, please check this report and contact the release-team asap if the nepomuk-core tarballs needs a respin

Git commit 3b64b5ac2372b216a25ca41d74b35b2e7bc59f1c by Simeon Bird.
Committed on 30/03/2013 at 00:17.
Pushed by sbird into branch 'KDE/4.10'.

Revert "ResourceWatcher: switch to using KDbusConnectionPool"

This reverts commit 1f48b58bdb585b8ddbc131f1500ceb718ce82363.
which is reported to cause extreme slowness in kmail, until we
understand why.

M  +2    -3    libnepomukcore/datamanagement/resourcewatcher.cpp

http://commits.kde.org/nepomuk-core/3b64b5ac2372b216a25ca41d74b35b2e7bc59f1c

Well nuts. Commit reverted. Will contact release-team@kde.org . The commit in question was a second fix for bug 305024, but the primary fix  will have to do until I figure out why this doesn't work. Thanks very much for finding this.

I'm seeing this in master as well.

I merged the revert to master - try it now.

Yup, problem gone now. Thanks.

OK I can reproduce this after restarting nepomuk (not just kmail, which is the only thing I tested before).

Good opportunity for trying Andreas' new dbus monitor (kde:scratch/ahartmetz/d-sel.git)
It shows this:
http://www.davidfaure.fr/2013/dbustimeout.png
25 million usec = 25 s = dbus timeout. The call to "sparqlQuery" (into the nepomukqueryservice) takes more than 25s so it times out. And meanwhile, the call made by the kmail messagelist, "watch", also going to the same process (under its alias "DataManagement"), has to wait for these 25s to pass.

Sounds to me like it only worked by change before: the queryservice was using a different dbus connection than the DataManagement, but only because one of them was using the main thread's dbus connection by mistake.

Sounds to me like the nepomukqueryservice should have its own separate dbus connection, if it's used for blocking calls that make soprano queries.

Why use kde functions when Qt has similar functions too? if they are missing anything, just contribute improvements to upstream Qt instead?

Long story: libdbus isn't threadsafe, so the only solution currently is to create one DBus connection per thread, which KDBusConnectionPool provides. Qt itself only has a basic singleton, which leads to crashes and races in libdbus if used from multiple threads. But Qt doesn't want to provide a per-thread-connection api (they just tell us to do that ourselves), because ideally at some point Qt will have its own DBus implementation (without libdbus) and then the single shared connection will be fine even in multiple threads.

I think the actual issue here is that one-connection-per-thread isn't enough if there are two very different services in that thread; we need one-connection-per-service, so that long-running sparql queries don't make other faster dbus calls wait. I'd like Vishesh to take a look though, I'm not sure about the overall design of the process that handles these services, and which threads are supposed to do what, etc.

(In reply to comment #8)
> It shows this:
> http://www.davidfaure.fr/2013/dbustimeout.png
> 25 million usec = 25 s = dbus timeout. The call to "sparqlQuery" (into the
> nepomukqueryservice) takes more than 25s so it times out. And meanwhile, the
> call made by the kmail messagelist, "watch", also going to the same process
> (under its alias "DataManagement"), has to wait for these 25s to pass.
> 
> Sounds to me like it only worked by change before: the queryservice was
> using a different dbus connection than the DataManagement, but only because
> one of them was using the main thread's dbus connection by mistake.
> 
> Sounds to me like the nepomukqueryservice should have its own separate dbus
> connection, if it's used for blocking calls that make soprano queries.

This is strange. Cause the query service calls are not blocking. I'll try to debug it using Andrea's tool.

because nepomuk was replaced by baloo