Bug 317569 - commit 1f48b58bdb585b8ddbc131f1500ceb718ce82363 causes huge lags.
Summary: commit 1f48b58bdb585b8ddbc131f1500ceb718ce82363 causes huge lags.
Status: RESOLVED NOT A BUG
Alias: None
Product: nepomuk
Classification: Unmaintained
Component: storage (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Nepomuk Bugs Coordination
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-03-29 21:37 UTC by Hussam Al-Tayeb
Modified: 2014-09-19 21:10 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Hussam Al-Tayeb 2013-03-29 21:37:21 UTC
http://quickgit.kde.org/?p=nepomuk-core.git&a=commit&h=1f48b58bdb585b8ddbc131f1500ceb718ce82363
this commit in 4.10 branch is causing huge lags in kmail.
kmails takes minutes to open and switching between folders in my inbox takes ages.
new email take a minute to get marked as read.
reverting that commit in kde 4.10 branch makes things fast again.


Reproducible: Always
Comment 1 Hrvoje Senjan 2013-03-29 22:00:58 UTC
I can confirm that i have extremely slow KMail, and that revert resolves the problem. I strongly advise this gets reverted for 4.10.2.
Comment 2 Christophe Marin 2013-03-29 22:01:31 UTC
Simeon, please check this report and contact the release-team asap if the nepomuk-core tarballs needs a respin
Comment 3 Simeon Bird 2013-03-29 23:19:54 UTC
Git commit 3b64b5ac2372b216a25ca41d74b35b2e7bc59f1c by Simeon Bird.
Committed on 30/03/2013 at 00:17.
Pushed by sbird into branch 'KDE/4.10'.

Revert "ResourceWatcher: switch to using KDbusConnectionPool"

This reverts commit 1f48b58bdb585b8ddbc131f1500ceb718ce82363.
which is reported to cause extreme slowness in kmail, until we
understand why.

M  +2    -3    libnepomukcore/datamanagement/resourcewatcher.cpp

http://commits.kde.org/nepomuk-core/3b64b5ac2372b216a25ca41d74b35b2e7bc59f1c
Comment 4 Simeon Bird 2013-03-29 23:24:50 UTC
Well nuts. Commit reverted. Will contact release-team@kde.org . The commit in question was a second fix for bug 305024, but the primary fix  will have to do until I figure out why this doesn't work. Thanks very much for finding this.
Comment 5 Blackpaw 2013-03-30 00:20:52 UTC
I'm seeing this in master as well.
Comment 6 Simeon Bird 2013-03-30 01:14:37 UTC
I merged the revert to master - try it now.
Comment 7 Blackpaw 2013-03-30 08:08:39 UTC
Yup, problem gone now. Thanks.
Comment 8 David Faure 2013-03-30 10:17:50 UTC
OK I can reproduce this after restarting nepomuk (not just kmail, which is the only thing I tested before).

Good opportunity for trying Andreas' new dbus monitor (kde:scratch/ahartmetz/d-sel.git)
It shows this:
http://www.davidfaure.fr/2013/dbustimeout.png
25 million usec = 25 s = dbus timeout. The call to "sparqlQuery" (into the nepomukqueryservice) takes more than 25s so it times out. And meanwhile, the call made by the kmail messagelist, "watch", also going to the same process (under its alias "DataManagement"), has to wait for these 25s to pass.

Sounds to me like it only worked by change before: the queryservice was using a different dbus connection than the DataManagement, but only because one of them was using the main thread's dbus connection by mistake.

Sounds to me like the nepomukqueryservice should have its own separate dbus connection, if it's used for blocking calls that make soprano queries.
Comment 9 Hussam Al-Tayeb 2013-03-30 20:07:26 UTC
Why use kde functions when Qt has similar functions too? if they are missing anything, just contribute improvements to upstream Qt instead?
Comment 10 David Faure 2013-03-31 15:42:13 UTC
Long story: libdbus isn't threadsafe, so the only solution currently is to create one DBus connection per thread, which KDBusConnectionPool provides. Qt itself only has a basic singleton, which leads to crashes and races in libdbus if used from multiple threads. But Qt doesn't want to provide a per-thread-connection api (they just tell us to do that ourselves), because ideally at some point Qt will have its own DBus implementation (without libdbus) and then the single shared connection will be fine even in multiple threads.

I think the actual issue here is that one-connection-per-thread isn't enough if there are two very different services in that thread; we need one-connection-per-service, so that long-running sparql queries don't make other faster dbus calls wait. I'd like Vishesh to take a look though, I'm not sure about the overall design of the process that handles these services, and which threads are supposed to do what, etc.
Comment 11 Vishesh Handa 2013-04-01 08:31:06 UTC
(In reply to comment #8)
> It shows this:
> http://www.davidfaure.fr/2013/dbustimeout.png
> 25 million usec = 25 s = dbus timeout. The call to "sparqlQuery" (into the
> nepomukqueryservice) takes more than 25s so it times out. And meanwhile, the
> call made by the kmail messagelist, "watch", also going to the same process
> (under its alias "DataManagement"), has to wait for these 25s to pass.
> 
> Sounds to me like it only worked by change before: the queryservice was
> using a different dbus connection than the DataManagement, but only because
> one of them was using the main thread's dbus connection by mistake.
> 
> Sounds to me like the nepomukqueryservice should have its own separate dbus
> connection, if it's used for blocking calls that make soprano queries.

This is strange. Cause the query service calls are not blocking. I'll try to debug it using Andrea's tool.
Comment 12 Hussam Al-Tayeb 2014-09-19 21:10:49 UTC
because nepomuk was replaced by baloo