Version: unspecified (using KDE 4.7.1) OS: Linux I have updated from KDE 4.7.0 to KDE 4.7.1 and now nepomukindexer processes overcome limits of processes per user. My ulimit settings: $ ulimit -u 500 I have to login as root kill all of the instances and do chmod -x on the corresponding binary to be able to stop nepomuk via system settings. Reproducible: Didn't try Steps to Reproduce: Login into the system and wait for 30min Actual Results: Everything becomes too slow. It's almost impossible to start any application. Expected Results: Nepomuk should use limited amount of processes for indexing. Ideally just one.
I can confirm this. Happened to me too, during initial indexing. A couple hundred nepomuk's processes (lots of zombies) and insane LA. http://img197.imageshack.us/img197/3553/nepomuk.png
Created attachment 63616 [details] Patch to kde-runtime/nepomuk The intend is to only use one. If more than one is spawned it is a bug. Since I could not reproduce the bug here yet I created a patch which should work around any scheduling bug. Please apply to kde-runtime/nepomuk and see if the problem is fixed.
Git commit 95b5269321074861cd848e4e4c22f3f7a3db8403 by Sebastian Trueg. Committed on 19/09/2011 at 10:18. Pushed by trueg into branch 'master'. Make sure we only run one instance of nepomukindexer at the time. This is more of a workaround/fail-safe in case some of the asnyc code is buggy and results in several instances of nepomukindexer. BUG: 281779 M +12 -4 services/fileindexer/indexscheduler.cpp M +4 -0 services/fileindexer/indexscheduler.h M +2 -0 services/fileindexer/nepomukindexer.h http://commits.kde.org/nepomuk-core/95b5269321074861cd848e4e4c22f3f7a3db8403
Git commit 5913d9ce6c2e804925b479e94d503bc05c9d1bf3 by Sebastian Trueg. Committed on 19/09/2011 at 10:26. Pushed by trueg into branch 'KDE/4.7'. Backport: Ensure that there is only one instance of nepomukindexer. CCBUG: 281779 M +12 -4 nepomuk/services/strigi/indexscheduler.cpp M +4 -0 nepomuk/services/strigi/indexscheduler.h M +2 -0 nepomuk/services/strigi/nepomukindexer.h http://commits.kde.org/kde-runtime/5913d9ce6c2e804925b479e94d503bc05c9d1bf3
Git commit db3d44a61129d608630ca48101e67a060117ef43 by Sebastian Trueg. Committed on 19/09/2011 at 10:26. Pushed by trueg into branch 'master'. Backport: Ensure that there is only one instance of nepomukindexer. CCBUG: 281779 M +12 -4 nepomuk/services/fileindexer/indexscheduler.cpp M +4 -0 nepomuk/services/fileindexer/indexscheduler.h M +2 -0 nepomuk/services/fileindexer/nepomukindexer.h http://commits.kde.org/kde-runtime/db3d44a61129d608630ca48101e67a060117ef43
*** Bug 282380 has been marked as a duplicate of this bug. ***
*** Bug 282440 has been marked as a duplicate of this bug. ***
I use Kubuntu 11.10, with kde-runtime 4:4.7.1-0ubuntu5 Today I started indexing through nepomuk, and after about an hour I had a few hundred nepomukindexer processes running on my machine. I stopped indexing in the control center and killed all nepomukindexer processes. After that I applied this patch to kderuntime (built and installed the new package), and then restarted indexing. I monitored the indexing in the control center for a while, and it seemed to go through files on my files system fairly fast. After a couple of hours of indexing, I noticed the fans on the machine started. ps aux | grep nepomukindexer told that I had two nepomukindexer processes running and top showed that virtuoso was using 135% of the cpu time. Also dbus-daemon and nepumukservices used some cpu time, as seen below. 1811 boerre 39 19 346m 291m 3028 S 142 14.6 631:19.14 virtuoso-t 1493 boerre 20 0 12972 4340 860 S 14 0.2 72:45.30 dbus-daemon 1785 boerre 39 19 494m 108m 4128 S 8 5.4 51:04.76 nepomukservices I stopped indexing in the control center again and killed both nepomukindexer processes. cpu usage went down and I started indexing again from the control center. After a minute or so ps aux again told I had two nepomukindexer processes running as shown below: ps ax | grep nepomukin 1051 ? SN 0:00 /usr/bin/nepomukindexer /home/boerre 1052 ? SN 0:00 /usr/bin/nepomukindexer /home/boerre/googleearth_5.1.3533.1731+0.5.6-1_i386.deb After about ten minutes the same command showed: 1052 ? SN 0:00 /usr/bin/nepomukindexer /home/boerre/googleearth_5.1.3533.1731+0.5.6-1_i386.deb 1233 ? SN 0:00 /usr/bin/nepomukindexer /home/boerre/nigo-13.pdf and top showed about the same result as above. I had a look at .xsession-errors and it had quite a few instances of this sentence at the end of the file: [/usr/bin/nepomukservicestub] "/usr/bin/nepomukservicestub(2106)" Soprano: "SQLExecDirect failed on query 'sparql select distinct ?r ?reqProp1 (bif:concat(bif:search_excerpt(bif:vector('Asbjørg'), ?v2))) as ?_n_f_t_m_ex_ where { { ?r <http://akonadi-project.org/ontologies/aneo#akonadiItemId> ?reqProp1 . ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#me' (iODBC Error: [OpenLink][Virtuoso iODBC Driver][Virtuoso Server]SQ074: Line 1: syntax error at '#')" Don't know if it is connected to the cpu usage of virtuoso and that there are two instances of nepomukindexer running, but I thought I'd mention it anyway. Since it seems that the indexing is hanging it is probably connected to this bug: https://bugs.kde.org/show_bug.cgi?id=276056 but since there are two nepomukindexer processes running I thought this bug was the best place to report these problems.
(In reply to comment #8) > I use Kubuntu 11.10, with kde-runtime 4:4.7.1-0ubuntu5 This bug has been fixed in 4.7.2. > I had a look at .xsession-errors and it had quite a few instances of this > sentence at the end of the file: > [/usr/bin/nepomukservicestub] "/usr/bin/nepomukservicestub(2106)" Soprano: > "SQLExecDirect failed on query 'sparql select distinct ?r ?reqProp1 > (bif:concat(bif:search_excerpt(bif:vector('Asbjørg'), ?v2))) as ?_n_f_t_m_ex_ > where { { ?r <http://akonadi-project.org/ontologies/aneo#akonadiItemId> > ?reqProp1 . ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#me' > (iODBC Error: [OpenLink][Virtuoso iODBC Driver][Virtuoso Server]SQ074: Line 1: > syntax error at '#')" This is unrelated to this bug and rather looks like an Akonadi bug to me. What would be interesting is to get my hands on the files that make nepomukindexer go crazy, ie. googleearth_5.1.3533.1731+0.5.6-1_i386.deb and nigo-13.pdf. However, this only if you are using libstreamanalyzer (strigi) 0.7.6 or later.
I've since encountered the same symptom - some 290 processes marked defunct, consuming in aggregate some 2GB of virtual (quick sum in my head). The search service desktop notifier was stalled. Using nepomuk controller, I stopped strigi indexer and semantic desktop, which cleared the defunct processes, then restarted both ... things seem fine after a couple of hours, indexing is chugging merrily away. This is a fully up to date gentoo (10.0 kde profile), nepomuk 4.7.3, with strigi 0.7.6, so I expect your patches are included. FWIW, this system is freshly upgraded to include nepomuk, so this behaviour manifested during the first time running. If the problem recurs, I'll try to determine what file(s) might have hung it up. If there are pointers in figuring this out, that'd be cool. Eg. is the desktop notifier likely to be accurate in this situation? Or is there a better log?