Bug 161380 - Nepomukservices process makes my CPU reach 65%
Summary: Nepomukservices process makes my CPU reach 65%
Status: RESOLVED FIXED
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: general (show other bugs)
Version: 4.1
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Sebastian Trueg
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-04-28 18:36 UTC by Anne-Marie Mahfouf
Modified: 2011-05-07 15:36 UTC (History)
14 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Anne-Marie Mahfouf 2008-04-28 18:36:39 UTC
Version:            (using Devel)
Installed from:    Compiled sources
OS:                Linux

A process called nepomukservices starts with my KDE and takes most of my CPU and I don't know what it is, what it does and how to manage it.I stopped it with ksysguard because when I killed it earlier then Dolphin crashed.
I don't want it. This seems a recent behaviour.

Also I added a component to the Product Nepomuk in this bug database so that people can actually issue bug reports. You might want to check the description of it.
Comment 1 Sebastian Trueg 2008-04-28 22:00:54 UTC
nepomukserver is not the problem and should not be killed. If CPU usage is high it is due to Strigi doing the initial indexing. The technology is new and thus, a proper GUI is missing (the plan is to have a silent popup which informs about the initial indexing and allows to postpone or cancel it).
Normally, Strigi should not be started by default. If that is the case, we have a bug and I will look into it.
In any case systemsettings provide a Nepomuk configuration GUI in the advanced section. Here you can disable Strigi.
Comment 2 Kishore 2008-05-10 18:19:20 UTC
I have the same issue. Strigi desktop search is disabled but automatically gets launched with nepomukserver and causes nepomukservices to become CPU heavy.
Comment 3 Marcin Juszkiewicz 2008-05-20 15:02:00 UTC
14429 hrw       20   0  225M 28344 13960 R 98.4  0.7  2:26.03 /usr/bin/nepomukservicestub nepomukstorage

98% of CPU use (according to htop) is on my system when nepomuk and strigi are enabled. 

Nepomuk 4.0.74 from Debian/experimental
Strigi daemon 0.5.9 from Debian/unstable
Comment 4 Sebastian Trueg 2008-05-23 16:50:02 UTC
that is because strigi does its initial indexing which takes a while and makes heavy use of the nepomuk storage service. I know that this is far from optimal and am working on optimizations.
Comment 5 Valerio Pilo 2008-05-24 11:01:08 UTC
It's very CPU intensive on my Core2duo; runs at an average of 50% usage. It was enabled by default on first run, but I highly doubt it's still doing that 'initial indexing' since it's sucking off 50% of my cpu since 4 days, and I only have some 14gb of data in my home, and i've excluded a lot of it.
Comment 6 Sebastian Trueg 2008-07-07 11:27:42 UTC
well, the initial indexing is slow. Sorry.
could you please check if the output of "sopranocmd --help" contains "sesame2". If that is not the case there is another bottleneck: the storage backend uses redland instead of sesame2 which is very slow.
Comment 7 Rafael Rodriguez 2008-07-22 01:45:31 UTC
Any way we can change the backend from redland to sesame2?
Comment 8 Sebastian Trueg 2008-07-22 12:17:07 UTC
once the sesame2 backend is installed, it is used automatically. Existing data is converted.
Comment 9 Raif Sarcich 2008-07-28 11:00:54 UTC
Every time KDE starts up, "nepomukservices" starts drawing the output of 1 CPU core. If I kill it, strigidaemon starts indexing, but it never stops and never seems to do anything, strigi-client says that 0 files have been indexed. Every now and again nepomukservices comes back to life and strigidaemon stops until I kill it. Using the Kubuntu Hardy rc1 packages. 
Comment 10 Sebastian Trueg 2008-07-28 11:37:54 UTC
Raif: if you kill "nepomukservices nepomukstorage" then strigi has no place where to store the data anymore. initial indexing takes time and cpu. no way around that except to disable strigi. You can have a look at the "nepomuk" module in kde's systemsettings.
Comment 11 Szczepan Hołyszewski 2008-07-29 13:55:20 UTC
Sebastian, please, did you read the comments above carefully enough to notice that for at least one user the "initial indexing" of a mere 14GB of data took at FOUR DAYS if not more? "Initial indexing is slow, sorry" is not an answer. This performance is problematic. It ***IS*** a bug.

Here's my experience: nepomukserver runs 4 instances of nepomukservices, 3 of which just sit there and do nothing, and the fourth eats about 50% CPU (which is probably the KDE4 System Monitor's way of saying that it uses about 100% of one core on a dual core system), PLUS it consumes 800+ MB RAM (and slowly growing) on a 2G system. Either these ***ARE*** bugs, or nepomuk is a bad design to begin with.
Comment 12 Sebastian Trueg 2008-07-29 14:21:10 UTC
Regarding the mem usage: this is due to the java vm which is used. This is bad 
but atm there is no better solution.

As for the initial indexing taking four days: I said that this happens with 
the redland backend. The one not using java. The redland backend is slow. 
That is why there is no better solution than the java one atm.

The two services "just sitting" there are used less frequently than the 
others.

So the only real problem is the mem usage. And that I was not able to solve 
until now. Sorry. You can always disable Nepomuk if that troubles you. Or 
help in improving it. But ranting never helped anyone. ;)
Comment 13 Rafał Rzepecki 2008-07-30 16:16:11 UTC
Could any of you see if the bug pertains in the current trunk? Fixing bug #167682 fixed this for me, it turned out that I had looping directory structure and nepomuk (well, KDirWatch really) followed it without end. This also led to a problem with gamin, so that nepomuk & gamin saturated the CPU and RAM.
Comment 14 Florent Morselli 2008-08-11 18:38:54 UTC
I have the same problem on my laptop.
Strigi does its initial indexing and the CPU usage decreased. Now its all right.

But can we limit CPU usage when nepomuk or Strigi do their indexing when the computer is a laptop on batteries? 
Comment 15 Sebastian Trueg 2008-08-11 20:53:39 UTC
> But can we limit CPU usage when nepomuk or Strigi do their indexing when
> the computer is a laptop on batteries?


the new strigi service will suspend indexing completely when on batteries once 
Solid has laptop support (currently being implemented AFAIK)
Comment 16 Nick Shaforostoff 2008-08-21 11:05:56 UTC
+1. very annoying and unneeded, especially on a 8gb solid-state drive.
Comment 17 Sebastian Trueg 2008-08-21 11:43:27 UTC
Nick: +1 for what?
Comment 18 Nick Shaforostoff 2008-08-21 11:48:13 UTC
for original poster. it would be cool if nepomuk detected such (ssd, or just small-capacity) systems and turned itself off
Comment 19 busytoby 2008-08-22 17:57:46 UTC
System is Debian Unstable pulling KDE apps from experimental.
Strigi version:  0.5.11-1

overall system uptime is currently at 18 days and the current X session has been running the entire time with nepomuk/strigi consuming 80-100% of a single 2.6ghz core.  Strigi is set to only index my home directory which contains a total of 5.5gb of data (primarily source code and pdfs) using the redline backend.

This seems totally unacceptable as this renders strigi essentially useless.  Is there any purpose of having the redland backend enabled at all?  Perhaps strigi should simply disable itself while sesame2 is not available for use.
Comment 20 George Kiagiadakis 2008-09-23 11:06:44 UTC
SVN commit 852681 by trueg:

- Added storage interface decsription
- Make sure the DBus service control interface is registered before the service is \
created.  Also make sure we already have an event loop at that point. This is \
important since the  server needs to catch the serviceInitialized signal.
- Do not start the strigi service if redland is in use.

BUG: 161380


 M  +1 -0      interfaces/CMakeLists.txt  
 A             interfaces/org.kde.nepomuk.Storage.xml  
 M  +23 -11    server/servicecontroller.cpp  
 M  +2 -0      services/storage/storage.cpp  
 M  +3 -0      services/strigi/CMakeLists.txt  
 M  +28 -16    services/strigi/strigiservice.cpp  
 M  +9 -42     servicestub/main.cpp  
 M  +55 -2     servicestub/servicecontrol.cpp  
 M  +16 -3     servicestub/servicecontrol.h  
Comment 21 Georg Grabler 2008-12-23 14:01:46 UTC
I don't know if it's really fixed.. at least with beta2 of kde4.2


(Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/sesame2backend.desktop"                                                                                      
(Soprano::PluginManager) plugin has proper version.

--backend    The backend to use when accessing a storage directly and not via the Soprano server.
             Possible backends are:                                                              
                       redland, sesame2


I'm just indexing my /home directory, running df -h:
/dev/sda4              59G   11G   46G  19% /home

The indexing now already takes 3 days... that's quite a long time for 11 gb of indexing.
Comment 22 Sebastian Trueg 2008-12-24 09:53:58 UTC
Do you have many archives (tar,zip,rpm,...) in the indexed files? That could be the reason also since Strigi does index all files in the archives which takes a long time. Strigi from svn does contain a hack to prevent this which is used by Nepomuk if compiled against that version of Strigi.
Final KDE 4.2 should be shipped with this "new" version of Strigi.
Comment 23 Georg Grabler 2008-12-30 16:50:19 UTC
Not too many in my home folder, maybe 20-30 packages (overall probably 200-300 MB). I got a lot of source code flying around there (so a lot of small files), including the kdelibs and qt source and the wine source.

Does stigri index . folders? like .svn? That could also be the reason, since I have some huge repositories here at work
[stiat@arch ~]$ find . |wc -l
288688

[stiat@arch ~]$ find . |grep -v .svn |wc -l
190679

I don't think that's uncommon on developer workboxes, but maybe I should help stigri excluding those development folders.
Comment 24 Mirek Mieszczak 2009-02-19 06:48:33 UTC
It seems, that nepomuk get into infinite loop on empty folder, at least in my system. In my case it is ~/smb4k.
Comment 25 Robbert Korving 2011-05-07 15:36:24 UTC
This is back in Kubuntu 11.04. First I though it was the ati drivers but now when I connect my external drive. Nepomuservices and kded4 both go to 50% cpu cycles making impossible to work with my system anymore. Even when save  removing my external drive. It jus keeps eating cycles.