Bug 293641

Happens to me too.

qdbus org.kde.nepomuk.services.nepomukqueryservice returns:

/
/nepomukqueryservice
/servicecontrol

KDE 4.8.00 OpenSuse

I did an "akonadictl restart" from a terminal (can see the output) and when I got this bug i can see this:

request for item 379156 succeeded 
posting retrieval request for item 379156  there are  1  queues and  0  items in mine 
request for item 379156 still pending - waiting 
processing retrieval request for item 379156  parts: ("RFC822")  of resource: "akonadi_imap_resource_2" 
akonadi_imap_resource_2(11025)/libakonadi Akonadi::ResourceBase::itemRetrieved: Item does not provide part "RFC822" 
continuing 

Repeting over and over and over

Archlinux KDE 4.8.0 here

I have the same problem; virtuoso-t has been utilizing 100% of one of my cores continuously since I upgraded to 4.8 when it came out. I even let it use 1GB of RAM in hopes that it would finish indexing faster that way. Needless to say, that didn't work.

This can be a duplicate of bug 289932

*** This bug has been marked as a duplicate of bug 289932 ***

Created attachment 69033 [details]
output of status('rhck');

The output of status('rhck') is what i get when I do like http://kdeatopensuse.wordpress.com/2011/11/09/debugging-nepomukvirtuosos-cpu-usage/ suggests.
Maybe this can help? I can post more of these outputs if needed?

Reopening, bug 289932 is fixed, but the bug manifested itself again today on 4.8.3

I'm seeing this as well. The indexer is idle, everything is indexed, no queries listed by "qdbus org.kde.nepomuk.services.nepomukqueryservice", yet virtuoso-t is still eating 20-50% CPU (which I believe translates to 40-100% of one core), occasionally spiking up to 70% or so. And nepomukservices isn't easing up either, at 10-20%.

I noticed this after the KDE upgrade last month (KDE 4.8.3 on Kubuntu 12.04 amd64), so I agree that it seems to be a regression.

To me this really seems like some database upgrade issue, incompatible old data, or something in that direction. I also had high virtuoso-t CPU load, all suggestions and the fixes of bug 289932 didn't really help. Then I got fed up with waiting for a fix and did what others suggested: I deleted .kde/share/apps/nepomuk/repository/main/data/virtuosobackend/. And since then everything is fine (well, at least it doesn't endlessly eat all CPU anymore;)

Confirmed. It made me disable that stuff again. The few advantages nepomuk and strigi brings is not worth the constant trouble with it.
On system startup virtuoso-t consumes 50% of CPU (ie. two cores fully used) and often it just randomly starts eating up 25% of CPU, which I just notice by the CPU fan running and remaining battery time going down rapidly.
It even does that on battery mode when the indexer is suspended. And when it hangs (ie. consumes 25% CPU) and then you disable it from the KCM, you can NOT kill the process at all. It just respawns and hangs again if you kill it. Only restart helps. This renders this entire "semantic desktop" useless.

Same here, after removing .kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/ cpu usage is back to normal.

Same here, KDE 4.8.3 on Kubuntu 12.04. No queries running according to "qdbus org.kde.nepomuk.services.nepomukqueryservice", still virtuoso-t using up one CPU core without anybody knowing what it's doing.

In 4.8.4 I've disabled "Enable Nepomuk Semantic Desktop", still there is virtuoso-t running and using one core. I also moved .kde/share/apps/nepomuk/repository/main/data/virtuosobackend away, without effect. 

qdbus org.kde.nepomuk.services.nepomukqueryservice says
"Service 'org.kde.nepomuk.services.nepomukqueryservice' does not exist."
 
This is very annoying...

I did a "complete message" search for ".mat" in one of my folders in Kontact (one with many thousands of messages, I can't tell you exactly right now because KMail is frozen), and hit this issue.

$ qdbus org.kde.nepomuk.services.nepomukqueryservice
/
/nepomukqueryservice
/nepomukqueryservice/query1110
/servicecontrol

$ qdbus org.kde.nepomuk.services.nepomukqueryservice /nepomukqueryservice/query1110 queryString
select distinct ?r ?reqProp1 (bif:concat(bif:search_excerpt(bif:vector('.mat'), ?v7),bif:search_excerpt(bif:vector('.mat'), ?v8),bif:search_excerpt(bif:vector('.mat'), ?v10),bif:search_excerpt(bif:vector('.mat'), ?v6))) as ?_n_f_t_m_ex_ where { { ?r <http://akonadi-project.org/ontologies/aneo#akonadiItemId> ?reqProp1 . ?r <http://www.semanticdesktop.org/ontologies/2007/01/19/nie#isPartOf> <nepomuk:/res/9bbe1c6a-fe40-4ed3-bb08-a9d98924e8ac> . { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#bcc> ?v2 . ?v2 <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#cc> ?v3 . ?v3 <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#to> ?v4 . ?v4 <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#hasEmailAddress> ?v5 . ?v5 <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#emailAddress> ?v6 . FILTER(bif:contains(?v6, "'.mat'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#messageSubject> ?v7 . FILTER(bif:contains(?v7, "'.mat'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#plainTextMessageContent> ?v8 . FILTER(bif:contains(?v8, "'.mat'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/01/19/nie#isPartOf> ?v9 . ?v9 <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#plainTextMessageContent> ?v10 . FILTER(bif:contains(?v10, "'.mat'")) . } . ?r a <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#Email> . } . ?r <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#userVisible> ?v1 . FILTER(?v1>0) . }

$ qdbus org.kde.nepomuk.services.nepomukqueryservice /nepomukqueryservice/query1110 close

$ qdbus org.kde.nepomuk.services.nepomukqueryservice
/
/nepomukqueryservice
/servicecontrol

But the CPU is still high. I'm about to reboot and see what happens.

I also suffer from Nepomuk/Virtuoso high CPU usage. There is no query (also Nepomuk in notification area shows that "File indexer is idle"). I have not yet found anything that may cause that - I have not had anything like that in few days, later I changed Nepomuk file indexer config (I have excluded one dir) and then Nepomuk started to take CPU after some time.

I am attaching output of status('rhck') command, gdb (for both virtuoso-t and nepomukservicestub) and xsession-errors. I hope they will help finding the source of bug.

I am using KDE 4.8.4 from Debian repository on Debian amd64. Verions:
#v+
$ dpkg -l |awk '/(akonadi|nepomuk)/ {print $2 " - " $3}'
akonadi-backend-mysql - 1.7.2-1
akonadi-server - 1.7.2-1
libakonadi-contact4 - 4:4.8.4-1
libakonadi-kabc4 - 4:4.8.4-1
libakonadi-kcal4 - 4:4.8.4-1
libakonadi-kde4 - 4:4.8.4-1
libakonadi-kmime4 - 4:4.8.4-1
libakonadiprotocolinternals1 - 1.7.2-1
libnepomuk4 - 4:4.8.3-2
libnepomukquery4a - 4:4.8.3-2
libnepomukutils4 - 4:4.8.3-2
#v-

Created attachment 72276 [details]
output of status('rhck')

Created attachment 72277 [details]
output of gdb's bt on virtuoso-t and nepomukservicestub

Created attachment 72278 [details]
akonadi/nepomuk related entries from xsession-errors

*** This bug has been confirmed by popular vote. ***

I have KDE 4.9 from kubuntu backports PPA and nepomukservicestub is consuming all available memory 1.5GB out of 2.0GB, makes the system totally unusable. Killing the servicestub just starts a new instance and the same procedure begins again, starting with some 50MB and then steadyly ever increasing until 1.5GB again.
Does not take 30hrs but rather 10 minutes.
Disabling Nepomuk Semantic Desktop in KDE settings "solves" the issue with the result of not being able to use CATEGORIES for contacts or calendar events any more.
I did delete all nepomuk system settings and the databases etc. Same procedure. No difference.
This is the most broken Nepomuk version ever.

4:4.9.0a-0ubuntu1~precise1~ppa1 32bit

(In reply to comment #21)
> I have KDE 4.9 from kubuntu backports PPA and nepomukservicestub is
> consuming all available memory 1.5GB out of 2.0GB, makes the system totally
> unusable. Killing the servicestub just starts a new instance and the same
> procedure begins again, starting with some 50MB and then steadyly ever
> increasing until 1.5GB again.

Could you please tell which of the nepomukservicestub's is causing it? A simple `ps aux | grep nepomuk` should do the trick.

> Does not take 30hrs but rather 10 minutes.
> Disabling Nepomuk Semantic Desktop in KDE settings "solves" the issue with
> the result of not being able to use CATEGORIES for contacts or calendar
> events any more.
> I did delete all nepomuk system settings and the databases etc. Same
> procedure. No difference.
> This is the most broken Nepomuk version ever.

Did you not encounter this problem with earlier Nepomuk versions? Also, next time, please open up a separate bug report. This one is for virtuoso consuming too much cpu. Not memory.

Thanks.
> 
> 4:4.9.0a-0ubuntu1~precise1~ppa1 32bit

> Could you please tell which of the nepomukservicestub's is causing it? A
> simple `ps aux | grep nepomuk` should do the trick.

ingo@ingo-MacBook:~/Documents$ ps aux | grep nepomuk
ingo      3126  0.0  0.8  94924 18104 ?        Sl   12:01   0:00 
/usr/bin/akonadi_nepomuk_feeder --identifier akonadi_nepomuk_feeder
ingo      3142  0.0  0.4  68672  9280 ?        Sl   12:01   0:00 
/usr/bin/nepomukserver
ingo      4106  0.3  1.0 165652 21624 ?        SNl  12:15   0:00 
/usr/bin/nepomukservicestub nepomukstorage
ingo      4121 94.9 21.7 527868 441088 ?       RNl  12:15   1:52 
/usr/bin/nepomukservicestub nepomukfilewatch
ingo      4122  0.0  0.8  79260 17116 ?        SN   12:15   0:00 
/usr/bin/nepomukservicestub nepomukbackupsync
ingo      4123  0.0  0.7  71116 15020 ?        SN   12:15   0:00 
/usr/bin/nepomukservicestub nepomukqueryservice
ingo      4149  0.0  0.0   4372   808 pts/1    S+   12:17   0:00 grep --
color=auto nepomuk
ingo@ingo-MacBook:~/Documents$ kill 4121

Looks like the filewatcher....
If I kill it, a new instance is launched automatically and the whole things 
starts from scratch.

> Did you not encounter this problem with earlier Nepomuk versions?

I had SIMILAR issues ( not that bad) in earlier versions. 4.8 was okay to use.

> Also, next
> time, please open up a separate bug report. This one is for virtuoso
> consuming too much cpu. Not memory.

I am sorry, I did not want to repeat the other issues. While 
nepomukservicestub is eating up all available (and unavailable) RAM, it is 
also using 100% CPU.
Which is not too good. A system with no ram and no cpu.... ;-)

Have now done the following "trick":
ingo@ingo-MacBook:~/Documents$ sudo chmod a-xr /usr/lib/kde4/nepomukfilewatch.so
ingo@ingo-MacBook:~/Documents$ ls -la /usr/lib/kde4/nepomukfilewatch.so
--w------- 1 root root 125304 Jul 31 22:43 /usr/lib/kde4/nepomukfilewatch.so

Result:

ingo@ingo-MacBook:~/Documents$ ps aux | grep nepomuk                                                                         
ingo      3126  0.0  0.8  94924 18104 ?        Sl   12:01   0:00 /usr/bin/akonadi_nepomuk_feeder --identifier akonadi_nepomuk_feeder
ingo      3142  0.0  0.4  68672  9336 ?        Sl   12:01   0:00 /usr/bin/nepomukserver
ingo      4276  1.1  1.0 116340 21644 ?        SNl  12:29   0:00 /usr/bin/nepomukservicestub nepomukstorage
ingo      4290  0.2  0.8  79260 17120 ?        SN   12:29   0:00 /usr/bin/nepomukservicestub nepomukbackupsync
ingo      4291  0.2  0.7  71116 15016 ?        SN   12:29   0:00 /usr/bin/nepomukservicestub nepomukqueryservice
ingo      4300  0.0  0.0   4372   804 pts/1    S+   12:30   0:00 grep --color=auto nepomuk

Yay, I have a working system again with CATEGORIES in KAddressbook. However, the category selection dialog show tickmarks for the number of categories, but no categories (like invisible text). Well, better than nothing....

(In reply to comment #24)
> Have now done the following "trick":
> ingo@ingo-MacBook:~/Documents$ sudo chmod a-xr
> /usr/lib/kde4/nepomukfilewatch.so
> ingo@ingo-MacBook:~/Documents$ ls -la /usr/lib/kde4/nepomukfilewatch.so
> --w------- 1 root root 125304 Jul 31 22:43 /usr/lib/kde4/nepomukfilewatch.so
> 
> Result:
> 
> ingo@ingo-MacBook:~/Documents$ ps aux | grep nepomuk                        
> 
> ingo      3126  0.0  0.8  94924 18104 ?        Sl   12:01   0:00
> /usr/bin/akonadi_nepomuk_feeder --identifier akonadi_nepomuk_feeder
> ingo      3142  0.0  0.4  68672  9336 ?        Sl   12:01   0:00
> /usr/bin/nepomukserver
> ingo      4276  1.1  1.0 116340 21644 ?        SNl  12:29   0:00
> /usr/bin/nepomukservicestub nepomukstorage
> ingo      4290  0.2  0.8  79260 17120 ?        SN   12:29   0:00
> /usr/bin/nepomukservicestub nepomukbackupsync
> ingo      4291  0.2  0.7  71116 15016 ?        SN   12:29   0:00
> /usr/bin/nepomukservicestub nepomukqueryservice
> ingo      4300  0.0  0.0   4372   804 pts/1    S+   12:30   0:00 grep
> --color=auto nepomuk
> 
> Yay, I have a working system again with CATEGORIES in KAddressbook. However,
> the category selection dialog show tickmarks for the number of categories,
> but no categories (like invisible text). Well, better than nothing....

There are easier ways to disable it. Anyway, please could you follow the instructions given in this bug report - https://bugs.kde.org/show_bug.cgi?id=304476

Thanks.

I got hit by it again today. This is what happened:
nepomukservicestub nepomukstorage was taking 200 MB of ram, so I killed it. After that akonadi_nepomuk_feeder started going crazy with the CPU, so I killed it, after that I had to kill kmail also, for the same reason. After that virtuoso-t went crazy with the CPU, still no queries. I'll try to reproduce that.

I am on Arch Linux with Kde 4.10.3 and this is still a problem. Everythings fine at first but after some uptime (24h +) I still get a constant 35 to 50% CPU usage by virtuoso-t ... 

Shouldn't this be fixed for a long time now? I mean we are talking about a so called "stable release", don't we ... I am a bit alarmed that the last entry here is over 6 month ago ... 

What is the status? Is there a confirnmed and working workaround? 

thx for reading, 
p.

Piedro, I experienced this problem in the recent past, but I think I found a workaround: I've noticed that the problem I'm experiencing is that the file .kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.db grows out of control (more than 1 GB), causing the i/o to go mad reading and writing to and from this file.

What that's happening, I simply logout from KDE and delete the file from the text-mode console. As soon as I re-login, the file is recreated with a reasonable size of 8MB.

Ok, it's an ugly workaround, but since I've performed this trick a couple of times, my system seems more performing and I didn't need to do this anymore.

thx, simone, for your hints... I tried that immediately. The result is not that great though: 

virtuoso-t now uses 15-20% of CPU but now
nepomukservicetub (2 pocesses) use another 25% cpu beetween them 

so all in all it's still around 50% of cpu (2 cores, 3GHZ) get consumed by this whole undelying stuff.  I will however try a few more times like you mentioned ...

I don't want to add to the discussion whether nepomuk/virtuoso/soprano/akonadi is necessary (4 buggy always running background processes to run below the desktop? really?). But if developers consider this essential, I really expect it to be fixed with a high priority - we are talking about a complete show stopper: How can I run a productive system (that is at least what I do, it's stable release, isn''t it?) if there are these workload peaks that prevent me from working? 

(If this is not so important, I'd like to ask the question why it has to be there in the first place...)

plz devs, comment on the status of this, 
thx for reading, 
piedro

Seems like I have to deal with the same behaviour of virtuoso-t. My soprano-virtuoso.db is over 3 GiB big! But I can't delete it without data loss, can I?

Hallo,

I got the virtuos-t cpu problem after I changed some multimedia things: so I changed the phonon-backend from gstreamer-backend to vlc-backend and installed some codecs and multimedia software.

Could some of these packages cause this problem:

libunique-3_0-0|3.0.2-6.1.2
gnac|0.2.4.1-2.1.2|
libechonest2_1|2.1.0-23.1|
libtomahawk-plugins|0.7.0-56.1|
libtomahawk0_7_0|tomahawk|0.7.0-56.1|tomahawk-kde|0.7.0-56.1|libvlccore5|2.0.6-127.11|libvlc5|2.0.6-127.11|vlc-noX|2.0.6-127.11|vlc-qt|2.0.6-127.11|vlc-codecs|2.0.6-127.11|
vlc|2.0.6-127.11|
vlc-aout-pulse|2.0.6-127.11|phonon-backend-vlc|0.6.0-1.5|libvlc-qt0_8|0.8.0-1.2|
libvlc-qt-widgets0_8|0.8.0-1.2|
tano|1.2.1-1.26
libgnome-media-profiles-3_0-0|3.0.0-7.1.2
libsidplay1|1.36.59-152.1.2
gstreamer-0_10-plugins-ugly
|audio-recorder|0.9.1-2.1.2
|kde3-krecord|1.16-268.1.2
|kdemultimedia3-CD|3.5.10.1-7.1.2
|kdemultimedia3-sound|3.5.10.1-7.1.2
|kdemultimedia3-video|3.5.10.1-7.1.2
|kdemultimedia3-extra|3.5.10.1-7.1.2
jakarta-commons-compress-1_1|1.1-1.51
apache-commons-parent|23-3.1.1|noarch|
jdom|1.1-19.1.1|noarch||repo-oss
jakarta-commons-lang|2.4-12.1.6
Mediathek|2.5.0-2.18|noarch|
Mediathek-doc|2.5.0-2.18
remove |Mediathek-doc|2.5.0-2.18|noarch
remove |Mediathek|2.5.0-2.18
libgstrtp-1_0-0|1.0.7-2.1
libgstsdp-1_0-0|1.0.7-2.1
libgstfft-1_0-0|1.0.7-2.1
gstreamer-0_10-plugins-good-extra|0.10.31-12.1
gstreamer-0_10-plugins-ugly-orig-addon|0.10.19-10.3|
gstreamer-plugins-ugly|1.0.7-2.1|
gstreamer-plugins-good|1.0.7-2.6
gstreamer-plugins-good-extra|1.0.7-2.6

I'm using openSuse12.2 with updated KDE 4.10.3 "release 563"

I am using KDE 4.10.5 on Arch (fully upgraded).  When starting my system everything is fine. 
After letting my system run overnight: - I get 96% CPU usage of the process virtuoso-t, though there is no indiation of anything being indexed at the time (indexing agent inactive) ... 

soprano-virtuso.db is 2.5Gb again! I don't nknow how often I deleted that file - it still growas out of control after some time ... 

plz fix, 
thx for reading, 
piedro

Hallo piedro,

For me helped:

1) disable email-Indexing
2) delete the .db file or directory of email indexing
3) kill the running virtuoso-t process

or in an another order. (I don't remember exact file names, but there was a description anywhere)

After this I could restart email-indexing without getting troubles.

Best regards,
Tobias

This is the correct order:

1) delete the .db file or directory of email indexing
2) disable email-Indexing
3) kill the running virtuoso-t process 

I hope this is helpful for you.

Thx for your help! 

First of all: 
I just logged out of KDE and logged in again, the high CPU usage has been gone (obviously it starts after a long runtime > 24h ...) 
... but: the .db file is now 2.6 GB 

My question is: 
Is the .db file supposed to stay small like someone mentioned only a few Mb? 
Or is this normal that the file grows to a few Gb if there is a lot of emails to index (20000+) ? 

Disabling email indexing seems no solution to me, I have a lot of emails and I need to search them, so indexing seems mandatory ... (or have you been suggesting to only disable indexing once and turn it on again after?) 

As said above: Now with 2.6 Gb of .db file the CPU usage is at less than 1% though I expect this to grow after several hours of uptime ... 

However we put it, there is still a strange bug in the akonadi/soprano/nepomuk subsystem. 
Though I really hoped that these matters have been fixed with the latest soprano improvements ... 

thx all for your time, 
p.

I've improved the email indexing scheduling quite a bit for this 4.11 release so this should hopefully be a lot better.

In terms of memory, you're stuck. That's not something we directly control, that's a virtuoso thing. However, it seems as though virtuoso 7 (recently released) has many improvements for reducing the database size, so that should help. But I haven't really tested virtuoso-7 that much.

I run on 4.11.1 in between but notice the same problems (since 4.11.1!)
Deletion of .kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/ did not help

I'm also experiencing this. virtuoso-t sometimes start to use expensively CPU for a period of couple minutes. strace shows that i's waiting on a futex:
 
top:                  
 2396 mkyral    39  19  713696 199316   2572 S  99.4  2.5  36:18.72 virtuoso-t  

$ strace -p 2396
Process 2396 attached
futex(0x156f444, FUTEX_WAIT_PRIVATE, 1387, NULL

Thank you for taking the time to file a bug report.

The Nepomuk project is no longer included in the KDE Software Compilation. With Plasma 5, we have replaced most of the underlying technology with Baloo and other components. Hopefully this will have addressed your concern.

We encourage you to try out Plasma 5 (+Baloo) and let us know if your problem persists.