Bug 162932

Summary: nepomukservices process blocks dolphin
Product: nepomuk Reporter: S. Burmeister <sven.burmeister>
Component: generalAssignee: Sebastian Trueg <trueg>
Status: RESOLVED FIXED    
Severity: normal CC: sebastian, stephan.menzel
Priority: NOR    
Version: 4.1   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: Deliver KDirNotify events asyncroneously to avoid blocking of emitting apps like Dolphin

Description S. Burmeister 2008-05-31 03:39:24 UTC
Version:            (using Devel)
Installed from:    Compiled sources
OS:                Linux

If nepomuk is enabled and I open and try to scroll in dolphin via the mouse-wheel, it is blocked for some seconds in which the nepomukservices process eats 100% of one core and only after it finished doing that dolphin starts scrolling.

If nepomukservices is not quick enough, dolphin's GUI should not wait for it but have an asynchonous way of communicating with it.
Comment 1 Peter Penz 2008-06-03 21:53:38 UTC
Sven, might it be possible that this is not related to Nepomuk and is just a form of bug #163050? I've turned Nepomuk + Strigi in my environment but cannot reproduce this issue...

But as soon as I hover huge files and the Information Panel is enabled, then generally Dolphin is blocked because of #163050 even when Nepomuk + Strigi is turned off.

Could you please turn off Nepomuk + Strigi temporary and let me know if the issue still happens? Thanks!
Comment 2 S. Burmeister 2008-06-06 19:46:51 UTC
> But as soon as I hover huge files and the Information Panel is enabled,
> then generally Dolphin is blocked because of #163050 even when Nepomuk +
> Strigi is turned off.


I do not hover files, I just scroll the filelist (details-view) with the 
mouse-wheel or by pressing the middle mouse-button and sliding the mouse.

> Could you please turn off Nepomuk + Strigi temporary and let me know if the
> issue still happens? Thanks!


It does not happen.

If I enable nepomuk and strigi during a session and start dolphin (with 
information bar), it takes very long and nepomukservice's CPU usage goes up 
during that time. After it falls back to 0% dolphin's window opens.

If I have the information-bar turned on and enter a new folder or change to 
the external harddisk attached via firewire, dolphin is blocked and 
nepomukservice uses CPU, if it falls back to 0% dolphin shows the folder. This 
variant of the issue does not happen when the bar is turned off.
Comment 3 Stefano 2008-06-17 10:10:01 UTC
That's true for me too.

Actually any nepomuk retreival usage (with nepomuksearchclient too, for example, or with nepomuksearch:/ protocol) cause this from time to time (let's say 80% of the times).

In dolphin (yesterday's SVN), it happens everytime you type for example "nepomuksearch:/tag:Amici" (an existing tag I have): CPU goes 100% due to nepomuk storage service, for let's say 30 seconds or so, then it calms down and no results are found anyway.
Comment 4 Peter Penz 2008-06-17 12:10:47 UTC
@Sebastian: I've reassigned this issue to Nepomuk, please let me know if I can support you. Thanks!
Comment 5 Sebastian Trueg 2008-06-26 13:10:19 UTC
Does this also happen if strigi is disabled?
Comment 6 Clovis Gladstone 2008-07-03 08:00:14 UTC
Yes it does on my system. I'm using 4.00.84. It seems to me that bug 163604 and Bug 164309 describe the same problem : high cpu usage of nepomukserver when starting dolphin, or when opening a new folder.
Comment 7 Sebastian Trueg 2008-07-03 10:11:10 UTC
May I ask you to check if the output of "sopranocmd --help" contains "sesame2".
If not, it meant that Nepomuk is using Redland as a database backend which is very very very slow. Since Sesame2 is based on java, it is not as easily enabled as redland.
Comment 8 S. Burmeister 2008-07-12 20:02:39 UTC
I don't see sesame2 in the output of soprano. I just noticed that while dolphin is blocked, the process list also displays a lot of zombied pdftotext processesm so I guess something is searching the content of the file I selected.

Strigi is not running, and everything within the nepomuk_kcm is disabled.
Comment 9 Sebastian Trueg 2008-07-14 09:21:24 UTC
> ------- I don't see sesame2 in the output of soprano. I just noticed that


If you dont have sesame2 you only have redland which makes Nepomuk very slow, 
but...

> while dolphin is blocked, the process list also displays a lot of zombied
> pdftotext processesm so I guess something is searching the content of the
> file I selected.


Don't know about that. No Nepomuk here.

> Strigi is not running, and everything within the nepomuk_kcm is disabled.


... Nepomuk is disabled anyway, so it cannot be the cause of the problem. But 
initially you wrote that nepomuk was enabled.
Comment 10 S. Burmeister 2008-07-14 09:52:43 UTC
> ... Nepomuk is disabled anyway, so it cannot be the cause of the problem.
> But initially you wrote that nepomuk was enabled.


Yes, I did not remember my own report, sorry. This one is about scrolling and 
not hovering, thus not related.
Comment 11 Stephan Menzel 2008-10-15 15:14:42 UTC
I have this problem too and to me it looks like it is rather independent from strigi. I have nepomuk enabled and stringi disabled. When I enable strigi it just eats up 100% CPU and lots of disk IO. Googleing for the problem I read it is doing initial index which would be OK but after 3 days of continous indexing fun I figure that's enough of that. Also I straced it and noticed it is running over the same directories again and again. strigiclient tells me there's nothing at all in the index too. So I had to consider it broken and turned it off. But i tried to play around with nepomuks tagging and think it's useful if it weren't for this bug.

I straced this one too and during the time dolphin is stalled the nepomuk process seems to read and parse a large file which I presume is the index. So that happens when the tags for one entry are accessed. Unfortunately it happens again when the next entries are accessed and so on. The index appears to be not persistent in mem or somewhere and so it is read and parsed again and again. At least this is what it looks like.

It is OpenSUSE 64 bit Factory KDE 4.1.2 here. 

HTH,

Stephan
Comment 12 Chad Martin 2008-11-08 05:27:14 UTC
I'm experiencing this problem as well. It happenned after upgrading to Kubuntu Intrepid. I tried a complete reinstall and the problem still persisted. After experimenting around the stalls only happen with nepomukservices enabled. Having Strigi disabled doesn't help.

After reading the information bar comments above I tried closing it and the slow performance was only with it active.

Redland is the only backend listed by sopranocmd --help.
Comment 13 S. Burmeister 2009-01-12 15:10:41 UTC
I think this can ahppen if you delete a large number of files, which were part of nepomuks index. Dolphin stalled for me after that, I assume because nepomukservices was busy. After a while, it continued to work as usual.
Comment 14 Sebastian Trueg 2009-01-13 20:20:20 UTC
Created attachment 30221 [details]
Deliver KDirNotify events asyncroneously to avoid blocking of emitting apps like Dolphin

Please test this patch. It should resolve the issue. However, annotations such as tags might get updated with a little delay.
Comment 15 S. Burmeister 2009-01-13 20:44:43 UTC
Can we test it with KDE 4.2 or only trunk?
Comment 16 S. Burmeister 2009-01-13 22:50:52 UTC
I tried to patch, i.e. recompiled filewatch and installed only those files. Logged out and back in. After that I moved a lot of folders with mp3s in them and if I'm not mistaken the blocking never gets beyond 2 seconds, mostly 1, which is better than before.
Comment 17 Sebastian Trueg 2009-01-14 11:34:19 UTC
SVN commit 910807 by trueg:

handle events asyncroneously

BUG: 162932


 M  +7 -2      nepomukfilewatch.cpp  
 M  +5 -0      nepomukfilewatch.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=910807
Comment 18 Sebastian Trueg 2009-01-19 13:39:32 UTC
SVN commit 913439 by trueg:

Handle all metadata moving in a separate thread. This keeps the main thread accessible and
avoids blocking clients such as Dolphin.

BUG: 162932


 M  +1 -0      CMakeLists.txt  
 A             metadatamover.cpp   [License: LGPL (v2)]
 A             metadatamover.h   [License: LGPL (v2)]
 M  +24 -175   nepomukfilewatch.cpp  
 M  +7 -8      nepomukfilewatch.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=913439