Thanks for the bug report.

Were you moving a number of files or directories during that time? Or were a lot of files being created or deleted?

Also, if possible please switch off the filewatch service via - 
$ qdbus org.kde.nepomuk.services.nepomukfilewatch /servicecontrol shutdown

and then run it again via $ nepomukservicestub nepomukfilewatch. Please attach the entire debug output. Also make sure you have nepomuk debug output enabled via kdebugdialog.

(In reply to comment #1)
> Were you moving a number of files or directories during that time? Or were a
> lot of files being created or deleted?
No, doubtful. The system was idling the whole time. It even happens right now, while I have not more than this browser window open…

> Also, if possible please switch off the filewatch service via - 
> $ qdbus org.kde.nepomuk.services.nepomukfilewatch /servicecontrol shutdown
> 
> and then run it again via $ nepomukservicestub nepomukfilewatch. Please
> attach the entire debug output. Also make sure you have nepomuk debug output
> enabled via kdebugdialog.
Will do that in a couple of days, when I have the time. (Same goes for the isql-vt->status() output you asked me for in the other report.)

Alright. Let me know.

*** Bug 304523 has been marked as a duplicate of this bug. ***

Please could you guys test the following patch - https://git.reviewboard.kde.org/r/105892/

It should fix the issue, but I'd like a confirmation before committing it. Specially since I cannot reproduce the issue.

(In reply to comment #5)
> Please could you guys test the following patch
I applied your patch and the memleak appears to be gone (htop shows no increase in memory usage and no process running at 100% cputime).

Below is the output of nepomukfilewatch AFTER applying the patch:
nepomukfilewatch(25714)/kdecore (KSycoca) KSycocaPrivate::openDatabase: Trying to open ksycoca from "/var/tmp/kdecache-USER/ksycoca4"
nepomukfilewatch(25714)/nepomuk (library) {anonymous}::GlobalModelContainer::init: Connecting to local socket "/tmp/.private/dschridde/ksocket-USER/nepomuk-socket"
nepomukfilewatch(25714)/nepomuk (filewatch service) Nepomuk2::FileWatch::watchFolder: "/home/USER"
nepomukfilewatch(25714)/nepomuk (filewatch service) KInotify::addWatch: "/home/USER"
nepomukfilewatch(25714)/nepomuk (filewatch service) KInotify::Private::open:
nepomukfilewatch(25714)/nepomuk (filewatch service) KInotify::Private::open: Successfully opened connection to inotify: 14
nepomukfilewatch(25714)/nepomuk (filewatch service) Nepomuk2::InvalidFileResourceCleaner::run: Searching for invalid local file entries
nepomukfilewatch(25714)/nepomuk (filewatch service) Nepomuk2::InvalidFileResourceCleaner::run: Done searching for invalid local file entries
nepomukfilewatch(25714)/nepomuk (filewatch service) KInotify::Private::_k_addWatches: Done adding watches

(In reply to comment #6)
> (In reply to comment #5)
> > Please could you guys test the following patch
> I applied your patch and the memleak appears to be gone (htop shows no
> increase in memory usage and no process running at 100% cputime).
> 

Thanks a lot. I'll commit the patch tomorrow. It'll be there in 4.9.1

I wish this had been caught during the beta or RC1 testing.

Thanks for your quick response! I pushed this into Gentoo as kde-base/nepomuk-core-4.9.1-r1 [1] (kde-overlay, soon to arrive in mainline, I hope).

[1] https://bugs.gentoo.org/show_bug.cgi?id=430226

Git commit a81f05034baef438eacd346fcbc97dd818a0b5bf by Vishesh Handa.
Committed on 06/08/2012 at 11:31.
Pushed by vhanda into branch 'KDE/4.9'.

kinotify: Do not store the paths to be added

kinotify used a QQueue<QByteArray> to store all the directories which
need to be added. Since we use QDirIterator, each path is presented as a
QString which is then encoded to its QByteArray (QFile::encodeName).
This results in *large* chunks of memory being allocated, and then
slowly being deallocated.

Instead, we now use a QDirIterator, and do not store all the directories
which need to be added, we simply iterate over them. This way we do not
allocate large amounts of memory.

There is a large performance improvement as well. On my system, with
38829 directories, adding all the watches now takes only about 10 seconds,
instead of about 65.

Patch possible due to massif output provided by Jure Repinc <jlp@holodeck1.com>.
Thanks a lot.

tldr: Use DFS instead of BFS -> Less memory consumption
REVIEW: 105892
DIGEST: Fix massive memory leak in Nepomuk File Monitoring Service

M  +14   -24   services/filewatch/kinotify.cpp

http://commits.kde.org/nepomuk-core/a81f05034baef438eacd346fcbc97dd818a0b5bf

Btw, in case someone is backporting this. Please either use the patch on the review request on the combination of the commit above and this one -

commit 7322b28f46969adb721daecf5a7fb67b8e0c6ae4
Author: Vishesh Handa <me@vhanda.in>
Date:   Wed Aug 8 14:33:02 2012 +0530

    Break out of kinotify's _k_addWatches loop
    
    Didn't test the patch out after making a super trivial change in the
    review request. Stupid Stupid me!

diff --git a/services/filewatch/kinotify.cpp b/services/filewatch/kinotify.cpp
index 47eb8ed..509abff 100644
--- a/services/filewatch/kinotify.cpp
+++ b/services/filewatch/kinotify.cpp
@@ -152,6 +152,7 @@ public:
             }
             else {
                 delete dirIterators.dequeue();
+                break;
             }
         }