Version: (using Devel) Compiler: g++ 4.3 OS: Linux Installed from: Compiled sources nepomuk crashing after indexing for a while with not enough memory errors: The system has a total of 6GB physical RAM, home directory size 150GB [/opt/kde/bin/nepomukservicestub] ' ' is not a UTF8 or latin1 string [/opt/kde/bin/nepomukservicestub] ' s not a UTF8 or latin1 string [/opt/kde/bin/nepomukservicestub] ' ' is not a UTF8 or latin1 string [/opt/kde/bin/nepomukservicestub] ' ' is not a UTF8 or latin1 string [/opt/kde/bin/nepomukservicestub] ' ' is not a UTF8 or latin1 string [/opt/kde/bin/nepomukservicestub] ' ' is not a UTF8 or latin1 string [/opt/kde/bin/nepomukservicestub] nepomukstrigiservice(1311)/nepomuk (strigi service) FileSystemWatcher::Private::run: woke up nepomukstrigiservice(1311)/nepomuk (strigi service) FileSystemWatcher::Private::checkFolders: [/opt/kde/bin/nepomukservicestub] QDBusConnectionPrivate::connectSignal: received error from D-Bus server while connecting signal to QDBusAbstractInterface::_q_serviceOwnerChanged(QString,QString,QString): org.freedesktop.DBus.Error.NoMemory (Not enough memory) [/opt/kde/bin/nepomukservicestub] QDBusConnectionPrivate::connectSignal: received error from D-Bus server while connecting signal to QDBusAbstractInterface::_q_serviceOwnerChanged(QString,QString,QString): org.freedesktop.DBus.Error.NoMemory (Not enough memory) ps: kde-devel@dummy:~$ ps gaxu | grep -i nepomuk kde-devel 1185 0.0 0.3 328392 20912 ? Sl 13:21 0:00 kdeinit4: nepomukserver [kdeinit] kde-devel 1208 5.1 0.4 367612 25064 ? Sl 13:21 2:03 /opt/kde/bin/nepomukservicestub nepomukstorage kde-devel 1310 1.4 19.3 1353600 1186852 ? Sl 13:22 0:34 /opt/kde/bin/nepomukservicestub nepomukfilewatch kde-devel 1311 6.7 2.1 445656 133556 ? SNl 13:22 2:42 /opt/kde/bin/nepomukservicestub nepomukstrigiservice kde-devel 1312 1.3 0.2 173496 17860 ? S 13:22 0:31 /opt/kde/bin/nepomukservicestub nepomukmigration1 kde-devel 1313 1.6 0.3 173708 18428 ? S 13:22 0:38 /opt/kde/bin/nepomukservicestub nepomukremovablestorageservice kde-devel 1314 1.3 0.2 184072 18172 ? S 13:22 0:32 /opt/kde/bin/nepomukservicestub nepomukqueryservice kde-devel 1315 1.7 0.4 257952 27304 ? Sl 13:22 0:41 /opt/kde/bin/nepomukservicestub nepomukontologyloader kde-devel 3614 0.0 0.0 7532 1008 pts/3 S+ 14:02 0:00 grep -i nepomuk kde-devel@dummy:~$ $free total used free shared buffers cached Mem: 6122888 6051944 70944 0 190732 2671552 -/+ buffers/cache: 3189660 2933228 Swap: 11896092 0 11896092 Surprisingly after I took the above stats, it started indexing again, but the memory usage keeps increasing, see http://img23.imageshack.us/img23/7286/snapshot2p.jpg kde-devel@dummy:~$ ps gaxu | grep -i nepomuk kde-devel 1185 0.0 0.3 328392 19692 ? Sl 13:21 0:01 kdeinit4: nepomukserver [kdeinit] kde-devel 1208 5.7 0.3 367612 23484 ? Sl 13:21 2:57 /opt/kde/bin/nepomukservicestub nepomukstorage kde-devel 1310 1.6 28.8 1935004 1764356 ? Sl 13:22 0:49 /opt/kde/bin/nepomukservicestub nepomukfilewatch kde-devel 1311 7.8 2.7 483172 169944 ? SNl 13:22 4:00 /opt/kde/bin/nepomukservicestub nepomukstrigiservice kde-devel 1312 1.4 0.2 173496 17724 ? S 13:22 0:45 /opt/kde/bin/nepomukservicestub nepomukmigration1 kde-devel 1313 1.8 0.2 173708 18288 ? S 13:22 0:55 /opt/kde/bin/nepomukservicestub nepomukremovablestorageservice kde-devel 1314 1.5 0.2 184072 18008 ? S 13:22 0:46 /opt/kde/bin/nepomukservicestub nepomukqueryservice kde-devel 1315 1.8 0.4 257952 25432 ? Sl 13:22 0:56 /opt/kde/bin/nepomukservicestub nepomukontologyloader $free total used free shared buffers cached Mem: 6122888 6079116 43772 0 49172 2228744 -/+ buffers/cache: 3801200 2321688 Swap: 11896092 0 11896092 The memory usage just keeps increasing. Let me know if you need any more information
I should also add this is with today's trunk qt-copy: kde-qt latest stable branch svn root repo version:1073373 last change versions kdesupport:1073355 kdelibs:1073371 kdebase:1073372
this is the latest after letting it index for a while and memory usage is still increasing (one of the nepomukservicestub instances is using about 2.5GB of memory) kde-devel@dummy:~$ ps gaxu | grep -i nepomuk kde-devel 1185 0.0 0.3 328392 18468 ? Sl 13:21 0:01 kdeinit4: nepomukserver [kdeinit] kde-devel 1208 3.3 0.3 368832 22036 ? Sl 13:21 4:46 /opt/kde/bin/nepomukservicestub nepomukstorage kde-devel 1310 0.9 43.5 2838396 2667848 ? Sl 13:22 1:18 /opt/kde/bin/nepomukservicestub nepomukfilewatch kde-devel 1311 4.1 4.2 573080 257188 ? SNl 13:22 5:54 /opt/kde/bin/nepomukservicestub nepomukstrigiservice kde-devel 1312 0.8 0.2 173496 16740 ? S 13:22 1:12 /opt/kde/bin/nepomukservicestub nepomukmigration1 kde-devel 1313 1.0 0.2 173708 17096 ? S 13:22 1:29 /opt/kde/bin/nepomukservicestub nepomukremovablestorageservice kde-devel 1314 0.8 0.2 184072 17008 ? S 13:22 1:14 /opt/kde/bin/nepomukservicestub nepomukqueryservice kde-devel 1315 1.0 0.3 257952 23712 ? Sl 13:22 1:25 /opt/kde/bin/nepomukservicestub nepomukontologyloader kde-devel 8220 0.0 0.0 7532 1008 pts/6 S+ 15:44 0:00 grep -i nepomuk kde-devel@dummy:~$
In KDE 4.4 the memory leaking is resolved with the usage of Virtuoso for data storage.
ahem I am using virtuoso 5.0.12, I heard that 6.0 is not yet supported? is that what I need to update to?
No, I thought you were running KDE 4.3 and had the typical sesame2 java related memory problems. Now I realize that the filewatch service is the problem. I will have a look into that.
thanks, I also have updated bug #222336 to unconfirmed for now. Hopefully there is a fix for 4.4 cheers
Seb, with today's trunk (4.5) updated about 2hrs ago, I still see the same behaviour for nepomukfilewatch (memory usage) kde-devel@dummy:~$ ps guax | grep -i nepomuk kde-devel 26828 0.0 0.3 330152 20936 ? Sl Jan25 0:00 kdeinit4: nepomukserver [kdeinit] kde-devel 26838 4.9 0.4 369148 25404 ? Sl Jan25 4:55 /opt/kde/bin/nepomukservicestub nepomukstorage kde-devel 26860 1.2 0.2 185736 18068 ? S Jan25 1:11 /opt/kde/bin/nepomukservicestub nepomukqueryservice kde-devel 26861 1.2 30.3 2029476 1857996 ? Sl Jan25 1:12 /opt/kde/bin/nepomukservicestub nepomukfilewatch kde-devel 26862 5.4 4.1 561940 254864 ? SNl Jan25 5:21 /opt/kde/bin/nepomukservicestub nepomukstrigiservice kde-devel 26863 1.4 0.2 175408 18320 ? S Jan25 1:24 /opt/kde/bin/nepomukservicestub nepomukremovablestorageservice kde-devel 26864 1.1 0.3 177500 20188 ? S Jan25 1:09 /opt/kde/bin/nepomukservicestub nepomukontologyloader kde-devel 31760 0.0 0.0 7532 1000 pts/2 S+ 01:16 0:00 grep -i nepomuk kde-devel@dummy:~$ I really hope this gets fixed for final 4.4sc or disabled on default? otherwise there are going to be a lot of annoyed users complaining about memory usage. Let me know if you need any other info
I have also found that dbus-daemon memory usage increases when any of the nepomukservicestub increases in memory (trunk compiled about an hour ago) kde-devel@dummy:~$ ps guax | grep -i dbus kde-devel 833 0.0 0.0 7528 972 pts/2 S+ 15:57 0:00 grep -i dbus 108 3050 0.0 0.0 21960 1712 ? Ss 14:20 0:00 /bin/dbus-daemon --system kde-devel 31202 0.0 0.0 23824 736 pts/0 T 15:11 0:00 dbus-launch --sh-syntax --exit-with-session kde-devel 31203 1.5 4.1 276600 256252 ? Ss 15:11 0:42 //bin/dbus-daemon --fork --print-pid 6 --print-address 9 --session kde-devel 31290 0.0 0.3 104876 22048 ? S 15:11 0:01 /usr/bin/python /usr/bin/coherence -o use_dbus:yes -o controlpoint:yes kde-devel@dummy:~$ ps guax | grep -i nepomuk kde-devel 835 0.0 0.0 7528 924 pts/2 R+ 15:57 0:00 grep -i nepomuk kde-devel 31277 0.0 0.3 330072 21128 ? Sl 15:11 0:00 kdeinit4: nepomukserver [kdeinit] kde-devel 31296 5.1 0.4 330832 29864 ? Sl 15:11 2:20 /opt/kde/bin/nepomukservicestub nepomukstorage kde-devel 31308 1.1 0.2 185748 18316 ? S 15:11 0:32 /opt/kde/bin/nepomukservicestub nepomukqueryservice kde-devel 31309 1.3 0.3 175396 18560 ? S 15:11 0:37 /opt/kde/bin/nepomukservicestub nepomukremovablestorageservice kde-devel 31310 1.1 0.3 224644 22968 ? S 15:11 0:31 /opt/kde/bin/nepomukservicestub nepomukontologyloader kde-devel 31311 1.2 3.2 368796 200960 ? Sl 15:11 0:33 /opt/kde/bin/nepomukservicestub nepomukfilewatch kde-devel 31496 6.0 2.0 382628 128032 ? SNl 15:14 2:36 /opt/kde/bin/nepomukservicestub nepomukstrigiservice
forgot to mention that I have set from kcm memory usage for nepomuk to be 100MB
I am a bit lost here since I never experienced this problem with the filewatch service. So could you please do the following: * Stop the service: qdbus org.kde.NepomukServer /servicemanager org.kde.nepomuk.ServiceManager.stopService nepomukfilewatch * Then restart it in valgrind: valgrind --leak-check=full --log-file=<SOMELOGFILE> nepomukservicestub nepomukfilewatch * Let it run until the mem usage goes up again * Stop it like before * Send me the logfile: trueg@kde.org
today's trunk compiled about 1hr ago, and the memory usage is back in the GB range: kde-devel@dummy:~$ ps guax | grep -i nepomuk kde-devel 7581 0.0 0.3 330084 21036 ? Sl 14:22 0:00 kdeinit4: nepomukserver [kdeinit] kde-devel 7591 2.8 0.7 387096 47376 ? Sl 14:22 2:35 /opt/kde/bin/nepomukservicestub nepomukstorage kde-devel 7613 0.7 0.2 185896 18300 ? S 14:22 0:38 /opt/kde/bin/nepomukservicestub nepomukqueryservice kde-devel 7614 0.7 0.3 224672 22840 ? S 14:22 0:39 /opt/kde/bin/nepomukservicestub nepomukontologyloader kde-devel 7615 0.7 21.9 1511020 1342104 ? Sl 14:22 0:39 /opt/kde/bin/nepomukservicestub nepomukfilewatch kde-devel 7616 0.8 0.3 175600 18844 ? S 14:22 0:46 /opt/kde/bin/nepomukservicestub nepomukremovablestorageservice kde-devel 7617 3.4 2.4 459360 147760 ? SNl 14:22 3:06 /opt/kde/bin/nepomukservicestub nepomukstrigiservice kde-devel 13400 0.0 0.0 7532 1008 pts/2 S+ 15:53 0:00 grep -i nepomuk As for the debug patch you sent me, the result is mostly 0 or in rare cases its 2. If you have any other debug patches do let me know, I will patch and report back.
once the memory usage got too high (2gb) I ran the kcmshell4 kcm_nepomuk and disabled nepomuk semantic desktop and click apply. memory usage got back to normal and I re-enabled Nepomuk semantic desktop and clicked apply. Then I started getting the messages below. I am not sure if this is of any help. uk:/ctx/3ebb9df6-cf32-457b-a578-61e1a3fb8f77>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/8ee58346-bc77-438d-84fa-bc5f4b59f34c>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/627909db-e601-41e4-9406-239dc928f07f>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/69d2708a-e058-4179-b457-6c2f218f5e4f>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/4ce21f10-e9c1-4a56-b924-ff065a246d4b>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/388c81bc-bd37-44e2-a10f-dc3bebdbe2f4>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/22abf83a-47ed-4736-9dfb-be26a70e9d88>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/0fbb9e24-0b55-48aa-b803-2131feb9a442>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/49fb6f77-8c5d-40c0-8640-3a321e3126f2>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/cdf16504-adcc-4f24-a4cb-48f14ac57291>" [/opt/kde/bin/nepomukservicestub] removeAllStatements query: "clear graph <nepomuk:/ctx/a2eaf1e5-8cc1-4712-b771-ddc4c81185c6>"
please try the patch attached to #219687 and see if it fixes the memory leak.
Please try the patch attached to bug #219687 and see if it fixes the memory leak.
The memory leak has been resolved in the meantime.