Version: unspecified (using Devel) OS: Linux When I try to index a file with nepomukindexer, the memory usage of both nepomukstorage and Virtuoso grows. 1. The growth is exponencial and depends on how high is the Virtuoso memory limit. In two tests, a limit of 50 MB yielded 630 MB of memory usage with unfinished indexing, and a limit of 160 MB yielded 1.5 GB of memory usage with unfinished indexing stopped at the same point. It's easy to see why I cancelled indexing; this had the potential to OOM my system. 2. Both nepomukstorage and Virtuoso grow. When stopped, nepomukstorage used 220 MB of memory itself. 3. This is manifested with nepomukstrigiservice and bangarangnepomukwriter. I can't test akonadi_nepomuk_email_feeder because of the in-built restrictions of that indexer. 4. The leak is the same if I use Virtuoso 6.1.2 or 6.1.3. I haven't tested 6.1.3 snapshot. 5. This mixes itself with another bug: Strigi being unable to remember what was indexed. This bug mixed with that one make a powerful combo that can OOM any system with Strigi indexing enabled and lots of files. 6. The leak doesn't manifest when the files themselves are being indexed, but only when nepomukstorage writes the changes. 7. The memory limits are relatively ignored. Higher the limit, higher the leak. Please, give me instructions of how to test a memory leak. This is the new Nepomuk code (Vishesh Handa backports, KDE 4.7.0 try2), so, let's be swift to kill this bug once and for all. Reproducible: Always Steps to Reproduce: 1. Install KDE 4.7.0 with the new Nepomuk code. 2. Start Strigi indexing. 3. See the following behaviour: - Files are indexed - normal memory usage, normal nepomukindexer system load. - nepomukstorage writes the changes. Virtuoso memory usage grows in several megabytes, nepomukstorage memory usage grows a hundred of kilobytes. - Repeat. Actual Results: System being progressively OOM'd. Expected Results: Trouble free indexing, memory limits being respected.
I'm closing this bug with a BIG AND FAT WARNING TO DISTRIBUTORS. You'll face this memory leak if you don't ship KDE 4.7 with Raptor 2.0.4 and Soprano 2.6.52 COMPILED AGAINST RAPTOR 2.0.4, the new Rasqal, and the new Redland. With the full new stack, there is no memory leak. I'm closing this as RESOLVED/REMIND, since this bug should be a note in your KDE 4.7 Release Notes. Otherwise, users of ill packaged KDE repositories will face massive memory leaks.
*facepalm* :/ I've been running valgrind like crazy trying to find the memory leak. I'm so happy it was just a packaging issue. I think there is still a minor one if virtuoso. Thanks a lot Alejando.
Yes, there's a minor leak. After a full reindexing, Virtuoso is using 150 MB. Thanks for caring about this, Vishesh. Please, put this warning in the KDE Release Notes (I don't want anybody to face this, so better safe than sorry ;))
*** Bug 279124 has been marked as a duplicate of this bug. ***
Hi I have just updated to kubuntu 11.10 and is experiencing massive memory leeks ( at least the processes uses massive amount of memory ) top - 08:52:44 up 23:16, 19 users, load average: 0.74, 0.81, 0.39 Tasks: 172 total, 1 running, 171 sleeping, 0 stopped, 0 zombie Cpu(s): 5.9%us, 2.3%sy, 0.0%ni, 89.8%id, 2.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 5776564k total, 5631620k used, 144944k free, 5332k buffers Swap: 6423548k total, 3712428k used, 2711120k free, 407820k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5235 vl 39 19 3906m 2.9g 3124 S 0.0 53.3 53:54.59 nepomukservices 6334 vl 39 19 568m 403m 2440 S 0.0 7.1 256:44.33 virtuoso-t 6871 vl 20 0 2161m 284m 14m S 0.0 5.0 18:11.67 kmail 8609 vl 20 0 1067m 268m 9628 S 3.0 4.8 33:03.06 firefox 5032 vl 20 0 1873m 186m 700 S 0.0 3.3 26:37.42 dbus-daemon kubuntu 11.10 ships with: kdebase-bin 4:4.7.1-0ubuntu3 libraptor1 1.4.21-5 libraptor2-0 2.0.4-1 libsoprano4 2.7.0+dfsg.1-0ubuntu2 soprano-daemon 2.7.0+dfsg.1-0ubuntu2 Any easy way I can find out if this is the same problem as described here, or if it is new different bug ? Best regards, Vidar
(In reply to comment #5) > I have just updated to kubuntu 11.10 and is experiencing massive memory leeks ( > at least the processes uses massive amount of memory ) please check which process it is exactly via: ps aux|grep nepomukservicestub It is very likely that you are experiencing bug 226676 instead.
thanx for your commment Sebastian. I had to reboot my computer in order to make it useable again. The huge memory leak has not occurred again during the last 7 days so I am unable to tell what process exactly caused the problem. I have however disabled the "Nepomuk Semantic Desktop" as it rendered the system completely unusable due to it's massive cpu/disk usage