Bug 279800

Summary:	akonadi_nepomuk_email_feeder uses huge amounts of memory
Product:	[Frameworks and Libraries] Akonadi	Reporter:	Todd <toddrme2178>
Component:	Nepomuk Feeder Agents	Assignee:	Tobias Koenig <tokoe>
Status:	RESOLVED FIXED
Severity:	normal	CC:	christian.fontana, frederic.coiffier, ismail, kdepim-bugs, kfunk, markus.blaschke, sven, t.zell, tim, vkrause
Priority:	NOR
Version:	4.7
Target Milestone:	---
Platform:	openSUSE
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:

Description Todd 2011-08-10 10:15:30 UTC

Version: 4.7 (using KDE 4.7.0)
OS: Linux

Whenever I use akonadi, the process akonadi_nepomuk_email_feeder uses a massive amount of memory, usually between 1 and 3 GB. This leave the system almost totally unresponsive due constant disk access. I can kill the process, but it just comes back again. Upon startup or the process restarting it uses GB of ram within seconds.

I deleted all my accounts besides one tiny one with about a dozen emails, and it still uses this much memory.

Here is the detailed information from the run command interface process monitor from just now:

Process 3454 - akonadi_nepomuk_email_feeder

Summary

The process akonadi_nepomuk_emai (with pid 3454) is using approximately 1.5 GB of memory.

It is using 1.5 GB privately, and a further 23.5 MB that is, or could be, shared with other programs.

Dividing up the shared memory between all the processes sharing that memory we get a reduced shared memory usage of 2.8 MB. Adding that to the private usage, we get the above mentioned total memory footprint of 1.5 GB.
Library Usage

The memory usage of a process is found by adding up the memory usage of each of its libraries, plus the process's own heap, stack and any other mappings.

Private:
1583064 KB [heap]
1384 KB /usr/lib64/libQtWebKit.so.4.7.3
280 KB /usr/lib64/libQtGui.so.4.7.3
184 KB /usr/lib64/libkdeui.so.5.7.0
140 KB /lib64/libcrypto.so.1.0.0

Shared:
3688 KB /usr/lib64/libQtWebKit.so.4.7.3
2824 KB /usr/lib64/libQtGui.so.4.7.3
1680 KB /usr/lib64/libkdeui.so.5.7.0
1372 KB /usr/lib64/libkdecore.so.5.7.0
1292 KB /usr/lib64/libQtCore.so.4.7.3

Totals

Private 1587592 KB (= 312 KB clean + 1587280 KB dirty)
Shared 24092 KB (= 24092 KB clean + 0 KB dirty)
Rss 1611684 KB (= Private + Shared)
Pss 1590497 KB (= Private + Shared/Number of Processes)
Swap 0 KB

Reproducible: Always

Steps to Reproduce:
Start the computer

Actual Results:
Huge memory footprint

Expected Results:
Smaller memory footprint

Comment 1 Thomas Zell 2011-08-15 20:49:40 UTC

Similarly as in bug 260647 the process keeps running and consuming memory even after Kontact has been shut down.

Comment 2 Frédéric COIFFIER 2011-09-27 10:32:15 UTC

It also makes lots of disk access [1GB after a 32 minutes uptime] (with iotop) :


  TID  PRIO  USER     DISK READ DISK WRITE>  SWAPIN      IO    COMMAND                                                                                                                                                                       
12339 be/4 fcoiffie      4.00 K   1286.59 M  0.00 %  0.01 % akonadi_nepomuk_email_feeder --identifier akonadi_nepomuk_email_feeder

Comment 3 Sven Eden 2011-09-27 13:14:49 UTC

I would be happy if the email_feeder *only* ate 1.5GB... Here is what top says (and htop confirms that):

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4571 sed       20   0 4879m 1.9g 2836 S    0 52.7   0:36.26 /usr/bin/akonadi_nepomuk_email_feeder --identifier akonadi_nepomuk_email_feeder

Right now it does not use any CPU, but the memory isn't freed, and I have ~300MB RAM and ~1.7GB swap remaining (from 4GB + 6GB!).

Why does the email_feeder need such a massive amount of ram?

My specs:
Gentoo Linux, everything built from source.
QT-4.7.4
KDE-4.7.1
strigi-0.7.6 (if that matters.)

Comment 4 Constantinos Maltezos 2011-11-04 10:09:20 UTC

*** This bug has been confirmed by popular vote. ***

Comment 5 Sven Eden 2011-11-05 11:17:00 UTC

I must say, that upgrading to KDE-4.7.3 seems to have changed the reported behavior. The stats from my last post no read:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 8517 sed       20   0  403m  36m  26m S    0  1.0   0:00.98 akonadi_nepomuk

The PID is the one from:

sed@sed-notebook ~ $ ps -efa | grep email_feeder
sed       8517  8486  0 10:44 ?        00:00:01 /usr/bin/akonadi_nepomuk_email_feeder --identifier akonadi_nepomuk_email_feeder

Therefore this bug/issue might be resolved now, but I'll keep an eye on my system.

Comment 6 Thomas Zell 2011-11-06 15:00:57 UTC

I'm still not happy about the numbers:

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
7248 xxx      20   0  880m 497m  25m S  0.7 24.8   0:41.18 akonadi_nepomuk

500MB out of 2GB is way too much!

Comment 7 Markus Blaschke 2011-11-11 08:08:13 UTC

I can confirm this bug on Kubuntu 11.10 with KDE 4.7.3.
akonadi_nepomuk_email_feeder takes consume ~6-8 GB RAM after login, have to kill.

Comment 8 Todd 2011-11-11 08:21:06 UTC

What version of soprano are you using?

Comment 9 Markus Blaschke 2011-11-11 20:12:02 UTC

soprano-daemon 2.7.0+dfsg.1-0ubuntu2
libsmokesoprano3 4:4.7.2-0ubuntu1~ppa1
libsmokesopranoclient3 4:4.7.2-0ubuntu1~ppa1
libsmokesopranoserver3 4:4.7.2-0ubuntu1~ppa1

akonadi_nepomuk_email_feeder is started before any akonadi resource is activated (i think, because i've not logged in into kwallet)

Comment 10 Todd 2011-11-11 21:49:03 UTC

soprano-daemon 2.7.0+dfsg.1-0ubuntu2

That may be the problem.  The current version is 2.7.3, which is the version recommended for KDE 4.7.3.  There have apparently been major improvements from 2.7.0 to 2.7.3, including performance and indexing improvements.  You should probably see if you can get the kubuntu devs to upgrade to the current version. soprano-daemon 2.7.0+dfsg.1-0ubuntu2

Comment 11 Sven Eden 2011-11-12 12:20:49 UTC

(In reply to comment #8)
> What version of soprano are you using?

(In reply to comment #10)
> soprano-daemon 2.7.0+dfsg.1-0ubuntu2
> 
> That may be the problem.  The current version is 2.7.3, which is the version
> recommended for KDE 4.7.3.  There have apparently been major improvements from
> 2.7.0 to 2.7.3, including performance and indexing improvements.  You should
> probably see if you can get the kubuntu devs to upgrade to the current version.
> soprano-daemon 2.7.0+dfsg.1-0ubuntu2

I do not know whether this is the culprit. As I have stated, I have no longer any problem with the e-mail-feeder, and I have soprano-2.7.0 (compiled from source) installed.

Further (might be) related packages:

clucene-0.9.21b
raptor-2.0.4
rasqal-0.9.27
redland-1.0.14
virtuoso-odbc-6.1.3
virtuoso-server-6.1.3
strigi-0.7.6
nepomuk-4.7.3

And strigi is the only package that got updated with the kde upgrade to 4.7.3 on my laptop. But I can't say from which version, sorry.

Comment 12 Todd 2011-11-13 09:59:54 UTC

Soprano really should be at 2.7.3, not 2.7.0.  You should contact Kubuntu and try to get them to update.

Comment 13 Christophe Marin 2012-01-07 20:36:08 UTC

*** Bug 290207 has been marked as a duplicate of this bug. ***

Comment 14 Christophe Marin 2012-01-07 20:36:17 UTC

*** Bug 285699 has been marked as a duplicate of this bug. ***

Comment 15 Christophe Marin 2012-01-07 20:37:21 UTC

Please retry when 4.8 is out. There were significant fixes in Nepomuk and Virtuoso (make sure your distro ship 6.1.4)

On the akonadi_nepomuk side, there's now only one feeder.

Comment 16 Volker Krause 2012-03-11 17:43:29 UTC

akonadi_nepomuk_email_feeder doesn't exist anymore in KDE PIM 4.8, its replacement has significantly improved throttling to address this issue.