Bug 294447 - akonadi_nepomuk_feeder and akonadi_mailfilter_agent memory leak
Summary: akonadi_nepomuk_feeder and akonadi_mailfilter_agent memory leak
Status: RESOLVED WORKSFORME
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: Nepomuk Feeder Agents (show other bugs)
Version: unspecified
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-02-19 19:31 UTC by Ilya Potapov
Modified: 2012-05-26 19:44 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ilya Potapov 2012-02-19 19:31:22 UTC
Version:           unspecified (using KDE 4.8.0) 
OS:                Linux

There is fast memory leak of akonadi_nepomuk_feeder and akonadi_mailfilter_agent processes after recent update. I have to completely disable akonadi as temporary workaround.

Reproducible: Always

Steps to Reproduce:
Enable akonadi server.

Actual Results:  
Fast memory usage grow

Expected Results:  
No memory leak at all
Comment 1 Laurent Montel 2012-02-20 08:51:43 UTC
Ok perhaps mem leak. but needs info.
How do you have see it ?
Where is it ?
How do you reproduce it ? 
etc.

We need usefull bug report not just "it leaks"...

Thanks 
Regards
Comment 2 Ilya Potapov 2012-02-25 12:15:18 UTC
For now i changed StartServer to false in ~/.config/akonadi/akonadiserverrc to prevent start of akonadi server. 
So, steps to reproduce are:
If i simply turn it back to "true" and reboot or just start something like kmail or simply click "start server" in akonadi tray icon menu, akonadi server starts, and, as i wrote, this cause fast memory usage grow every time.

I do not actually know if it is memory leak, but just after server start CPU goes to 100% and amount of memory used by akonadi_nepomuk_feeder and
akonadi_mailfilter_agent starts to grow very fast. I can see it with htop or ksysguard. In two minutes RAM is over and system starts swapping a lot, i experience perfomance issues and freezes, and only way i found is to kill that to processes and to stop akonadi server to prevent restarting of that processes.

I can definetly say that this behaviour is abnormal.

Please, tell me, how can i provide any usfull information. I will do it with pleasure.

That procecesses not crashing and i thus can not provide any crash backtrace. Maybe i should install any additional debugging packages and use them somehow. But i just do not know how.
Comment 3 Jörg Schaible 2012-02-25 21:30:38 UTC
Since today a face exactly the same. When KMail starts the process takes within seconds a lot of memory (several Gigs) until I kill it.
Comment 4 Jörg Schaible 2012-02-25 21:32:17 UTC
Forget to mention: I am using kmail 4.7.4 installed a month ago. Worked until now.
Comment 5 Jörg Schaible 2012-02-25 21:57:32 UTC
For me it's getting worse now. I use "akonadictl restart" to reinitialize akonadi and start Kmail then ... first it looks normal, but after a view seconds it starts to allocate memory and finally crashes after the process took over 4GB:

Application: KMail (kmail), signal: Segmentation fault
[Current thread is 1 (Thread 0x7fb6fbb4a780 (LWP 16155))]

Thread 3 (Thread 0x7fb6e163b700 (LWP 16159)):
#0  pthread_cond_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00007fb6ee251214 in scavengerThread (this=0x7fb6eead8e60) at wtf/FastMalloc.cpp:2378
#2  WTF::TCMalloc_PageHeap::runScavengerThread (context=0x7fb6eead8e60) at wtf/FastMalloc.cpp:1497
#3  0x00007fb6f713fd1c in start_thread () from /lib64/libpthread.so.0
#4  0x00007fb6f923c93d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 2 (Thread 0x7fb6e0d3a700 (LWP 16160)):
#0  0x00007fb6f92323e3 in poll () from /lib64/libc.so.6
#1  0x00007fb6f1d9369b in g_main_context_poll (n_fds=1, fds=0x1d8d2b0, timeout=-1, context=0x1d8b8a0, priority=<optimized out>) at gmain.c:3402
#2  g_main_context_iterate (context=0x1d8b8a0, block=<optimized out>, dispatch=1, self=<optimized out>) at gmain.c:3084
#3  0x00007fb6f1d93884 in g_main_context_iteration (context=0x1d8b8a0, may_block=1) at gmain.c:3152
#4  0x00007fb6f9b404de in QEventDispatcherGlib::processEvents (this=0x1d8b6c0, flags=<optimized out>) at kernel/qeventdispatcher_glib.cpp:422
#5  0x00007fb6f9b217ef in QEventLoop::processEvents (this=<optimized out>, flags=...) at kernel/qeventloop.cpp:149
#6  0x00007fb6f9b219cc in QEventLoop::exec (this=0x7fb6e0d39de0, flags=...) at kernel/qeventloop.cpp:201
#7  0x00007fb6f9a88532 in QThread::exec (this=<optimized out>) at thread/qthread.cpp:498
#8  0x00007fb6f9a8a339 in QThreadPrivate::start (arg=0x1d8a700) at thread/qthread_unix.cpp:331
#9  0x00007fb6f713fd1c in start_thread () from /lib64/libpthread.so.0
#10 0x00007fb6f923c93d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 1 (Thread 0x7fb6fbb4a780 (LWP 16155)):
[KCrash Handler]
#6  0x00007fb6f9a8c10a in QByteArray::append (this=0x7fff2e7a25c0, ba=...) at tools/qbytearray.cpp:1587
#7  0x00007fb6f5d8b34e in operator+= (a=..., this=0x7fff2e7a25c0) at /usr/include/qt4/QtCore/qbytearray.h:492
#8  KMime::Content::encodedBody (this=<optimized out>) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4/kmime/kmime_content.cpp:345
#9  0x00007fb6f5d8b45a in KMime::Content::encodedContent (this=0x2159fa0, useCrLf=false) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4/kmime/kmime_content.cpp:284
#10 0x00007fb6db5ac6aa in Akonadi::SerializerPluginMail::serialize (this=<optimized out>, item=<optimized out>, label=..., data=..., version=<optimized out>) at /var/tmp/portage/portage/kde-base/kdepim-runtime-4.7.4/work/kdepim-runtime-4.7.4/plugins/akonadi_serializer_mail.cpp:169
#11 0x00007fb6f5a6eced in Akonadi::ItemSerializer::serialize (item=..., label=..., data=..., version=@0x7fff2e7a2f8c) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4/akonadi/itemserializer.cpp:126
#12 0x00007fb6f5a6ed7f in Akonadi::ItemSerializer::serialize (item=..., label=..., data=..., version=@0x7fff2e7a2f8c) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4/akonadi/itemserializer.cpp:116
#13 0x00007fb6f5a70af8 in Akonadi::ItemModifyJobPrivate::nextPartHeader (this=0x22d98b0) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4/akonadi/itemmodifyjob.cpp:59
#14 0x00007fb6f5a70d30 in Akonadi::ItemModifyJob::doHandleResponse (this=<optimized out>, _tag=..., data=<optimized out>) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4/akonadi/itemmodifyjob.cpp:229
#15 0x00007fb6f5a75479 in Akonadi::JobPrivate::handleResponse (this=<optimized out>, tag=..., data=...) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4/akonadi/job.cpp:80
#16 0x00007fb6f5a958df in Akonadi::SessionPrivate::dataReceived (this=0x164a9f0) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4/akonadi/session.cpp:218
#17 0x00007fb6f5a97399 in Akonadi::Session::qt_metacall (this=0x15d2c00, _c=QMetaObject::InvokeMetaMethod, _id=5, _a=0x7fff2e7a35c0) at /var/tmp/portage/portage/kde-base/kdepimlibs-4.7.4-r1/work/kdepimlibs-4.7.4_build/akonadi/session.moc:96
#18 0x00007fb6f9b31aaa in QMetaObject::activate (sender=0x1a76cd0, m=<optimized out>, local_signal_index=<optimized out>, argv=0x0) at kernel/qobject.cpp:3278
#19 0x00007fb6f9b62247 in QIODevice::qt_metacall (this=0x1a76cd0, _c=QMetaObject::InvokeMetaMethod, _id=0, _a=0x7fff2e7a36f0) at .moc/release-shared/moc_qiodevice.cpp:77
#20 0x00007fb6f82df08a in QLocalSocket::qt_metacall (this=0x1a76cd0, _c=QMetaObject::InvokeMetaMethod, _id=<optimized out>, _a=0x7fff2e7a36f0) at .moc/release-shared/moc_qlocalsocket.cpp:81
#21 0x00007fb6f9b31aaa in QMetaObject::activate (sender=0x1688620, m=<optimized out>, local_signal_index=<optimized out>, argv=0x0) at kernel/qobject.cpp:3278
#22 0x00007fb6f82dbad1 in QAbstractSocketPrivate::canReadNotification (this=0x1673c80) at socket/qabstractsocket.cpp:643
#23 0x00007fb6f82d0400 in QReadNotifier::event (this=<optimized out>, e=<optimized out>) at socket/qnativesocketengine.cpp:1103
#24 0x00007fb6fa0129e8 in QApplicationPrivate::notify_helper (this=0x189b520, receiver=0x16608b0, e=0x7fff2e7a3c70) at kernel/qapplication.cpp:4481
#25 0x00007fb6fa0198a9 in QApplication::notify (this=<optimized out>, receiver=0x16608b0, e=0x7fff2e7a3c70) at kernel/qapplication.cpp:4360
#26 0x00007fb6fb5a286a in KApplication::notify (this=0x7fff2e7a4020, receiver=0x16608b0, event=0x7fff2e7a3c70) at /var/tmp/portage/portage/kde-base/kdelibs-4.7.4/work/kdelibs-4.7.4/kdeui/kernel/kapplication.cpp:311
#27 0x00007fb6f9b22290 in QCoreApplication::notifyInternal (this=0x7fff2e7a4020, receiver=0x16608b0, event=<optimized out>) at kernel/qcoreapplication.cpp:787
#28 0x00007fb6f9b4033a in socketNotifierSourceDispatch (source=0x189ddd0) at kernel/qeventdispatcher_glib.cpp:110
#29 0x00007fb6f1d93127 in g_main_dispatch (context=0x189dce0) at gmain.c:2441
#30 g_main_context_dispatch (context=0x189dce0) at gmain.c:3011
#31 0x00007fb6f1d936ee in g_main_context_iterate (context=0x189dce0, block=<optimized out>, dispatch=1, self=<optimized out>) at gmain.c:3089
#32 0x00007fb6f1d93884 in g_main_context_iteration (context=0x189dce0, may_block=1) at gmain.c:3152
#33 0x00007fb6f9b404de in QEventDispatcherGlib::processEvents (this=0x156da60, flags=<optimized out>) at kernel/qeventdispatcher_glib.cpp:422
#34 0x00007fb6fa08f3e4 in QGuiEventDispatcherGlib::processEvents (this=<optimized out>, flags=<optimized out>) at kernel/qguieventdispatcher_glib.cpp:204
#35 0x00007fb6f9b217ef in QEventLoop::processEvents (this=<optimized out>, flags=...) at kernel/qeventloop.cpp:149
#36 0x00007fb6f9b219cc in QEventLoop::exec (this=0x7fff2e7a3ef0, flags=...) at kernel/qeventloop.cpp:201
#37 0x00007fb6f9b2478e in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1064
#38 0x0000000000403470 in main (argc=<optimized out>, argv=<optimized out>) at /var/tmp/portage/portage/kde-base/kmail-4.7.4-r1/work/kmail-4.7.4/kmail/main.cpp:145
Comment 6 Jörg Schaible 2012-02-25 22:50:57 UTC
Process 8218 - kmail

Summary

The process kmail (with pid 8218) is using approximately 4.1 GB of memory.
It is using 4.1 GB privately, and a further 16.8 MB that is, or could be, shared with other programs.
Dividing up the shared memory between all the processes sharing that memory we get a reduced shared memory usage of 1666.0 KB. Adding that to the private usage, we get the above mentioned total memory footprint of 4.1 GB.
70.3 MB is swapped out to disk, probably due to a low amount of available memory left.
Library Usage

The memory usage of a process is found by adding up the memory usage of each of its libraries, plus the process's own heap, stack and any other mappings, plus the stack of its 3 threads. 
Private
more
4252072 KB	[heap]
1364 KB	/usr/lib64/qt4/libQtWebKit.so.4.7.4
700 KB	/usr/lib64/libkmailprivate.so.4.7.0
484 KB	/usr/lib64/libmailcommon.so.4.7.0
436 KB	/usr/lib64/libmessagelist.so.4.7.0
Shared
more
3848 KB	/usr/lib64/qt4/libQtGui.so.4.7.4
1752 KB	/usr/lib64/libkdeui.so.5.7.0
1228 KB	/usr/lib64/qt4/libQtCore.so.4.7.4
1216 KB	/usr/lib64/libkdecore.so.5.7.0
904 KB	/usr/lib64/libakonadi-kde.so.4.7.0
Totals

Private	4259560 KB	(= 67816 KB clean + 4191744 KB dirty)
Shared	17176 KB	(= 17168 KB clean + 8 KB dirty)
Rss	4276736 KB	(= Private + Shared)
Pss	4261226 KB	(= Private + Shared/Number of Processes)
Swap	71940 KB
Comment 7 Jörg Schaible 2012-02-27 18:50:55 UTC
As it turns out, for me it is no memory leak, but a bug with the KMail/clamav integration that causes mail to double size each time they're scanned. One of the mails in my inbox have been grown over two weeks now to 0.5GB (and it not the only one growing), so that KMail finally crashes when it tries to handle it (scan it again ??).

https://bugs.kde.org/show_bug.cgi?id=289277
Comment 8 Nicolas Dubuit 2012-03-25 14:45:44 UTC
I can confirm that bug.

I'm using kubuntu packages, versions :
* akonadi-server 1.7.0
* kdepim 4.8.1

I'm not seeing any akonadi_nepomuk_feeder.

At every kde login, a process akonadi_mailfilter_agent starts eating memory very fast (->4GB in a few seconds) then swaps furiously. I have to kill the process (3 times, I suppose that's because I have 3 mail accounts)

I'm totally willing to report any additional information needed, I just don't know what to add...
Comment 9 Jessie A. Morris 2012-03-29 21:04:55 UTC
I too get this problem, but it grows to about 6.5 GB in memory usage for me, and then starts thrashing. This is visible in KDE 4.8.1 on Kubuntu 12.04 beta.

This was fine yesterday, but the most recent updates killed me. I tried removing all of the Kmail related stuff, but it is akonadi_nepomuk_feeder that eats all of my memory.
Comment 10 Nicolas Dubuit 2012-04-08 11:05:14 UTC
Sorry, I didnt specify that I'm using kubuntu 11.10 packages with KDE 4.8.2 updates.
Comment 11 Mikael Bergqvist 2012-04-11 09:13:53 UTC
Kubuntu 12.04, KDE 4.8.2, akonadi_mailfilter_agent running at 100% CPU and it consumes a lot of memory when just logged in, (and later, when it needs to update). The system becomes sluggish and unresponsive when this happens.
Comment 12 Christophe Marin 2012-04-22 17:23:54 UTC
(In reply to comment #11)
> Kubuntu 12.04, KDE 4.8.2, akonadi_mailfilter_agent running at 100% CPU and
> it consumes a lot of memory when just logged in, (and later, when it needs
> to update). The system becomes sluggish and unresponsive when this happens.

Try that please:
- 'akonadictl stop' and wait until all processes are stopped (check in ksysguard)
- rename ~/.config/akonadi/agent_config_akonadi_mailfilter_agent_changes.dat
- 'akonadictl start'

and check if the memory usage is better
Comment 13 Nicolas Dubuit 2012-04-23 13:24:13 UTC
It solves the problem for me.
Thanks!
Comment 14 Ilya Potapov 2012-04-23 13:56:20 UTC
(In reply to comment #12)
> (In reply to comment #11)
> > Kubuntu 12.04, KDE 4.8.2, akonadi_mailfilter_agent running at 100% CPU and
> > it consumes a lot of memory when just logged in, (and later, when it needs
> > to update). The system becomes sluggish and unresponsive when this happens.
> 
> Try that please:
> - 'akonadictl stop' and wait until all processes are stopped (check in
> ksysguard)
> - rename ~/.config/akonadi/agent_config_akonadi_mailfilter_agent_changes.dat
> - 'akonadictl start'
> 
> and check if the memory usage is better

Hello! I tried that and akonadi_mailfilter_agent is not eating memory for now! Unfortunately, akonadi_nepomuk_feeder process still grows (but now much more 'slowly' - by ~5 MB per second) and seems to go on. It utilizes 50% CPU and after few minutes system starts swapping and freezing.
Comment 15 Christophe Marin 2012-04-23 14:30:28 UTC
Ilya: which KDE version ? 
make sure you use at least 4.8.2, it has some very important fixes wrt Nepomuk
Comment 16 Ilya Potapov 2012-04-23 15:28:37 UTC
I am on KDE 4.8.2 from Kubuntu development branch.
Comment 17 Christophe Marin 2012-04-23 20:32:50 UTC
(In reply to comment #13)
> It solves the problem for me.
> Thanks!

Thanks for the feedback. I created a bug report for this issue:  bug 298399

I'm not marking this bug as duplicate since the issue wasn't fixed for the reporter.
Comment 18 Ilya Potapov 2012-05-26 19:44:44 UTC
Good day! Now i have new laptop instead of that i was reporting for. All works fine with akonadi and nepomuk enabled. May be the cause of such difference is the further fact: here i have a clean install, and on the old one i had upgrade cycles since kubuntu 10.10 so that some old configs or data structure were the cause.
Unfortunately i have no access to old laptop and can not try out a clean install. Nevertheless i think the bug may be closed as nobody else said he still experience this issue.