Heya, I often see memory usage of 800 mb and up so I decided to run valgrind. I managed to get memory usage up to almost 2 GB, how's that? Hope the valgrind reports here are helpful. /Jos Reproducible: Always Steps to Reproduce: 1. open KMail 2. open some tabs and open folders in those tabs 3. wait and see what happens to the memory usage... Actual Results: Could get it to almost 2GB of ram (with valgrind). Without valgrind, 1.5 gb is doable. That is JUST the KMail process! Expected Results: Something like 300 mb seems far more reasonable...
Created attachment 78261 [details] massif.out output last run (almost 2 gb of ram reached in total)
See https://bugs.kde.org/show_bug.cgi?id=317120 for Korgac going crazy, too.
Right now I've (with current self compiled 4.10 branch): 1.2G MySQL 900M Kontact
610M akonadiserver 200M virtuoso 155M akonadi_nepomuk_feeder 70M akonadi_agent_launcher several resources with 10-20M The numbers are from KDE's System Activity. Jos, I also had a look with massif. That seems to be the headers of the mails (from the visited folders). I guess that the headers are not freed when you switch the folders.
that's still far from what I see here. akonadiserver is using less than 20M. same for the resources (eg: the nepomuk feeder is taking 11M) I'm using a self-built kdepim/kdepim-runtime/kdepimlibs/akonadi stack, all from master and built with the debugfull flag. Did you try with a self compiled kdepim ?
Yep, as noted: self compiled 4.10 branch. I forgot to mention that this is on x86_64, but 3GB for a mail client is a bit too heavy even with 64 bit. Do you have some folders with several thousands mails? I've with maildir and online IMAP resources.
I'm only using dimap accounts. some folders have several thousands email When one process starts using an unusual amount of memory, please try: gdb -p `pidof <executable_name` -ex "call malloc_trim(0)" -ex detach -ex quit and check if this has a significant effect eg: gdb -p `pidof akonadi_nepomuk_feeder` -ex "call malloc_trim(0)" -ex detach -ex quit
not really :-( I tried MySQL, akonadiserver, kontact without effect. On akonadi_nepomuk_feeder and maildir resource it results in 30-40% less memory, but they were not that big before.
(In reply to comment #8) > On akonadi_nepomuk_feeder and maildir resource it results in 30-40% less > memory, but they were not that big before. in your kdepim-runtime build directory, look at ./agents/nepomukfeeder/nepomukfeeder-config.h. Are HAVE_MALLOC_H and HAVE_MALLOC_TRIM #defined ?
(In reply to comment #9) > in your kdepim-runtime build directory, look at > ./agents/nepomukfeeder/nepomukfeeder-config.h. I only have #define NEPOMUK_FEEDER_INDEX_COMPAT_LEVEL 3 > Are HAVE_MALLOC_H and HAVE_MALLOC_TRIM #defined ? Is this from master?
commit bd48587 in the KDE/4.10 branch
I see. I didn't update pim for some time to debug another issue. I update now and let you know the results in the next days.
akonadi_nepomuk_feeder seems to be better now. Sometimes it grows but after some time it also shrinks again. Right now it uses 20MB. But the main problem remains: MySQL, Kontact and akonadi_server together use between 2 and 3 GB after some hours.
- about mysql, how much ram does it use on akonadi startup ? afaics, mariadb uses much more memory on startup than the 'real' mysqld - I can't reproduce the akonadiserver memory usage. even after a couple hours, it's still at ~20M.
(In reply to comment #14) > - about mysql, how much ram does it use on akonadi startup ? afaics, mariadb > uses much more memory on startup than the 'real' mysqld After akonadictl restart MySQL starts with 100MB but grows quickly, even without Kontact running. > - I can't reproduce the akonadiserver memory usage. even after a couple > hours, it's still at ~20M. Well, it starts with 20MB,
Created attachment 80398 [details] valgrind/massif output
Created attachment 80399 [details] valgrind/massif output
I added valgrind/massif output from akonadiserver and MySQL. Maybe it helps you. Even with not doing that much Akonadiserver used 160MB. I'm running Debian Sid on x86_64 with MySql 5.5.31.
Can you install libjemalloc stop all akonadi processes export LD_PRELOAD=/usr/bin/libjemalloc and measure again through ksysguard ?
(In reply to comment #19) > Can you install libjemalloc and measure again through ksysguard ? No improvement :-(
@Andre, can you try with latest akonadiserver from master. Some memory usage fixes were done there.
(In reply to comment #21) > @Andre, can you try with latest akonadiserver from master. > > Some memory usage fixes were done there. Hi Sergio, I already replied to your fix (QueryCache cleanup). It improved the situation a lot but e. g. right now after a day of work MySQL consumes 350MB and akonadiserver 140 MB. So there is still room for further improvement :-)
Andre, which resources do you have ? are you using kontact or kmail standalone, if kontact, which other resources ? Open akonadiconsole, browser tab, how many items do you have on each collection ?
(In reply to comment #23) > Andre, > > which resources do you have ? 5 imap 4 pop3 maildir birthday davgroup ical addressbook notes > are you using kontact or kmail standalone, if kontact, which other resources > ? I'm using kontact. > Open akonadiconsole, browser tab, how many items do you have on each > collection ? several 10k mails. I switched to 4.11 branch a few days ago (which was painfull again :-( ) and now I see your numbers :-) Now the resources could be optimized (up to 20MB for imap/pop3 and 30MB for the maildir resource). I don't know what the MySQL's minimum usage is.
Git commit 815957a4a9ae567f271453aac222abf4c09900f9 by Sergio Martins. Committed on 17/07/2013 at 09:45. Pushed by smartins into branch 'KDE/4.11'. mixedmaildir: don't use 1.5GB of memory when importing a very big folder Subjobs pile up too fast because they are cheap to create and because it's taking around 5 event loops for one to start. Related: bug 260647, bug 309732, bug 307066, bug 295651 M +41 -6 resources/mixedmaildir/retrieveitemsjob.cpp M +4 -2 resources/mixedmaildir/retrieveitemsjob.h http://commits.kde.org/kdepim-runtime/815957a4a9ae567f271453aac222abf4c09900f9
Jos, Andre, thank you for your bug report and measurements. Jos, Andre, can you still reproduce this with KDEPIM 4.14 and Akonadi 1.13, preferably compiled from 4.14 and 1.13 branches? I have seen quite some optimization work going, including fixes for memory usage issues, into KDEPIM and Akonadi since 2013. So please try to reproduce with more recent versions of KDEPIM + Akonadi. Please also include your current usage scenario and if possible steps to reproduce. Thank you, Martin
Setting to waiting for information after I received bugzilla rights to do that.
It's much better nowadays :-) I've: mysql about 110 MB akonadiserver 60 MB but also akonadi_agent_launcher at 66 MB. IMO that could be still improved (lets see what akonadi next will bring) but I'm fine with closing this bug.
My findings are: merkaba:~> smem -r | head -1 ; smem -r | egrep "mysql|akonadi" PID User Command Swap USS PSS RSS 2454 martin /usr/sbin/mysqld --defaults 21092 614016 614033 616552 1645 martin /usr/local/bin/akonadi_agen 0 310020 310916 343472 1639 martin /usr/bin/akonadi_baloo_inde 0 79436 81108 121608 1552 martin akonadiserver 0 65660 67361 94440 25339 martin /usr/local/bin/akonadiconso 0 26308 39426 111656 1647 martin /usr/local/bin/akonadi_mail 0 17680 21420 74980 1644 martin /usr/local/bin/akonadi_imap 0 18292 20464 64540 1651 martin /usr/local/bin/akonadi_pop3 0 15604 16510 51944 1638 martin /usr/local/bin/akonadi_arch 0 12548 15961 68684 1657 martin /usr/local/bin/akonadi_send 0 10168 13043 63676 1650 martin /usr/local/bin/akonadi_note 0 10724 12702 59024 1649 martin /usr/local/bin/akonadi_newm 0 8592 9741 47788 1646 martin /usr/local/bin/akonadi_mail 0 7584 8488 44908 1652 martin /usr/local/bin/akonadi_pop3 0 7300 8168 43756 1643 martin /usr/local/bin/akonadi_ical 0 7080 8058 42976 1642 martin /usr/local/bin/akonadi_foll 0 5940 6752 40792 1641 martin /usr/local/bin/akonadi_agen 0 5932 6678 37896 1637 martin /usr/local/bin/akonadi_agen 0 5804 6665 38440 1648 martin /usr/local/bin/akonadi_migr 0 5860 6603 40060 1640 martin /usr/local/bin/akonadi_birt 0 5792 6553 40400 1635 martin /usr/local/bin/akonadi_agen 0 5656 6548 38772 1656 martin /usr/local/bin/akonadi_pop3 0 5612 6546 42104 1655 martin /usr/local/bin/akonadi_pop3 0 5696 6486 40608 1653 martin /usr/local/bin/akonadi_pop3 0 5608 6440 40572 1654 martin /usr/local/bin/akonadi_pop3 0 5600 6375 40492 2297 martin /usr/bin/akonaditray -sessi 2028 3704 4075 27756 1549 martin /usr/local/bin/akonadi_cont 0 2496 2828 12660 7331 root grep -E mysql|akonadi 0 368 389 2220 This is with an enormous maildir with more than 1100000 unread mails. 1645 is maildir resource. And mysql is with innodb_buffer_pool_size=512M and SizeThreshold=32768 in akonadiserverc and this DB: 2,4G akonadi martin@merkaba:~/.local/share/akonadi/db_data> LANG=C du -sch * | sort -rh 2.7G total 2.4G akonadi 178M ibdata1 64M ib_logfile1 64M ib_logfile0 1020K mysql 208K performance_schema 200K mysql.err.old 4.0K merkaba.pid 20K mysql.err
Memory usage is still high but has become far better. And: Akonadi Next should have FAR better memory usage by design, so let's close this one.