Created attachment 69073 [details] decompressed about 2 megabytes of strace log Version: 4.7 (using KDE 4.7.4) OS: Linux I am using Akonadi 1.6.2 and KMail 4.7.4 on a Gentoo platform. I deleted several thousand files in an IMAP folder (with activated disconnected feature) and another thousand files in my local trash folder. This resulted in heavy disk I/O about 15-20 MBytes/sec for at least 15 minutes, probably longer. $ dstat ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 22 10 41 26 0 0| 443k 6371k| 0 0 | 0 0 | 598 704 38 46 0 16 0 1| 0 37M| 0 0 | 0 0 |1718 731 18 17 0 64 0 1| 0 14M| 0 0 | 0 0 | 739 659 28 30 0 41 0 1| 0 26M| 0 0 | 0 0 |1319 545 19 22 0 58 0 0| 0 18M| 0 0 | 0 0 | 919 407 23 24 0 52 0 1| 0 17M| 0 0 | 0 0 | 889 614 23 23 0 53 0 1| 0 19M| 0 0 | 0 0 | 868 604 29 23 0 48 0 0| 0 19M| 0 0 | 0 0 | 980 501 9 7 0 83 0 1| 0 4880k| 0 0 | 0 0 | 259 171 11 8 0 81 0 0| 0 5140k| 0 0 | 0 0 | 313 587 17 12 0 71 0 0| 0 10M| 0 0 | 0 0 | 521 337 6 5 0 89 0 0| 0 2262k| 0 0 | 0 0 | 222 270 12 8 0 80 0 0| 0 5392k| 0 0 | 0 0 | 374 598 I traced the culprits to the Akonadi processes associated with the mail folders from where I deleted the mails: Both processes were truncating (down to zero) and writing one file each over and over again: 1. process as shown in ps: "/usr/bin/akonadi_agent_launcher akonadi_maildir_resource akonadi_maildir_resource_2" path to heavy i/o file: ~/.config/akonadi/agent_config_akonadi_maildir_resource_2_changes.dat 2. ps: "/usr/bin/akonadi_imap_resource --identifier akonadi_imap_resource_12" path: ~/.config/akonadi/agent_config_akonadi_imap_resource_12_changes.dat At the beginning (at least the moment when I realized something was wrong) the latter file was about 200 KBytes and the former about 2 MBytes. The file size was shrinking each truncate/write cycle by a small amount (around 1 kilobytes/sec). I see no need for I/O of that magnitude, just because I deleted a huge bunch of obsolete e-mails. Just tried to reproduce the problem and it seems that pretty much any huge change to a mail folder will have this effect. I copied an IMAP folder of a thousand e-mails to my local mail folder and the fun begun. Using a SSD I am pretty sure that this behaviour will shorten the lifespan of my disk unneccessarily. By now I have migrated the problematic files temporarily to a classic hard disk and will stay away from huge changes in my mail folders without a hard disk at hand. PS: I thought Linux's disk cache would take care of situations like that, but my HDD LED is flickering like crazy. Reproducible: Always Steps to Reproduce: either delete a huge number (more than 1000) of e-mails from one folder or copy a huge number of e-mails from a IMAP folder to a local mail folder Actual Results: seemingly unnecessary I/O Expected Results: much less I/O I have attached an unfiltered strace from one of the processes going wild. Maybe bug #293175 is related to mine, but the information given there is too little to be sure.
if your are on gentoo perhaps you can compile kdepim-*-4.8.x akonadi-1.7 We made a lot of improvment so it will better to test with new version. 4.7.x is unsupported now. Regards
Sorry, took my computer some time to build that on an old Pentium 4. :) I can now verify that the problem still exists in 4.8 and Akonadi 1.7. Addionally, I just saw, that the file agent_config_akonadi_nepomuk_feeder_changes.dat shows the same behaviour, but grows in size. I assume the not-shrinking is to blame to the fact, that I have Nepomuk deactivated and so no one is on the receiving end. Its about 6.5 megabytes by now. I think it's growing with every change to my mail directories. Probably another bug, which I will file as soon as this one is resolved for me. I observed an interessting fact: The problem only occurs if I move around or copy mail within Kontact. If a big pile of mail is introduced from the IMAP server, the relevant Akonadi agent keeps quiet and just the nepomuk agent starts acting up. I will attach two new strace outputs from an IMAP resource process, both compressed with XZ again. PS: During a move of mail folder Kontact crashed and afterwards the mail folder at the old position was gone, at the new position it was empty and then the new empty folder disappeared into thin air after I checked my mail in the account containing the new folder. I begin to understand why some people say KMail2 is problematic to say the least. This would probably not have happened with KMail1, where all mail is backed up by a maildir folder and there was no "complex" Akonadi metadata scheme sandwiched inbetween. Don't worry I will bear with you. So, what do you need from me?
Created attachment 69254 [details] XZ-compressed strace from Akonadi IMAP resource process writing to IMAP
Created attachment 69255 [details] XZ-compressed strace from Akonadi IMAP resource process deleting mails
Thank you Christian for your report. It is about a version of KMail which uses Nepomuk and is unmaintained. Thus closing. I have seen issues like this back then as well, but not with more recent versions of KDEPIM. If you still see this issue with KDEPIM 4.14 please open a new report. Also note that KDEPIM and Akonadi 15.08 contains some and KDEPIM and Akonadi 15.12 will contain even more, massive performance improvements. Thank you and greetings from KDE Randa Meetings, Martin