Bug 294772

Created attachment 69073 [details]
decompressed about 2 megabytes of strace log

Version:           4.7 (using KDE 4.7.4) 
OS:                Linux

I am using Akonadi 1.6.2 and KMail 4.7.4 on a Gentoo platform. I deleted several thousand files in an IMAP folder (with activated disconnected feature) and another thousand files in my local trash folder. This resulted in heavy disk I/O about 15-20 MBytes/sec for at least 15 minutes, probably longer.

$ dstat 
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
 22  10  41  26   0   0| 443k 6371k|   0     0 |   0     0 | 598   704
 38  46   0  16   0   1|   0    37M|   0     0 |   0     0 |1718   731
 18  17   0  64   0   1|   0    14M|   0     0 |   0     0 | 739   659
 28  30   0  41   0   1|   0    26M|   0     0 |   0     0 |1319   545
 19  22   0  58   0   0|   0    18M|   0     0 |   0     0 | 919   407
 23  24   0  52   0   1|   0    17M|   0     0 |   0     0 | 889   614
 23  23   0  53   0   1|   0    19M|   0     0 |   0     0 | 868   604
 29  23   0  48   0   0|   0    19M|   0     0 |   0     0 | 980   501
  9   7   0  83   0   1|   0  4880k|   0     0 |   0     0 | 259   171
 11   8   0  81   0   0|   0  5140k|   0     0 |   0     0 | 313   587
 17  12   0  71   0   0|   0    10M|   0     0 |   0     0 | 521   337
  6   5   0  89   0   0|   0  2262k|   0     0 |   0     0 | 222   270
 12   8   0  80   0   0|   0  5392k|   0     0 |   0     0 | 374   598

I traced the culprits to the Akonadi processes associated with the mail folders from where I deleted the mails: 

Both processes were truncating (down to zero) and writing one file each over and over again:

1.
process as shown in ps: "/usr/bin/akonadi_agent_launcher akonadi_maildir_resource akonadi_maildir_resource_2"
path to heavy i/o file: ~/.config/akonadi/agent_config_akonadi_maildir_resource_2_changes.dat

2.
ps: "/usr/bin/akonadi_imap_resource --identifier akonadi_imap_resource_12"
path: ~/.config/akonadi/agent_config_akonadi_imap_resource_12_changes.dat 

At the beginning (at least the moment when I realized something was wrong) the latter file was about 200 KBytes and the former about 2 MBytes. The file size
was shrinking each truncate/write cycle by a small amount (around 1 kilobytes/sec). I see no need for I/O of that magnitude, just because I deleted a huge bunch of obsolete e-mails.

Just tried to reproduce the problem and it seems that pretty much any huge change to a mail folder will have this effect. I copied an IMAP folder of a thousand e-mails to my local mail folder and the fun begun.

Using a SSD I am pretty sure that this behaviour will shorten the lifespan of my disk unneccessarily. By now I have migrated the problematic files temporarily to a classic hard disk and will stay away from huge changes in my mail folders without a hard disk at hand.

PS: I thought Linux's disk cache would take care of situations like that, but my HDD LED is flickering like crazy.

Reproducible: Always

Steps to Reproduce:
either
delete a huge number (more than 1000) of e-mails from one folder
or
copy a huge number of e-mails from a IMAP folder to a local mail folder

Actual Results:  
seemingly unnecessary I/O

Expected Results:  
much less I/O

I have attached an unfiltered strace from one of the processes going wild.

Maybe bug #293175 is related to mine, but the information given there is too little to be sure.