Bug 290393

Summary:	moving the contents of a mail folder is extremely slow
Product:	[Frameworks and Libraries] Akonadi	Reporter:	Bruno Haible <bruno>
Component:	Mixed Maildir resource	Assignee:	kdepim bugs <kdepim-bugs>
Status:	RESOLVED UNMAINTAINED
Severity:	normal	CC:	krammer, Martin
Priority:	NOR
Version:	4.7
Target Milestone:	---
Platform:	openSUSE
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:

Description Bruno Haible 2012-01-02 12:26:08 UTC

Version:           4.7.2 (using KDE 4.7.3) 
OS:                Linux

kmail 4.7.4, akonadi 4.7.4

Inside a mixedmaildir resource, I moved the entire contents of a mail folder to
an empty mail folder. Both mail folders are in mbox format, on a local disk.

Time it takes to move 139 mails: 12 sec.
Time it takes to move 263 mails: 45 sec.
Time it takes to move 4462 mails: 32340 sec.

As you can see, the time needed is O(N^2) (or more), where N is the number of
emails to be moved.

More precisely, when moving the 4462 mails:
- Moving the mails to the new folder on disk took 41 minutes (on a slow USB 1.1 disk),
- The size of the old folder then went down to 0,
- For 485 minutes, akonadi updated the $HOME/.local/share/akonadi/db_data/* files, kmail using 100% CPU most of the time,
- Finally, 13 minutes after akonadi/parttable.ibd was last modified, also akonadi/pimitemtable.ibd was last modified. The view updated to empty, as it should.

Reproducible: Didn't try

Steps to Reproduce:
In a mixedmaildir resource with at least 2 mbox folders:
Select a folder.
Press Ctrl-. to view all mails.
Select one of the mails.
Press Ctrl-A to select all mails.
Drag-and-drop the selection to the other folder.


Actual Results:  
Run time that is quadratic in the number of mails.

Expected Results:  
Run time that is linear in the number of mails.
In previous releases of kmail, operations like this one used to take
minutes, not hours. Cf. https://bugs.kde.org/show_bug.cgi?id=104721

Comment 1 Bruno Haible 2012-01-02 12:29:33 UTC

The reason why I moved the contents of the folder, instead of renaming the
folder, is that the first folder has some subfolders.
Namely, the source folder is 'gnulib', with subfolders 'incoming' and 'waiting'.
I move the contents of 'gnulib' to 'gnulib-2011', but the subfolders 'incoming'
and 'waiting' are not specific to a single year and therefore stay under 'gnulib'.

Comment 2 Kevin Krammer 2012-01-08 18:28:25 UTC

Interesting observation.

I haven't had time yet to try this myself, but my guess is the problem is the "compaction" of the source mbox.

Could you check if copying from an mbox to an mbox inside mixedmaildir shows the same behavior?

Comment 3 Bruno Haible 2012-01-08 19:26:22 UTC

> my guess is the problem is the "compaction" of the source mbox.

As I wrote in the original description, 90% of the time was spent after the size of the original mbox file
had gone down to 0 and the size of the destination mbox file had reached its final size. During this long time,
akonadi was updating databases. Each database update took ca. 7 seconds.

> Could you check if copying from an mbox to an mbox inside mixedmaildir shows
> the same behavior?

That's precisely what I did. From an mbox to an mbox inside the same mixedmaildir resource.

Is it that hard to reproduce? Do you want me to send you an mbox folder with 4000 emails?
If so, please contact me by private email.

Comment 4 Martin Steigerwald 2016-06-08 19:50:25 UTC

Bruno, thank you for your report. Does it still happen for you with KDEPIM + Akonadi 16.04.

I bet it may still happen as I find moving folder on maildir *extremely* inefficient with that version still, but please confirm. Otherwise I intend to close this bug report, as KDEPIM 4 and Akonadi 1 is unmaintained. Thanks, Martin

Comment 5 Martin Steigerwald 2016-06-08 20:08:49 UTC

If Akonadi behaves similarily as I described in

364114 - moving a folder within one maildir resource is extremely slow and inefficient

with mbox folders, this may explain quite a lot of the slowness you described.

Comment 6 Denis Kurz 2016-09-24 20:37:51 UTC

This bug has only been reported for versions older than KDEPIM 4.14 (at most akonadi-1.3). Can anyone tell if this bug still present?

If noone confirms this bug for a recent version of akonadi (part of KDE Applications 15.08 or later), it gets closed in about three months.

Comment 7 Denis Kurz 2017-01-07 22:18:37 UTC

Just as announced in my last comment, I close this bug. If you encounter it again in a recent version (at least 5.0 aka 15.08), please open a new one unless it already exists. Thank you for all your input.