Bug 299417 - Incorrect local Maildir handling
Summary: Incorrect local Maildir handling
Status: RESOLVED UNMAINTAINED
Alias: None
Product: kmail2
Classification: Applications
Component: commands and actions (show other bugs)
Version: 4.8.3
Platform: Ubuntu Linux
: NOR critical
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-05-05 04:34 UTC by Ivan Adzhubey
Modified: 2015-09-11 19:24 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ivan Adzhubey 2012-05-05 04:34:04 UTC
Hi,

I have noticed a number of bugs in the way Kmail2 (SC 4.8.2/4.8.3) handles local mail:

1.When copying or moving messages from one folder (remote or local) to another local folder, messages are initially placed into "new" subdirectory in the corresponding Maildir folder, regardless of the source message real status (e.g. even if messages are marked as read). Messages should go to "cur" subfolder instead.

2.Message files stored in "new" subfolder after copy/move operation lack any Maildir suffixes, e.g. ":2,S". I assume that the combination of bugs 1&2 is what causes infamous "unreadable messages" bug when clicking on a message header does not display it's text. This may happen due to race condition, see below.

3.Clicking on a newly copied message initiates the following process in the backgound: a) message file is moved from "new" to "cur" subfolder and b) message file is renamed by appending correct ":2,S" suffix to it. This process is ***extremely*** slow and is done in two steps (first move, then rename). In addition, there is absolutely no progress indication in the GUI or any notification whatsoever. This creates a major race condition and critical failure risk point. Because this operation is a) very slow, b) non-transactional (completed in at least two steps) and c) does not notify user of critical background operation in progress, a user would proceed working with folders as usual, e.g. clicking on message headers, attempt to further move/copy messages to another folders or delete them, etc. This would quickly exhaust backend capabilities (akonadi?) to manage file transactions and at some point provoke a crash due to race condition, leaving Maildir tree in unpredictable state and possibly corrupting akonadi database. After that, local mail folders structure becomes completely unaccessible through GUI.

4.A workaround for bugs 1&2 which prevents bug 3 from stricking is to do the follwoing immediately after copying/moving messages between folders in GUI: a) select all messages in the target folder (Ctrl-A), b) mark all messages as Unread (Ctrl-U), c) mark folder as read via its context menu, d) wait until all file operations in Maildir folders described in (3) are completed, tracking progess by watching actual status of the corresponding local Maildir folders (i.e. under ~/.local/share/loacl-mail). The latter may take a long time, up to 48 hours for a copy operation on a folder worth 90,000 messages (Core i7 3GHz, 6GB RAM, 10Krpm drive). If the operation is iterrupted before completion (e.g. by shutting down computer) your local folders will be rendered unusable and you will have to start from scratch.

5.An additional unrelated bug: neither Ctrl-A shortcut nor Edit->Select->All can select all messages in a folder with the message threading enabled and at least some message threads collapsed. You need to uncollapse all threads before issuing "select all" otherwise a number of messages will be randomly left out.


Reproducible: Always




Tested with Kubuntu Desktop 12.04 (x86_64) & KDE SC 4.8.2, reproduced after updating to SC 4.8.3.
Comment 1 Myriam Schweingruber 2012-08-18 08:47:02 UTC
*** Bug 210590 has been marked as a duplicate of this bug. ***
Comment 2 Myriam Schweingruber 2012-08-18 09:04:35 UTC
nvm the dupe, wrong number, sorry for the noise.
Comment 3 Martin Steigerwald 2015-09-11 19:24:28 UTC
Ivan, thank you for your detailed bug report which contains more than one report in one.

It is about a version of KMail which uses Nepomuk and is unmaintained. Thus
closing. If you still see performance issues please open new reports. But
please follow the following guide lines to avoid unnecessary work for the
developers:

- Ideally test with KDEPIM and Akonadi 15.08. It contains some performance
improvements like the binary protocol.

- Otherwise at least use KDEPIM 4.14.10 and newest Akonadi 1.13 you can get as
it already contains some performance improvements.

- If you can wait, please retest with KDEPIM and Akonadi 15.12 once they become
available for you. Akonadi 15.12 will contain *massive* performance
improvements implemented by Dan due to new database indexes, optimized queries
and leveled file_db directory. All of these are in master already, so if you
dare use kdesrc-build to compile KF5, kdepim and kdepim-runtime. I am using
this currently and it basically moves the bottleneck to KMail (displaying message list
of huge folder). It is a *huge* improvement. And also Volker and Dan work to improve
message list display speed as well.

- Also please report just one issue per bug. I would like if you could specically verify whether the maildir related issues still exist. If so please open a separate bug for each issue you find. Please test at least against KMail from KDEPIM 4.14 and latest Akonadi 1.13 you can get.

Thank you and greetings from KDE Randa Meetings,