Bug 367892 - During folder synchronisation Akonadi blocks out other operations like deleting or viewing mails
Summary: During folder synchronisation Akonadi blocks out other operations like deleti...
Status: CONFIRMED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: server (show other bugs)
Version: 5.14.1
Platform: Debian unstable Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-27 11:39 UTC by Martin Steigerwald
Modified: 2021-04-20 13:57 UTC (History)
9 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Steigerwald 2016-08-27 11:39:37 UTC
This is a split out from Bug #338571 (See comment 28 there https://bugs.kde.org/show_bug.cgi?id=338571#c28).

During folder synchronisation KMail blocks out other operations. This causes delays for the user of KMail when the folder synchronisation takes a long time. This happens with IMAP and Maildir resources at least. I bet it would happen with any resource.

Reproducible: Always

Steps to Reproduce:
IMAP:
1. Have some IMAP folder large enough (several ten thousand mail).
2. Hit delete key on mails in the folder mail list often enough, probably with pauses to trigger a folder synchronisation. (Do so with a mails you can afford to delete only of course!)

Maildir:
1. Have a POP3 account with mails from mailings or so and with filters sorting them into different folders.
2. Download new mail.
3. Maildir resource starts to synchronize (I think this is unnecessary as well, see bug 334216).

Of course there are other ways to trigger a folder synchronisation but these worked quite reliably for me.

Then both Maildir and IMAP:
1. Do some operations like deleting or viewing mails while Akonadi is still busy with the folder synchronisation.

Actual Results:  
Akonadi blocks on other operations. Operations that are more important for the user I think, cause the other waits for KMail to view or delete mails, but it does not until Akonadi is responsive again.

Expected Results:  
Akonadi can handles several operations at the same time. Bonus points if it prioritizes short term interactive user requests over longer time background operations.
Comment 1 Martin Steigerwald 2016-08-27 11:44:14 UTC
For the Maildir case this is just a minor annoyance as folder synchronisation is pretty fast with Akonadi 16.04. But with IMAP this can easily cause delays of a minute or more on large folders.

Gunther talks about 87 seconds for a Sent folder of 41000 mails in comment 18 of bug 338571 with a local !() IMAP server (see https://bugs.kde.org/show_bug.cgi?id=338571#c18). I can easily can trigger stalls of a minute or more with a remote Dovecot and folders of about 10000-25000 mails. 

Bug #338571 is about the performance of the IMAP folder synchronisation itself. This bug is about blocking out other operations during folder synchronisation.
Comment 2 Daniel Vrátil 2016-09-10 16:38:01 UTC
Hi Martin,

I'm not sure I understand the issue here correctly. Are you saying that when you hit "Delete" while a folder is being synced, the email does not disappear from KMail until the sync is done? Or is this about how changes are propagated back to the server?
Comment 3 Daniel Vrátil 2016-09-10 16:38:16 UTC
* propagated back to the IMAP server.
Comment 4 Martin Steigerwald 2016-09-10 20:13:51 UTC
Hello Daniel. I am talking about the visible response in KMail. I did not check if there is something happening on the IMAP server. For IMAP, in company with Exchange, but I also tested with Dovecot IMAP on my private server, it is really easy to trigger here: Hit delete key on mails in the same folder until Akonadi starts folder synchronization. Then hit delete key again. KMail will grey out the mail, but only remove it from the list after folder synchronization is complete. To me it seems like Akonadi postpones any interactive user requests until a folder synchronization is complete.

For me two things don´t make sense:

1) Why does it synchronize the folder anyway when I hit delete the folder mail list several times? To make sure the IMAP server has done its work?

2) Postponing user input just leads to user frustration. As a user I expect Akonadi to handle background tasks like folder synchronization as background tasks.
Comment 5 Martin Steigerwald 2020-06-05 16:08:00 UTC
This still happens with Akonadi 5.14.1 (20.04)
Comment 6 Erik Quaeghebeur 2020-06-05 20:06:45 UTC
(In reply to Martin Steigerwald from comment #4)
> 1) Why does it synchronize the folder anyway when I hit delete the folder
> mail list several times? To make sure the IMAP server has done its work?
If that would be the case, it would be doing needless work: the IMAP server will provide a response to the deletion (expunge?) request and that should be enough (unless of course the server responds with an error).

What I'm wondering is whether KMail can be involved here: perhaps KMail requests a synchronization?
Comment 7 Martin Steigerwald 2020-06-06 07:41:00 UTC
(In reply to Erik Quaeghebeur from comment #6)
> (In reply to Martin Steigerwald from comment #4)
> > 1) Why does it synchronize the folder anyway when I hit delete the folder
> > mail list several times? To make sure the IMAP server has done its work?
> If that would be the case, it would be doing needless work: the IMAP server
> will provide a response to the deletion (expunge?) request and that should
> be enough (unless of course the server responds with an error).

I do not have a KMail/Akonadi setup with a large IMAP folder at the moment. But I tested it again with POP3. Just hitting delete inside the mail list of the trash folder which had new spam and KDE CI messages in it. After I hit about five times, it was displaying that it synchronized the trash folder although Akonadi should have been aware of what it deleted and what it did not delete and the BTRFS filesystem certainly did not say "Hey, I am not going to delete that file just to annoy you".

I could temporarily setup such an IMAP account again tough.

As for IMAP I heard that some of the folder synchronization is done for two reasons:

1) Sometimes the connection to the IMAP server may drop or be interrupted. But I am pretty certainly that this was not the case here. It was with a Dovecot based IMAP server that is even hosted in the same city and only serves me and a few friends.

2) It can be that some other IMAP mail client does something at the same time. But then not hitting the delete key several time should trigger a re-sync. Basically it would suffice to trigger a re-sync when I click on the folder to see its contents and probably there has been no re-sync after some time. Ideally Akonadi would use some IMAP thing to find out whether there have been changes to begin with, but I am not sure whether the IMAP protocol allows for that.

In any way I never saw another IMAP client behaving in such a way and I never saw another IMAP client *blocking* out users requests for the payload of a mail during a re-sync. Either they re-sync or not, but if they do they do it in a much less intrusive way. But again this is another issue I reported (bug #367892). These two bugs combined for local mail access with the recent regression in maildir resource folder synchronization speed (bug #422336) and reliability (bug #422490) make a "wait for Akonadi" game out of KMail. Which is a pity, cause KMail is an awesome mail client.

> What I'm wondering is whether KMail can be involved here: perhaps KMail
> requests a synchronization?

I doubt it. I think this is all happening within Akonadi. But I don't know for sure.