Bug 362715 - Imap constant refresh of IMAP directory
Summary: Imap constant refresh of IMAP directory
Status: REPORTED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: IMAP resource (show other bugs)
Version: 5.1
Platform: Ubuntu Linux
: NOR normal with 20 votes (vote)
Target Milestone: ---
Assignee: Christian Mollekopf
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-05-05 19:26 UTC by Michael Butash
Modified: 2018-03-20 20:55 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Butash 2016-05-05 19:26:41 UTC
I'd upgraded from 14.04 ubuntu to 16.04, and with it kde5.  I've been attempting to migrate from thunderbird to kmail, but tying in my primary account, began noticing it having problems with constantly synching, particularly one directory.  I get a count mismatch every time, and it just starts over, which happens to be some 26k messages from some perpetual customer spam in the past year.

Starting and stopping akonadi from cli, I see it hitting the same directory constantly that the local count mismatch doesn't match, tries again, over and over:

log_imapresource: Detected inconsistency in local cache, we're missing some messages. Server:  26320  Local:  25004
log_imapresource: Refetching complete mailbox.

It begins its fetch, gets to the end:

...
Received:  11 In total:  24989  Wanted:  25098
Received:  11 In total:  25000  Wanted:  25098
Wrote 4357 bytes to  "/home/$dir/.local/share/akonadi/file_db_data/26/1088326_r813"
finished
localListDone:  false  deliveryDone:  true
localListDone:  true  deliveryDone:  true
Nothing to do
Received:  0 Removed:  0 In total:  0  Wanted:  -1
finished
...
Received:  0 Removed:  0 In total:  0  Wanted:  -1
finished
log_imapresource: Detected inconsistency in local cache, we're missing some messages. Server:  25098  Local:  25004
log_imapresource: Refetching complete mailbox.
25098
Received:  4 In total:  4  Wanted:  25098

... and starts over, again and again, collecting tons of email headers in a steady 15mbps suck.

Watching my bandwidth, it generates a good 14mbps or so of traffic, and this has been going on for a week or so, so amazed someone hasn't shut me down yet.

Thunderbird handles this directory just fine, not sure why kmail has this issue, or obscurely related to some other component I'm not finding.

I'd even moved the items at a webmail level into another directory, and the problem persisted. I suspect the imap server (godaddy, go figure) really is doing something bad like not indexing them properly, or missing indices perhaps, but akonadi:imap is misbehaving epically like a thrashing child hogging a good clip of bandwidth someone is paying for (yay for unlimited bandwidth on cable still here).

I'm going to largely purge that directory, but wanted to see if I could offer anything to help fix this before doing so, but mostly it seems behavioral with akonadi imap collector not knowing when to stop after x failed attempts at the same intensive collection routine.

This also seems to cause email not to be actually commited, as I seem to not always get all mail, and not always send all mail.  It's very spotty, but I'll notice usually mail will stop and start collecting at odd intervals, presumably as a result of this folder constantly refreshing as a priority.

Reproducible: Always

Steps to Reproduce:
1. Create an imap account with bad indices in its imap folder email counts (pulls 25k emails, server reports 26k).
2. Start Akonadi from command line with akonadictl, watching the imap collection on the errant folder as it pulls headers, finds mismatch, tries again.
3. Watch Akonadi incessantly try to get the proper count in futility, using all of your bandwidth constantly.

Actual Results:  
Akonadi never completes polling the folder, just trying over and over, using a very large amount of bandwidth constantly and perpetually until akonadi is force shutdown.

Expected Results:  
* Recover from a bad index automatically (if possible)
* Offer to correct bad index on server (if possible)
* Exit and give error condition of unstable directory structure, server problem
* Simply ignore the index being bad, grab what it can grab, and poll emails normally around the bad index (seems to be thunderbird behaviour)

I've left this to be somewhat broken, but the fact that kde5 is a bit unstable with multi-monitor support still means I restart often, which respawns akonadi until I notice bandwidth usage being excessive and kill it again.  I can provide better debug, but again, seems more just methodical fixes needed to be agreed upon.