Bug 397316

Summary: akonadi loses connection with IMAP server after suspend (but not only, DB error too)
Product: [Frameworks and Libraries] Akonadi Reporter: yves.caniou
Component: IMAP resourceAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: normal CC: asturm, dilfridge, dvratil, matija, miso, muesli
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Gentoo Packages   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description yves.caniou 2018-08-09 13:17:53 UTC
Hi,

I have redundant problems with kmail/akonadi, see:
    https://bugs.gentoo.org/661856

kmail shows it contacts IMAPS server (progression bar), but actually does not or does not show newer mails. And access to mails (even local) can be [too] long (minutes): I have to make many akonadictl fsck/restart, and each time, konsole reports a lot of errors.

Many errors such as:
Cleaning up missing external file: /home/ycaniou/.local/share/akonadi/file_db_data/47/794047_r0 for item: 218189 on part: 794047
org.kde.pim.akonadicore: Creating/updating items from the akonadi database failed: ""
org.kde.pim.akonadicore: ""
org.kde.pim.akonadicore: Creating/updating items from the akonadi database failed: "Multiple merge candidates, aborting"
org.kde.pim.akonadicore: Error during ItemSync:  "Multiple merge candidates, aborting"
org.kde.pim.akonadiserver: DATABASE ERROR:
org.kde.pim.akonadiserver:   Error code: "1205"
org.kde.pim.akonadiserver:   DB error:  "Lock wait timeout exceeded; try restarting transaction"
org.kde.pim.akonadiserver:   Error text: "Lock wait timeout exceeded; try restarting transaction QMYSQL3: Unable to execute statement"
org.kde.pim.akonadiserver:   Query: "UPDATE PimItemTable SET atime = ? WHERE ( ( PimItemTable.id = ? ) )"
org.kde.pim.akonadiserver: Unable to update item access time
org.kde.pim.akonadiserver: DATABASE ERROR:
org.kde.pim.akonadiserver:   Error code: "1205"
org.kde.pim.akonadiserver:   DB error:  "Lock wait timeout exceeded; try restarting transaction"
org.kde.pim.akonadiserver:   Error text: "Lock wait timeout exceeded; try restarting transaction QMYSQL3: Unable to execute statement"
org.kde.pim.akonadiserver:   Query: "UPDATE PimItemTable SET rev = ?, remoteId = ?, remoteRevision = ?, gid = ?, collectionId = ?, mimeTypeId = ?, datetime = ?, atime = ?, dirty = ?, size = ? WHERE ( id = ? )"
org.kde.pim.akonadiserver: Error during updating record with id 218741  in table "PimItemTable" "Lock wait timeout exceeded; try restarting transaction QMYSQL3: Unable to execute statement"
org.kde.pim.akonadiserver: Error during updating record with id 794048  in table "PartTable" "Driver not loaded Driver not loaded"
Cleaning up missing external file: /home/ycaniou/.local/share/akonadi/file_db_data/50/794050_r0 for item: 218190 on part: 794050
org.kde.pim.akonadiserver: Error during updating record with id 794050  in table "PartTable" "Driver not loaded Driver not loaded"
Cleaning up missing external file: /home/ycaniou/.local/share/akonadi/file_db_data/51/794051_r0 for item: 218190 on part: 794051
Found duplicates 60
org.kde.pim.akonadiserver: DATABASE ERROR:
org.kde.pim.akonadiserver:   Error code: "1064"
org.kde.pim.akonadiserver:   DB error:  "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') AND PartTable.storage = 1 )' at line 1"
org.kde.pim.akonadiserver:   Error text: "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') AND PartTable.storage = 1 )' at line 1 QMYSQL: Unable to execute query"
org.kde.pim.akonadiserver:   Query: "SELECT PartTable.id, PartTable.pimItemId, PartTable.partTypeId, PartTable.data, PartTable.datasize, PartTable.version, PartTable.storage FROM PartTable WHERE ( PartTable.pimItemId IN (  ) AND PartTable.storage = :0 )"
Error while deleting duplicates Driver not loaded Driver not loaded

Let me know what I can do to help.
Comment 1 yves.caniou 2018-11-01 08:47:27 UTC
I had to stop akonadi each time I loggued on my computer, like returning from suspend mode (is it normal that akonadi restarts after suspend if it was stopped beforehands?).
If I did not stop it, I realized that the computer turned slow, making a lot of swap. It turned out that akonadi requires a lot of swap space which is encrypted on my computer, thus wasting a lot of cpu too.

Gentoo proposed upgrades for akonadi, so:
Akonadi is v18.04.3
I gave a try to fsck and vacuum, with a renice 19, and see what happens.

fsck lasted a few hours. Revealed many errors.
vacuum is now sticked to "optimizing table CollectionTable..." since ~3days.
No cpu, and akonadiserver is using more than 27.1GB of memory
-> "vacuuming database that'll take some time and require a lot of temporary disk space..." should say that one should had some Gigs to its swap space (I thought /tmp space would be ok. It's not).

How long is "some time"?
There's nothing on the activity of vacuum...

In addition, i tried kmail: 
- kmail lost contact with two smtp servers (M$ and linux) but says it still manages another one and local mails.
- Trying send a mail results in a error message (partly in french, so translated): "There have been problems trying to put the message in sending: unable to append item flags."

It's been several months now.
What can I do?
Comment 2 Christian Muehlhaeuser 2019-03-23 03:37:49 UTC
Same here, happy to provide more logs. Restarting the akonadiserver only lasts for a few minutes here, then I'm back to square 1.
Comment 3 Matija Šuklje 2019-06-18 13:39:48 UTC
I seem to be getting the same issue on NixOS with Akonadi 18.12.3.

Currently running `akonadictl fsck && akonadictl vacuum` and it’s been an hour at least already.
Comment 4 Matija Šuklje 2019-06-19 10:55:30 UTC
I suspect that this might be related to (and fixed by): https://phabricator.kde.org/D21455

My current workaround is to kill kmail, set the IMAP resource in Akonadi Console as Offline, and use Trojita until this gets fixed. Hope that helps at least a bit.
Comment 5 Daniel Vrátil 2019-06-19 11:16:04 UTC
There seem to be multiple issues mixed in here:

1) Multiple Merge Candidates error (bug #338658) - we will have automatic recovery in 19.08 release.
2) Transaction timeout/deadlock - this should be fixed in 19.04 already through various fixes from David.
3) SQL error when running akonadictl fsck - fixed in 18.08.3 (commit 89727ad4da4d)
4) IMAP/SMTP getting stuck - this should mostly be fixed in 19.04 by the transaction timeout fixes
5) akonadi fsck/vacuum running while database is in use - there's a task for that (https://phabricator.kde.org/T9753)

Note that vacuum never fixes anything (it's just supposed to optimize the database for index performance and storage size) and fsck is not a solution to unstuck Akonadi if it stops syncing.

All in all, I think this can be closed, if you hit any of the described issues, please file separate bug reports.