On akonadictl vacuum my machine hung – which may a BTRFS hanging issue, thus not covered here – I rebooted. Database was broken due to zero byte file – which IMHO is a bug in either MySQL or Akonadi, cause zero byte file with delayed allocation in filesystem would only happen if someone does not fsync() properly, a separate bug report. As Akonadi was slow anyway, I just removed ~/.local/share/akonadi as well as ~/local/share/baloo as well as ~/.kde/share/config/baloo* – as Baloo was broken too, crash on startup again and crash in KMail on search, which is now also fixed… anyway the point this bug report is about: Akonadi / KMail breaks all folder assignments: - Filters point to wrong folders - Identity folders for Sent, Drafts, Templates point to wrong folders. - POP3 resource incoming folder points to wrong folder. I know the technicalities behind it. But from a user point of view this is a bug. I think this needs fixing! That a database loss never happens cannot be guarenteed. Thus the software should handle this case gracefully. Reproducible: Always Steps to Reproduce: 1. Loose database, for example by akonadictl vacuum and switch of the machine 2. … or even just remove ~/.local/share/akonadi Actual Results: Folder assignments are broken. Which has the following dire data loss consequences if the user doesn´t intervene in time – consider automatic retrieval intervalls: 1. Mails are sorted to the wrong folders, which may even lead to mails for POP3 account being sorted to an IMAP acount. 2. Mails are downloaded to the wrong folders. 3. Sent mails, drafts, templates are stored in wrong folders. I consider this to be *data loss*. Expected Results: At least: - KMail / Akonadi *invalidates* folder assignments to avoid that unexpected results happen Better: - For Maildir resource: Akonadi goes the route of Baloo an uses extented attributes or even just a little file inside each mail folder to store a UUID for the folder. - For IMAP: Akonadi somehow stores a folder ID with the IMAP server. Is there a way to do that with all current mail servers out there? Please don´t just close it. This has potential of data loss and at least causes a user with a complex mail setup an hour or more to fix stuff back into operation. Really, whatever the concept in Akonadi is: This *is* a bug. Thats at least my oppinion. Its a behaviour I wouldn´t feel good to let any user be confronted with.
Given that one can re-import filter settings and KMail then tries to select right folders by name… I think there should be something more intelligent possible than "oops, we lost your data, fix all your filter rules and identity and pop3 folder settings manually and don´t forget any of those". But even just at least *invalidating* folder assignments if the database changes would help here to avoid data loss by executing *wrong* filter rules.
For reference: [Akonadi] [Bug 336582] New: Sudden write interruption during akonadictl vacuum causes database corruption
I have the same problem over several installations of KMail/Akonadi, with MySQL as a standalone server and with the dedicated MySQLd, and for **years** now. The bug appeares on different hardware, after reinstallation, etc. Sooner or later, with no discernible reason for me, Akonadi mixes up the folders. Suddenly the sent mail folder of one account is lost, for another account the trash folder is the inbox folder of another account, or, strange enough, an ownCloud CalDAV account becomes the outbox of an Imap account. Expire dates and so on also get messed up which leads important mails getting silently deleted. We are talking about *** data loss *** here.
Same as bug#283345 I guess I have a similar one with kaddressbook bug#357266 - and my guess is that the application uses short numeric ids to reference akonadi database records which, after a rebuild, have completely different ids at random. Just a hunch.
Also bug#340529
David, that is exactly what happens. It is known to the developers. Challenge is to get unique ids working with all Akonadi resources, say for example IMAP. How do uniquely identify a mail with IMAP? Always check for the message-id. We discussed this on KDE Randa Meetings last year and we didn´t come to an easy implementable solution so far. I do not recall all of the discussion. If you want to move this further along I suggest you bring up the topic on kde-pim mailing list. Other than that thank you that you create bug relations. There are several duplicates of it. I am pretty sure I reported it as well..
This bug has only been reported for versions older than KDEPIM 4.14 (at most akonadi-1.3). Can anyone tell if this bug still present? If noone confirms this bug for a recent version of akonadi (part of KDE Applications 15.08 or later), it gets closed in about three months.
Just as announced in my last comment, I close this bug. If you encounter it again in a recent version (at least 5.0 aka 15.08), please open a new one unless it already exists. Thank you for all your input.
Thanks, Denis. This is a principal issue of the current Akonadi design and still present, as I am not aware of any design change. Thus reopening. There is no need to open a new bug report about it.
This is indeed still the case with KMail 5.2.3 (Debian Unstable unfortunately does not have any newer packages due to the difficulty to properly package Qt Webengine in order to enable newer KDEPIM versions, which likely means that Debian will go with KDEPIM 16.04 for next stable version Stretch). The principal issue of relying on database IDs outside of the database itself and thus using wrong folders in case of a database loss, potentially causing data loss, is still present. For mailfilters: ~/.config> grep action- akonadi_mailfilter_agentrc | head -20 […] action-args-0=95 action-name-0=transfer action-args-0=209 action-name-0=transfer action-args-0=209 action-name-0=transfer action-args-0=208 action-name-0=transfer action-args-0=211 action-name-0=transfer action-args-0=422 action-name-0=transfer Action "transfer" means to move the mail into a folder. The corresponding args-0 is the numerical id of the folder. For folder settings: ~/.config> fgrep '[Folder-' kmail2rc | head [Folder-1] [Folder-10] [Folder-100] [Folder-101] [Folder-102] [Folder-103] [Folder-104] [Folder-105] [Folder-106] [Folder-107] For default folders for mail identities: ~/.config> egrep -i "draft|template" emailidentities | head -10 Drafts=21 Templates=31 Drafts=21 Templates=31 Drafts=21 Templates=31 Drafts=21 Templates=31 Drafts=411 Templates=378 Default inbox folder for POP mail resources: martin@merkaba:~/.config> grep targetCollection akonadi_pop3_resource_0rc targetCollection=361 I bet is similar for IMAP resources. My expected result is this: 1) Either keep / rebuild / reassign references after a database loss properly *or* 2) *loose* the references and require the user to reconfigure. Yet *never ever* use *wrong* references, just do to a database loss. Use a unique hash for identification, not just the index number of the database entry. In that way you can at least *know* when the reference is no more valid. *Or* store *all* of these reference into the database itself, so that they are gone for good if the database is rebuilt. For user convenience I would aim for approach 1 wherever possible which is easier to do with a hash based approach. For maildir resource, you can store the folder identification in the maildir filesystem, or IMAP it becomes more difficult tough, yet you could at least also store the name and then if the reference by hash is gone, you can show the folder name to the user and possibly even autoselect it for confirmation. You can argue that a database loss should not happen and with proper MariaDB configuration, proper filesystem and proper storage media you might well be right about that – *yet* in practice it did happen to a lot of users. So please make Akonadi *failure proof* and more *robust* in cases like this. Another approach would be to use something else than a database for metadata or rethink the Akonadi caching concept completely like Aaron and others do with Sink for Kubemail. I don´t see this happening for KMail from KDEPIM soon, so I think it makes sense to make it more failure proof even before that may happen.
A short comment on this, since Martin perfectly summarizes my frustration. A user's opinion, I am not a developer. Since introduction of akonadi, I experience broken database issue about 1-2 times a Year (last time some days ago) and none of the proposed solutions anywhere in posts resolves my given issues. Because I don't have the time for extensive investigations I have to delete ~/.local/share/akonadi which then totally mixes up folder associations with the described impacts on filters etc. as described above. Email is among the most critical applications for daily use and this weakness in architecture (as I learned from above) is a total mess. This is not a bug, but a totally broken architecture !!! Martin's suggestions seem already very reasonable to me, and promising.
As to broken architecture... The dreaded KMail hanging with the message "Retrieving message/ folder content screen" showing indefinitely is back. On top of that KMail now always crashes when writing a new email using autofill for the receiver (start typing the email address and when trying to select one of the suggestions made, KMail will crash). I will search if a bug report has been filed for that one and if not I will file it myself. KMail/ Akonadi had been "OK-ish" for a couple of months (to the extent I even removed my backup solution of Thunderbird), but it is back to being near useless again.
(In reply to Frits Spieker from comment #12) Dear Frits, please refrain from writing unrelated stuff into this bug report. The bug report is about the effects of accidental database loss on folder assignments and nothing else. It is not a discussion forum or a mailing list. Using it for any random comment makes it more difficult to read by any of the developers. Please use KDE Forum or preferably kdepim-users mailing list to vent your frustration. Have a Merry Christmas, Martin
Just to confirm that the issue is still there for Kmail 5.20.2. The akonadi database gets corrupted randomly (happened to me after a distro upgrade that otherwise went perfectly fine) and after removing Akonadi's corrupted data folders and syncing again, all folder associations were messed up. Can't way for the day when our whole company can start relying on KDE PIM's framework. It looks very promising and the vision behind akonadi is brilliant, but this data corruption issue is a deal breaker. I hope you manage to fix this soon, stability and durability are so important for this kind of application. If a solution is known but what is lacking is manpower, please let the community know so that we can try to organize sponsorship. This is such a critical issue (I am referring to the data corruption. Folder assignment is more a symptom. If data corruption wasn't there in the first place, folder assignment mix up wouldn't be an issue. I feel it would probably be better to fix the fragility of the storage system to make such symptoms non-issues by removing a vast class of failure modes rather than rather than trying to come up with global ids).