I created a folder in "Local Folders" called "Storage" and a subfolder called "Old" Now I move a few hundred mails to that subfolder. These mails should be located on my harddisk in ".local/share/.local-mail.directory/.Storage.directory/Old" but they are not. I see them in kmail, I can delete or copy them but they do not appear on the harddisk. The folder ".local/share/.local-mail.directory/.Storage.directory/Old" does exist but it is empty (apart from the three empty folders "new", "cur" and "tmp" of course) No further hidden directories show up. Now where is my mail? How can I physically locate it if it is not written to the disk (which it should!)? How can I make a backup of my mailfolders if the mails are not physically in there? Are they in a kind of cache somewhere hidden within the akonadi database? Reproducible: Always Steps to Reproduce: 1. create a subfolder of a new folder in "Loacl Folders" in kmail... 2. copy mails into that subfolder 3. try to locate these folders on your disk Actual Results: No mails to be found... Expected Results: Mails should be sitting somewhere in "local-mail" or ".local-mail.directory" By the way if I physically copy mails in the subfolder ".local/share/.local-mail.directory/.Storage.directory/Old" these mails show up inkmail just fine beneath all the other mails that have no presence in that folder... This is an absolute awful bug because it makes handling mail out of kmail near to impossible (and kmail indicates in no manner at all where the mail is physically stored) If kmail would be a rocksolid system that works just fine with internal backups I might be reluctantly ok with that but kmail breaks all the time so at least the physical mail locations should be transparent for easy backups and the possibility to import everything into other mail programs. Even more annoying I also have folders in kmail with so called "invalid entries". These folders cannot be deleted within kmail and since they are not found physically I can#T delete them elsewhere either.
I can confirm this bug and would call it severe, since it can cause the loss of mails. @piedro Your mails are probably in ~/.local/share/akonadi/file_db_data Further reading about what is happening there you'll find in these bug reports and forum posts: https://bugs.kde.org/show_bug.cgi?id=333514 https://forum.kde.org/viewtopic.php?f=215&t=122797 My case is similar and to a certain extend worse. Since Kmail2 and Akonadi are not very reliable I keep besides my local archive on the hard disk backup copies of all archived mails on my mail server for a while. These copies as well as my inbox, sent mails folder etc. I access by IMAP. All these mails do not appear in the directory tree under ~/.local/share/.local-mail.directory/ but are stored in above mentioned directory and the Akonadi database as much as I understood. When I archive mails I copy them from my inbox or send folder (IMAP) to the backup folder (IMAP) and then I move them from the inbox or send folder (IMAP) to a subfolder of my local archive which is stored under ~/.local/share/.local-mail.directory/. But not all of the moved mails physically appear in the archive folders. Even though I could access them in kmail they were not stored in the respective directories under ~/.local/share/.local-mail.directory/. But it got worse when my home directory ran out of space. After a short investigation into the matter I found out that ~/.local/share/akonadi/file_db_data had reached almost 3 GB and held more than 15.000 files, most of them emails which were up to two years old. Besides there was a huge file_lost+found directory holding almost 3.000 files. After reading above mentioned posts I found it save to delete the latter. I also deleted ~/.local/share/akonadi/file_db_data after making a backup copy. This did not seem to have any of the negative effects described in the forum post. I could access all my mails in the IMAP folders and no body was missing. But when I searched for locally archived mails I saw, that some of the mail bodies were empty. That was not true for the copies of these mails in the IMAP backup folder. Digging a bit deeper into the matter I found out, that there were no files for the empty mails in the subdirectories of ~/.local/share/.local-mail.directory/. They obviously had not been moved or copied there when I moved the mails from the IMAP folder to the local archive. After playing back the 3GB file_db_data directory - and all other akonadi files backed-up with it - I got back again the contents of all these mails. Big drawback of this "solution": my home directory was running out of space again and I had lost all notes attached to mails after the backup. I eventually found a better solution. Since only my locally stored mails were affected and not the contents of the IMAP folders and I had not stored anything locally after deleting the file_db_data directory I just used the kmail archive function to archive the local "archive folder" as a tar.bz. Then I went back to the before saved current state of Akonadi, deleted the corrupted archive and imported the tar.bz file. This brought back my complete archive folder with all missing mail bodies. The afore missing mail files were, as expected, stored in the subfolders of ~/.local/share/.local-mail.directory/ too. Only drop of bitterness, a few thousand mails were marked "unread" now. After rescanning my local mail archive - which temporally let grow file_db_data up to 10.000 files again - I could not reproduce this behaviour anymore. Everything seems to work as expected now. Right now I'm running KDE and Kontact 4.14.5 as well as Akonadi 1.13.0 under Chakra Linux. My system always is kept up-to-date. The last update of KDE-Pim and Akonadii was on March 4th 2015, a day before I deleted ~/.local/share/akonadi/file_db_data.
thx for your clarification. Though since other solutions involved deleting the .local/share/akonadi folder I have to browse through old backups to see what is still there - probably old stuff, duplicatesand so forth. Let's assume I find an old backup file_db_data folder... what exactly would i have to do to intergrate these database files into a my kmail structure without deleting everything else? Is there a way to import these lost mails into a folder? thx, piedro
(In reply to piedro from comment #2) > Let's assume I find an old backup file_db_data folder... what exactly would > i have to do to intergrate these database files into a my kmail structure > without deleting everything else? Is there a way to import these lost mails > into a folder? What I did I had tried to describe in my above post. In case you find a backup made after moving your mails to Storage/Old and shortly before deleting the Akonadi stuff I would do the following: 1. exit all applications using Akonadi like Kontact (Kmail etc.), Knotes, Kalarm etc. 2. stop Akonadi 3. make a backup copy of your current ~/.local/share/.local-mail.directory/ and ~/.local/share/akonadi/ - just to be on the save side in case something goes wrong In the forum post a developer recommended deleting other Akonadi related stuff. So I would suppose it to be wise to make backup copies of these files too. I did not. 4. play back the old Akonadi stuff as well as the old ~/.local/share/.local-mail.directory/ and ~/.local/share/akonadi/ directories 5. start Akonadi and Kmail 6. look for your Storage folder in Local Folders and make a right click. You will be offered to create a backup archive. Do it. 7. go back to the current state of Akonadi and local maiil folder by playing back the backup copies. Don't forget to exit all Akonadi related stuff and stop Akonadi before. 8. Start Akonadi and Kmail again and delete all the stuff in Local Folders you had exported before. 9. import the afore made backup copy of your stored mails. After this you should find them all in ~/.local/share/.local-mail.directory/. I actually never replaced ~/.local/share/.local-mail.directory/ with an old copy since it had not been touched meanwhile. The good thing about all this: after Akonadi had rescanned all ~/.local/share/.local-mail.directory/ again, the problem seemed to have vanished. At least with regard to mails. But there are still loads of mails and other stuff held in the file_db_data directory. This seems to be related the the other bug I mentioned . I will post there too. Hope I could help and you will be able to recover your mails.
> After rescanning my local mail archive - which temporally let grow file_db_data up to 10.000 files again - I could not reproduce this behaviour anymore. Everything seems to work as expected now. Well, bad news. What I wrote recently is not one hundred percent true. I did some testing copying one or more messages at a time. Everything looked alright. But when I moved a whole thread today only the first message was actually copied physically, even though all messages showed up in "Local Folders". After more testing I found out that only the first message is physically copied to the subfolders of ~/.local/share/.local-mail.directory when several messages are moved. This explains why not all messages were missing in my local archive. As I described before, my usual procedure is moving message or whole threads of messages from the IMAP accounts after making a backup copy there. BTW, archiving the affected folder and importing it after deleting the original copied all the mail files into the ~/.local/share/.local-mail.directory.
I would call it severe since it does lead to loss of e-mail. It happened here: For some reason kmail does not save mails that were copied from an IMAP account to subfolders of "Local Folders". To be more precise: These subfolders (together with the mail that they contain) have vanished from ~/.local/share/local-mail. They were there a few weeks ago, as backups show. I have not noticed the different behaviour, since, in kmail, subfolders and mail were visible alright. Obviously, these were read from the akonadi databases but not from ~/.local/share/local-mail. When the akonadi database becomes corrupt, the mails can't be recovered. Unfortunately, this does happen. This is on opensuse 42.1 with KDE 4.11.2
> This is on opensuse 42.1 with KDE 4.11.2 Sorry: KDE 4.14.17
Guido: Still very old. Yet, in this case it likely doesn´t matter: Do you see "item without rid" messages during an akonadictl fsck? I do think what you see is basically Bug 360834 - no mechanism to reattempt to store items without rid (just in db) into the resource So its currently still also in the newest Akonadi versions. Can you verify that you have items without rid?
(In reply to Martin Steigerwald from comment #7) > Can you verify that you have items without rid? "Found 6567 items without RID" [...] "Found 91 dirty items" Although Akonadi was up and running, fsck only created output after restart of Akonadi. Looks like Bug 360834, thanks!
Thanks, Guido, that confirms it. So basically my bug report is a duplicate of this one, but as my bug report already contains a clear description on whats the cause, I will mark this one as a duplicate of it. (piedro, no offense meant, I know your report is sooner:). For now there is a simple rule: Do *not* wipe your database *ever*. Otherwise you would loose that items. Keep regular backups of both the resource data + the database. Backup the database while Akonadi + MySQL are not running. *** This bug has been marked as a duplicate of bug 360834 ***
None offense taken, I am happy to see some progress after all. This should have a very high priority as most forums, threads and websites dealing with akonadi problems explicitely recommend deleting your akonadi folder to remedy a corrupted database! thx, p.