| Summary: | Mails that are copied do not appear on my harddrive | ||
|---|---|---|---|
| Product: | [Applications] kmail2 | Reporter: | piedro <piedro.kulman> |
| Component: | folders | Assignee: | kdepim bugs <kdepim-bugs> |
| Status: | RESOLVED DUPLICATE | ||
| Severity: | normal | CC: | danielroschka, guido.pinkernell, Martin, piedro.kulman, post |
| Priority: | NOR | ||
| Version: | 4.14.4 | ||
| Target Milestone: | --- | ||
| Platform: | Arch Linux | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | ||
| Sentry Crash Report: | |||
|
Description
piedro
2015-02-16 14:41:00 UTC
I can confirm this bug and would call it severe, since it can cause the loss of mails. @piedro Your mails are probably in ~/.local/share/akonadi/file_db_data Further reading about what is happening there you'll find in these bug reports and forum posts: https://bugs.kde.org/show_bug.cgi?id=333514 https://forum.kde.org/viewtopic.php?f=215&t=122797 My case is similar and to a certain extend worse. Since Kmail2 and Akonadi are not very reliable I keep besides my local archive on the hard disk backup copies of all archived mails on my mail server for a while. These copies as well as my inbox, sent mails folder etc. I access by IMAP. All these mails do not appear in the directory tree under ~/.local/share/.local-mail.directory/ but are stored in above mentioned directory and the Akonadi database as much as I understood. When I archive mails I copy them from my inbox or send folder (IMAP) to the backup folder (IMAP) and then I move them from the inbox or send folder (IMAP) to a subfolder of my local archive which is stored under ~/.local/share/.local-mail.directory/. But not all of the moved mails physically appear in the archive folders. Even though I could access them in kmail they were not stored in the respective directories under ~/.local/share/.local-mail.directory/. But it got worse when my home directory ran out of space. After a short investigation into the matter I found out that ~/.local/share/akonadi/file_db_data had reached almost 3 GB and held more than 15.000 files, most of them emails which were up to two years old. Besides there was a huge file_lost+found directory holding almost 3.000 files. After reading above mentioned posts I found it save to delete the latter. I also deleted ~/.local/share/akonadi/file_db_data after making a backup copy. This did not seem to have any of the negative effects described in the forum post. I could access all my mails in the IMAP folders and no body was missing. But when I searched for locally archived mails I saw, that some of the mail bodies were empty. That was not true for the copies of these mails in the IMAP backup folder. Digging a bit deeper into the matter I found out, that there were no files for the empty mails in the subdirectories of ~/.local/share/.local-mail.directory/. They obviously had not been moved or copied there when I moved the mails from the IMAP folder to the local archive. After playing back the 3GB file_db_data directory - and all other akonadi files backed-up with it - I got back again the contents of all these mails. Big drawback of this "solution": my home directory was running out of space again and I had lost all notes attached to mails after the backup. I eventually found a better solution. Since only my locally stored mails were affected and not the contents of the IMAP folders and I had not stored anything locally after deleting the file_db_data directory I just used the kmail archive function to archive the local "archive folder" as a tar.bz. Then I went back to the before saved current state of Akonadi, deleted the corrupted archive and imported the tar.bz file. This brought back my complete archive folder with all missing mail bodies. The afore missing mail files were, as expected, stored in the subfolders of ~/.local/share/.local-mail.directory/ too. Only drop of bitterness, a few thousand mails were marked "unread" now. After rescanning my local mail archive - which temporally let grow file_db_data up to 10.000 files again - I could not reproduce this behaviour anymore. Everything seems to work as expected now. Right now I'm running KDE and Kontact 4.14.5 as well as Akonadi 1.13.0 under Chakra Linux. My system always is kept up-to-date. The last update of KDE-Pim and Akonadii was on March 4th 2015, a day before I deleted ~/.local/share/akonadi/file_db_data. thx for your clarification. Though since other solutions involved deleting the .local/share/akonadi folder I have to browse through old backups to see what is still there - probably old stuff, duplicatesand so forth. Let's assume I find an old backup file_db_data folder... what exactly would i have to do to intergrate these database files into a my kmail structure without deleting everything else? Is there a way to import these lost mails into a folder? thx, piedro (In reply to piedro from comment #2) > Let's assume I find an old backup file_db_data folder... what exactly would > i have to do to intergrate these database files into a my kmail structure > without deleting everything else? Is there a way to import these lost mails > into a folder? What I did I had tried to describe in my above post. In case you find a backup made after moving your mails to Storage/Old and shortly before deleting the Akonadi stuff I would do the following: 1. exit all applications using Akonadi like Kontact (Kmail etc.), Knotes, Kalarm etc. 2. stop Akonadi 3. make a backup copy of your current ~/.local/share/.local-mail.directory/ and ~/.local/share/akonadi/ - just to be on the save side in case something goes wrong In the forum post a developer recommended deleting other Akonadi related stuff. So I would suppose it to be wise to make backup copies of these files too. I did not. 4. play back the old Akonadi stuff as well as the old ~/.local/share/.local-mail.directory/ and ~/.local/share/akonadi/ directories 5. start Akonadi and Kmail 6. look for your Storage folder in Local Folders and make a right click. You will be offered to create a backup archive. Do it. 7. go back to the current state of Akonadi and local maiil folder by playing back the backup copies. Don't forget to exit all Akonadi related stuff and stop Akonadi before. 8. Start Akonadi and Kmail again and delete all the stuff in Local Folders you had exported before. 9. import the afore made backup copy of your stored mails. After this you should find them all in ~/.local/share/.local-mail.directory/. I actually never replaced ~/.local/share/.local-mail.directory/ with an old copy since it had not been touched meanwhile. The good thing about all this: after Akonadi had rescanned all ~/.local/share/.local-mail.directory/ again, the problem seemed to have vanished. At least with regard to mails. But there are still loads of mails and other stuff held in the file_db_data directory. This seems to be related the the other bug I mentioned . I will post there too. Hope I could help and you will be able to recover your mails. > After rescanning my local mail archive - which temporally let grow file_db_data up to 10.000 files again - I could not reproduce this behaviour anymore. Everything seems to work as expected now.
Well, bad news. What I wrote recently is not one hundred percent true. I did some testing copying one or more messages at a time. Everything looked alright. But when I moved a whole thread today only the first message was actually copied physically, even though all messages showed up in "Local Folders". After more testing I found out that only the first message is physically copied to the subfolders of ~/.local/share/.local-mail.directory when several messages are moved. This explains why not all messages were missing in my local archive. As I described before, my usual procedure is moving message or whole threads of messages from the IMAP accounts after making a backup copy there.
BTW, archiving the affected folder and importing it after deleting the original copied all the mail files into the ~/.local/share/.local-mail.directory.
I would call it severe since it does lead to loss of e-mail. It happened here: For some reason kmail does not save mails that were copied from an IMAP account to subfolders of "Local Folders". To be more precise: These subfolders (together with the mail that they contain) have vanished from ~/.local/share/local-mail. They were there a few weeks ago, as backups show. I have not noticed the different behaviour, since, in kmail, subfolders and mail were visible alright. Obviously, these were read from the akonadi databases but not from ~/.local/share/local-mail. When the akonadi database becomes corrupt, the mails can't be recovered. Unfortunately, this does happen. This is on opensuse 42.1 with KDE 4.11.2 > This is on opensuse 42.1 with KDE 4.11.2
Sorry: KDE 4.14.17
Guido: Still very old. Yet, in this case it likely doesn´t matter: Do you see "item without rid" messages during an akonadictl fsck? I do think what you see is basically Bug 360834 - no mechanism to reattempt to store items without rid (just in db) into the resource So its currently still also in the newest Akonadi versions. Can you verify that you have items without rid? (In reply to Martin Steigerwald from comment #7) > Can you verify that you have items without rid? "Found 6567 items without RID" [...] "Found 91 dirty items" Although Akonadi was up and running, fsck only created output after restart of Akonadi. Looks like Bug 360834, thanks! Thanks, Guido, that confirms it. So basically my bug report is a duplicate of this one, but as my bug report already contains a clear description on whats the cause, I will mark this one as a duplicate of it. (piedro, no offense meant, I know your report is sooner:). For now there is a simple rule: Do *not* wipe your database *ever*. Otherwise you would loose that items. Keep regular backups of both the resource data + the database. Backup the database while Akonadi + MySQL are not running. *** This bug has been marked as a duplicate of bug 360834 *** None offense taken, I am happy to see some progress after all. This should have a very high priority as most forums, threads and websites dealing with akonadi problems explicitely recommend deleting your akonadi folder to remedy a corrupted database! thx, p. |