If you copy a mail from input folder to another one, you can see it in the new folder under Kmail, but if you look in the respective maildir sub-folders, it happens very often, that you cannot find this mail in the choosed sub-folder nor in the input one. It is loss. This data loss happens also on creating new sub-folders. They are not written in maildir resource. It happens also under Version 4.13.3 Reproducible: Sometimes Steps to Reproduce: already described in Details Actual Results: Enormous data loss, but I think the lost mail are saved in Akonadi database without folder reference Expected Results: Mail lists in each folder should synced with the data in maildir
I have noticed this too. After some testing I have been able to verify it since version 4.11 and it is still present in 4.14 of kmail (I have tested this using the Debian and openSUSE packages). I think that it is probably a bug in akonadi. I believe that the problem is always reproducible if multiple emails are moved in one operation. To reproduce: 1. Highlight multiple emails (they can be in an imap account or a maildir resource) in kmail. 2. Move or copy them to a local maildir resource, either by dragging them, or via the right click menu. 3. The emails are removed from the origin mentioned in 1 both in kmail and by inspecting the resource manually. 4. All of the emails appear in the maildir when accessed via kmail. 5. Only one of the emails is actually written to the maildir on disk (the others only exist in the akonadi database). This also affects "expiring" a folder to a maildir. In addition to confirming this (and hopefully seeing it resolved), I would like to ask if there is a way to force kmail to write the emails to disk. Currently I have hundreds of emails that only exist in akonadi that I would like to actually have in the maildir. One solution would be to move everything to an imap folder (so that they actually exist somewhere outside of akonadi) but the maildir contains thousands of emails and I am not sure which are missing (moving all of them would be unfeasible).
Can you confirm that the mails won´t appear after *some* time? I believe that Akonadi caches things in database upto SizeThreshold, standard is 4096 bytes, or in ~/.local/share/akonadi/file_db_data for larger items. Only if they are not there and are not moved after some time, where I am totally not sure what that "some time" amounts to, there is really a data loss.
My experience: the mails are not loss in Kmail view, but in the maildir sub-folders, where you have saved them. So you notice about the mails loss, when you need to recover the maildir from a backup. I was able to recover all "loss" mails through importing all mails from akonadi. They was been handled each one as a mail folder: a lot of manual job! And not, they are not reappeared until the hard drive crash.
Giuseppe, next, when it happens, can you grep for some mail content inside ~/.local/share/akonadi/file_db_data and ~/.local/share/akonadi/db_data/akonadi. I think Akonadi has the mail somewhere in there, but *only* in there, for *some* amount of time, until it eventually chooses to save it at its final place, caching it for a while. I do not agree that this kind of write caching is a wise choice, as it as you point out create and issues with backuping Akonadi data, but I do think technically Akonadi may not *loose* the mail, and if you would backup the Maildir *and* ~/.local/share/akonadi *both* and restore *them both* the mail would be there again. What I am into is now: Is this the case or does Akonadi *really* completely loose the mail. I bet it doesn´t cause, how could KMail display a mail that is not physically stored on mass storage media unless Akonadi just holds it in RAM, but then it would have been gone after a reboot. If its the case that technically, Akonadi does not loose the mail, this, and if its the case for the other bug report about something that sounds quite similar [1], the two bugs may be pointing at the same issue: That, as to my knowledge Akonadi may choose to cache a mail after a move for some while just inside ~/.local/share/akonadi and will write it to its final location just after some time. I still think thats worth a bug report, cause it makes what one things may be just a cache which can be rebuilt upon corruption something that is a cache that actually needs to be backuped and restored alongside with Akonadi and causes data loss if its corrupted. I think for robustness Akonadi should not write cache things in there, at least not for long. I think it should write out the moved mails to their final location as soon as possible. But so far I found no option or manual way to trigger this behavior. [1] [Bug 339181] moving mails from cached imap to local folder leaves mails without RID
Martin, thanks for your suggestions about backupping kmail data. I really expected this tip and I had quite written the answer "in advance", in order to explain because it was not helpful for me. But I think kde.bug.org is not a forum area, so I didn't. I think as you, Akonadi does not really loose mails even if some of them are not "moved" in the maildir subfolder as wished. But it needs one or more experimental envirroments to test that definetively. As workaround I operate so: a) I copy the mails to be moved (i.e. from inbox folder) in the choosed subfolder b) I delete the "original" copy (in imbox folder) of copied mails I tested this method a couple of months and I didn't find any "loss". That means all moved mails are really copied in the correspective subfolder. Maybe can the development team "translate" this to a command (as a script?) , which runs every time a mail is moved into a subfolder. Btw. sorry for the uncorrect technical items, I am not a software programmer ;-) About considering this bug as duplicate of [Bug 339181]: again, I am not a software programmer. Yes, the other bug sounds similar, but could it be helpful to mix IMAP items with POP ones?
I've tested this today and I can confirm, it happens yet. I moved five mails in a inbox subfolder. If you look in this subfolder with kmail, you find them, but not with Dolphin (or other file manager), neither in the source folder (inbox) nor in the target subfolder.
Thanks, Giuseppe, if I get this right you are not comfortable with that Akonadi temporarily stores moved mails in database or file_db_data (depending on payload size) instead of moving them directly to the destination folder and that by copying and deleting you instead get an instant move? Well, currently this Akonadi works as designed. Its in the design that Akonadi will also cache *writes* for some amount of time. I don´t know why according to how I understand you it doesn´t do that on copy/delete instead of move, but I know that basically its currently how its working. For some usecases this is may even be needed, i.e. when a backend, like IMAP server, is unreachable and KMail would like to store a sent mail into sent folder of it. If its this, then I suggest I set the report to confirmed but also make it a wishlist item. I also would like Akonadi to *only* be a read cache for all use cases, so that you can safely remove the cache at any time. But currently this is not how Akonadi is designed to work. And for some workloads it may not even be possible to do it this way (like storing mail on IMAP server). So please tell me, whether I got your concern correctly and I will make the suggested changes to the bug report. As far as I got during KDE Randa Meetings the team is aware of this behavior. And there are ideas to change it at least partly. Feel free to discuss on IRC #kontact or mailinglist. (I think the bug should just track the request / wish as such.)
Thank you for reporting this bug in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version? If you can reproduce the issue, please change the status to "CONFIRMED" when replying. Thank you!
No, I can not reproduce the issue with my current software version, 5.14.2 on OpenSuse Leap 15.3
Well, this seems to be a duplicate of or at least related to Bug 374925.
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please mark the bug as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone!