copy mail folder (mixed maildir ressource) to other folder on other ressource (mail dir) results in empty mails Reproducible: Always Steps to Reproduce: 1. create mixed maildir ressource on maildir containing a lot of mail 2. create maildir ressource on empty maildir 3. let kmail copy a folder from mixed maildir ressource to the maildir ressource (takes very long time without progress) Actual Results: folder created with mails and size 1 byte in new sub dir of target folder in file system => Mails lost Expected Results: folder created with mails and real size in cur sub dir of target folder in file system
Git commit ff9edf8c0d96b179116558020d5a571a25142795 by Dan Vrátil. Committed on 05/04/2014 at 21:29. Pushed by dvratil into branch '1.12'. Fix potential severe data loss during copy and move operations Move and copy operations on larger sets of items can take some time, because we need to make sure that all items have full payload stored in Akonadi. This is necessary especially with local resources, like maildir, which have cache timeout set to 5 minutes (after that all their payload is removed from Akonadi and is fetched from HDD on demand, because it's cheap, fast and does not unnecessarily duplicate emails in maildir and in database). However fetching large amount of items via ItemRetriever takes a lot of time, sometimes it can take even more than 5 minutes. And in such case there is a very high chance, that the CacheCleaner will just remove the newly cached payloads from Akonadi again and so when ItemRetriever finishes, many items have empty payload in Akonadi again. ItemRetriever nor handlers are aware of this howerver, so they will just make copies or moves of empty items, causing data loss. This patch introduces CacheCleanerInhibitor, a class which when it is created will pause the timer in CacheCleaner and resume the timer again when it's destroyed (so usually when it goes out of scope). Also, this patch adds the inhibitor to all handlers that use ItemRetriever, so that the the situation described above does not happen. The current solution is not perfect because it pauses the entire CacheCleaner while I think it would be better to be able to temporarily 'blacklist' only specific collections or items. That would however require much more complex code and changes, which makes it unsuitable for 1.12 release. I tried moving 78 000 emails from one maildir to another and all emails were moved correctly. Move itself has many other problems (CPU, IO, memory, KMail responsivness, etc.) but that's out of scope of this fix. M +49 -0 server/src/cachecleaner.cpp M +30 -0 server/src/cachecleaner.h M +94 -1 server/src/collectionscheduler.cpp M +4 -1 server/src/collectionscheduler.h M +3 -0 server/src/handler/colcopy.cpp M +3 -0 server/src/handler/colmove.cpp M +3 -0 server/src/handler/copy.cpp M +3 -0 server/src/handler/fetch.cpp M +4 -0 server/src/handler/modify.cpp M +3 -0 server/src/handler/move.cpp http://commits.kde.org/akonadi/ff9edf8c0d96b179116558020d5a571a25142795
Does this produce 1-byte sized file in target folder? If so: Bug 319226 - produces 1-byte-sized on processing crm114 spam filter rules might (!) be related as well as Bug 331486 - creation of 1000s of 1byte files in inbox/new/ for "ghost" mails Bug 290422 - lost many email while moving them from a local folder to a local folder
Also bug 279260 could be related to this.
*** Bug 290422 has been marked as a duplicate of this bug. ***
Christoph, thank you for your report. I think that Dan has fixed this as described in comment #1. I didn´t test with moving mails between mixedmaildir / maildir resources yet, so… Christoph, do you still see this with KDEPIM 4.14 or even 15.08 if you can get it already? Thanks and greetings from KDE Randa Meetings, Martin
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days, the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please set the bug status as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone!
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 30 days. The bug is now closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging Thank you for helping us make KDE software even better for everyone!