| Summary: | copy mail folder to other folder on other ressource results in empty mails | ||
|---|---|---|---|
| Product: | [Frameworks and Libraries] Akonadi | Reporter: | Christoph Thielecke <crissi99> |
| Component: | Maildir Resource | Assignee: | kdepim bugs <kdepim-bugs> |
| Status: | RESOLVED WORKSFORME | ||
| Severity: | major | CC: | auxsvr, bruno, Martin, montel |
| Priority: | NOR | Keywords: | triaged |
| Version: | 4.12 | ||
| Target Milestone: | --- | ||
| Platform: | Kubuntu | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | ||
| Sentry Crash Report: | |||
|
Description
Christoph Thielecke
2014-02-07 17:30:05 UTC
Git commit ff9edf8c0d96b179116558020d5a571a25142795 by Dan Vrátil. Committed on 05/04/2014 at 21:29. Pushed by dvratil into branch '1.12'. Fix potential severe data loss during copy and move operations Move and copy operations on larger sets of items can take some time, because we need to make sure that all items have full payload stored in Akonadi. This is necessary especially with local resources, like maildir, which have cache timeout set to 5 minutes (after that all their payload is removed from Akonadi and is fetched from HDD on demand, because it's cheap, fast and does not unnecessarily duplicate emails in maildir and in database). However fetching large amount of items via ItemRetriever takes a lot of time, sometimes it can take even more than 5 minutes. And in such case there is a very high chance, that the CacheCleaner will just remove the newly cached payloads from Akonadi again and so when ItemRetriever finishes, many items have empty payload in Akonadi again. ItemRetriever nor handlers are aware of this howerver, so they will just make copies or moves of empty items, causing data loss. This patch introduces CacheCleanerInhibitor, a class which when it is created will pause the timer in CacheCleaner and resume the timer again when it's destroyed (so usually when it goes out of scope). Also, this patch adds the inhibitor to all handlers that use ItemRetriever, so that the the situation described above does not happen. The current solution is not perfect because it pauses the entire CacheCleaner while I think it would be better to be able to temporarily 'blacklist' only specific collections or items. That would however require much more complex code and changes, which makes it unsuitable for 1.12 release. I tried moving 78 000 emails from one maildir to another and all emails were moved correctly. Move itself has many other problems (CPU, IO, memory, KMail responsivness, etc.) but that's out of scope of this fix. M +49 -0 server/src/cachecleaner.cpp M +30 -0 server/src/cachecleaner.h M +94 -1 server/src/collectionscheduler.cpp M +4 -1 server/src/collectionscheduler.h M +3 -0 server/src/handler/colcopy.cpp M +3 -0 server/src/handler/colmove.cpp M +3 -0 server/src/handler/copy.cpp M +3 -0 server/src/handler/fetch.cpp M +4 -0 server/src/handler/modify.cpp M +3 -0 server/src/handler/move.cpp http://commits.kde.org/akonadi/ff9edf8c0d96b179116558020d5a571a25142795 Does this produce 1-byte sized file in target folder? If so: Bug 319226 - produces 1-byte-sized on processing crm114 spam filter rules might (!) be related as well as Bug 331486 - creation of 1000s of 1byte files in inbox/new/ for "ghost" mails Bug 290422 - lost many email while moving them from a local folder to a local folder Also bug 279260 could be related to this. *** Bug 290422 has been marked as a duplicate of this bug. *** Christoph, thank you for your report. I think that Dan has fixed this as described in comment #1. I didn´t test with moving mails between mixedmaildir / maildir resources yet, so… Christoph, do you still see this with KDEPIM 4.14 or even 15.08 if you can get it already? Thanks and greetings from KDE Randa Meetings, Martin Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days, the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please set the bug status as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone! Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 30 days. The bug is now closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging Thank you for helping us make KDE software even better for everyone! |