Bug 375149

Summary: Kmail deletes emails from folder after switching to folder
Product: [Applications] kmail2 Reporter: tprotopopescu
Component: message listAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED WAITINGFORINFO    
Severity: major CC: christian.saemann, lrajchel1981, robby.engelmann, roberto.castagnola, smorg, tickets, vylu, whatifgodwasoneofus
Priority: NOR    
Version: 5.4.1   
Target Milestone: ---   
Platform: openSUSE   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description tprotopopescu 2017-01-16 17:02:53 UTC
When switching to a folder from another kmail deletes, not moves to trash, emails as though I were holding down the delete key. 

These are all emails downloaded with POP. I do not know if IMAP folders would also be effected (I was not able to reproduce when switching to folders with messages from an IMAP account, but maybe I just got lucky.) 

Reproducible: sometimes (but it has happened already three times today, twice after an update). 


I have noticed is that it happens when I switch folders (including to the inbox). Sometimes it continues deleting if I switch to another folder, sometimes it stops, that is sometimes it does not continue deleting after switching. Quiting kmail seems to make it stop, at least emails that had not been deleted are still there upon restart. I have not kept kmail open long enough to see if the process will stop after some time.

This also seems to coincide with an uptick in the incidence of the akonadi hanging with  "Retrieving folder contents ...  please wait", necessitating an 'akonadictl restart' as mentioned here: https://bugs.kde.org/show_bug.cgi?id=297930 . For a while this latter problem had been better, and today it has become more frequent again.



For what it is worth: Here is terminal output for kmail, the collections named in the output were all effected:

Pass a valid window to KWallet::Wallet::openWallet().
sonnet.core: No language dictionaries for the language: "en"
sonnet.core: deleting 0x1dd56b0 for "en"
sonnet.core: No language dictionaries for the language: "en"
sonnet.core: No language dictionaries for the language: "en"
sonnet.core: No language dictionaries for the language: "en"
sonnet.core: No language dictionaries for the language: "en"                                                                         
sonnet.core: deleting 0x2240740 for "en"                                                                                             
sonnet.core: deleting 0x2063a28 for "en"                                                                                             
sonnet.core: Loading trigrams from "/usr/share/kf5/sonnet/trigrams.map"                                                              
sonnet.core: No scores for "Hi, "                                                                                              
sonnet.core: No language dictionaries for the language: "en"                                                                         
sonnet.core: Unable to identify string with dictionaries: "Hi, "                                                               
sonnet.core: No language dictionaries for the language: "en"                                                                         
sonnet.core: No language dictionaries for the language: "en"                                                                         
sonnet.core: No language dictionaries for the language: "en"                                                                         
sonnet.core: No language dictionaries for the language: "en"                                                                         
this does not work on a KActionCollection containing actions!                                                                        
org.kde.akonadi.ETM: GEN true false false                                                                                            
org.kde.akonadi.ETM: collection: QVector()                                                                                           
org.kde.akonadi.ETM:                                                                                                                 
org.kde.akonadi.ETM: Subtree:  4 QSet(64, 4, 65, 5, 67, 6, 70, 11, 72, 12, 13, 74, 14, 76, 78, 80, 20, 21, 82, 23, 24, 26, 28, 29, 90, 31, 33, 34, 36, 38, 40, 41, 43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63)                                                            
org.kde.akonadi.ETM: collection: "GP"                                                                                     
org.kde.akonadi.ETM: Subtree:  116 QSet(116, 117, 118, 119, 128, 120, 129, 121, 122, 123, 124, 125, 126, 127)                        
org.kde.akonadi.ETM: Fetch job took  506 msec                                                                                        
org.kde.akonadi.ETM: was collection fetch job: collections: 57                                                                       
org.kde.akonadi.ETM: first fetched collection: "Local Folders"                                                                       
org.kde.akonadi.ETM: collection: QVector()                                                                                           
org.kde.akonadi.ETM: Fetch job took  521 msec                                                                                        
org.kde.akonadi.ETM: was collection fetch job: collections: 17                                                                       
org.kde.akonadi.ETM: first fetched collection: "Search"                                                                              
org.kde.akonadi.ETM: Fetch job took  88 msec                                                                                         
org.kde.akonadi.ETM: was item fetch job: items: 62                                                                                   
org.kde.akonadi.ETM: Fetch job took  59 msec                                                                                         
org.kde.akonadi.ETM: was collection fetch job: collections: 0                                                                        
org.kde.akonadi.ETM: collection: "Foundations of Mathematics"
org.kde.akonadi.ETM: Fetch job took  125 msec
org.kde.akonadi.ETM: was item fetch job: items: 2277
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45189 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45188 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45188 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45117 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45117 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45117 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45117 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45046 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45046 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45046 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45046 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44995 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44995 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44995 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44995 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44945 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45220 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45220 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45218 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45218 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45191 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45191 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45190 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45190 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45190 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 45190 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44477 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44477 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44477 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44477 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44473 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44473 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44473 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44473 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44290 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44289 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44289 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44289 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44289 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44288 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44288 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44288 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44288 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44060 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 44060 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 43991 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 43991 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 43991 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 43991 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 43990 ""
org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 43990 ""
org.kde.akonadi.ETM: collection: "HSE"
org.kde.akonadi.ETM: Fetch job took  42 msec
org.kde.akonadi.ETM: was item fetch job: items: 653
org.kde.akonadi.ETM: collection: "Reviews"
org.kde.akonadi.ETM: Fetch job took  218 msec
org.kde.akonadi.ETM: was item fetch job: items: 4533
org.kde.akonadi.ETM: collection: "trash"
org.kde.akonadi.ETM: Fetch job took  50 msec
org.kde.akonadi.ETM: was item fetch job: items: 411
org.kde.akonadi.ETM: collection: "inbox"
org.kde.akonadi.ETM: Fetch job took  108 msec
org.kde.akonadi.ETM: was item fetch job: items: 2139

org.kde.pim.akonadicore: Got a stale notification for an item which was already removed. 46295 ""



WARNING: Phonon::createPath: Cannot connect  Phonon::MediaObject ( no objectName ) to  Phonon::AudioOutput ( no objectName ).
sonnet.core: deleting 0x21ee600 for "en"
sonnet.core: deleting 0x241e358 for "en"


sonnet.core: Removing loader:  Sonnet::Loader(0x7f2bf0cb0600)
Comment 1 Skander Morgenthaler 2017-01-16 21:27:30 UTC
Unfortunately I was just surprised by a mail loss as well :-(

Some observations from before I closed kmail and stopped akonadi:
- using pop3 to fetch mail (I am not using imap with kmail), local storage is maildir
- just upgraded to kde-apps 16.12.1 and frameworks 5-30 recently
- never happened with older kmail / akonadi versions, e.g. 16.12.0
- it also happened when clicking on (older) mails in a folder, not only when changing folders
- both folders where I could observe mails disapearing were inboxes (for different accounts) - but I did not try any other folders
- using a current gentoo install (~amd64)
Comment 2 tprotopopescu 2017-01-20 13:41:29 UTC
After some investigation it appears that the problem is not quite as bad as it first appears. It seems kmail is not deleting the email but moving them, in my case to a folder that holds email from a high traffic email list. It took a while to notice since I don't check that folder often, and since the emails moved had already been read the 'unread' count for the list folder did not change. (I did in fact lose some emails since that folder has an expiry date, and some of the messages moved were older).

Switching folders is not the trigger. Just switching emails in the same folder is enough for the move to start. Sometimes only about 10 emails are moved. But after switching from message to message, and having 10 or so messages moved, the process finally just moved every message in the folder.

Another thing I have noticed is that automatic filtering no longer works. Messages that should be moved to other folders whenever I check mail are not moved, but stay in the inbox. Applying filters manually moves messages.

I tried killing the `akonadi-mailfilter-agent' process while messages were being moved, and that did not stop the move.

Since data is not being deleting, but email do not stay where they should, it seems a severity of 'major' is now more appropriate.
Comment 3 Vytautas 2017-01-24 15:57:33 UTC
I can confirm, I have (ok, had, until moved my mail to thunderbird last week) the same problem. I have 4 mail accounts configured, one POP and three IMAP. Mail is getting moved from any of these accounts to single folder, in which I used filter messages from one of quite intensive forums. At first I was blaming filter, but now I'm not that sure. 

First time I saw this problem on about 20 Dec 2016 (I'm using KDE Neon), however, nobody else seemed to have exactly same problem, so, I blamed my config and didn't filled bug report (original post in https://forum.kde.org/viewtopic.php?f=215&t=137807)

Not sure about decreasing bug priority, as last time my messages where actually removed from IMAP server (OK, I could find these messages in one of my local folder, but it is not the same).
Comment 4 whatifgodwasoneofus 2017-01-24 21:37:54 UTC
I am also experiencing this since December. I am not able to identify the first kmail (or akonadi?) version with this issue.

I am using Manjaro Linux/KDE on amd64 (testing branch).
Comment 5 Robby Engelmann 2017-01-29 07:33:45 UTC
This morning I saw the very same issue using latest openSUSE Tumbleweed. The hint, that mails were moved rather than delete was quite helpful.
Here, it were also inbox folders from different accounts. That sounds to me, that Mail Filter Agent got crazy. I have one Filter set-up here, which moved mails of a mailing list to a subfolder. And I did find the missing mails within that mailing list folder.
Comment 6 whatifgodwasoneofus 2017-01-29 17:18:22 UTC
(In reply to Robby Engelmann from comment #5)

I have the same issue and I also think it's related to the mail filter agent, as ALL of my "disappearing" e-mails always go into at least one of the folders which are the target folders for filter rules.

I have noticed that, when this occurs, the following happens:
1. I (manually) move one big message (e.g., with attachments of 10 MB in size) from the source IMAP folder to a "local" folder;
2. KMail and/or Akonadi freeze, I can't say which one is hanging, but nothing moves on any more;
3. I restart Akonadi (with or without killing KMail, it makes no difference);
4. When I re-open KMail and I click on a folder, messages from that folder start to disappear (I can later find out where they go). I mean, I really see messages disappearing "live" from the list of messages;
5. I click on another folder, and now the messages from THAT folder (the one just clicked) start to disappear;
6. I shutdown KMail and Akonadi, restart Akonadi and KMail, and everything is "calm".

I can find out where the messages have gone because I keep a backup of my local mail folders (which are all "Maildir" folders) and I can DIFF against the backup copy, so I can see which messages disappeared from one directory and went into another, and can move them back to the respective ones.

I wonder what could happen if one of the mail filter rules is to DELETE messages instead of moving them into another folder...

Please help!
Comment 7 tprotopopescu 2017-02-03 08:10:01 UTC
I now see this behaviour in imap accounts also; in this case I cannot find the messages that were moved, so maybe deletion is still possible with this problem.

There is certainly something broken with the filters. Automatic filtering only works intermittently, messages that should be moved are not. On the other hand, sometimes messages that should not be moved from the inbox end up in other folders!

Combined with an increase in the severity of this problem https://bugs.kde.org/show_bug.cgi?id=297930 KDE pim applications are now unusable (can't access email or calendars and tasks reliably) and destroy data.

Akonadi used to work fine a few months ago, is it possible to roll back to a known good state while this mess is sorted out?
Comment 8 tprotopopescu 2017-02-09 08:28:45 UTC
For what it is worth. I turned off all filters a few days ago and have not yet had any messages moved or deleted.
Comment 9 tprotopopescu 2017-02-13 09:18:17 UTC
After selectively re-enabling some filters this problem has returned. Further, on another computer which accesses mail only through imap, and all filtering is done on the server, I have not seen this problem at all.

An obvious work around for this problem is to disable all filters, and occasionally run the filters manually. However, not even that works reliably since the mail filter agent does not always move emails even if only one filter is applied (through the filter configuration, 'run now' option)!
Comment 10 Christian Saemann 2017-02-16 21:38:11 UTC
I've had the same problem and it was definitely related to filtering. I had to disable the filter agent in akonadi console and delete/setup again all filter rules. During this process, I noticed that the target folders under "filter action" were sometimes not set correctly (i.e. the textbox was empty). If I clicked on the button to set the folder, however, the correct folder was selected.
Comment 11 tprotopopescu 2017-02-21 10:40:36 UTC
I tried removing the mail filter agent, through akonadi console, and starting again with filters. Unfortunately this did not work in two respects. 1) after defining a new filter and running it for the first time messages not supposed to be moved were moved. 2) subsequently no new messages which should be moved according to the filter rule are moved. Applying the filter manually works only sometimes!

So the mail filter agent is broken in two ways, it either does too much or too little.
Comment 12 Vytautas 2017-03-02 07:43:41 UTC
Interesting, that filter logs doesn't show no suspicious activity.
Comment 13 Łukasz 2017-03-25 10:37:09 UTC
This is related to a bug https://bugs.kde.org/show_bug.cgi?id=376202, most likely a duplicate.
Comment 14 Justin Zobel 2022-11-03 01:40:17 UTC
Thank you for reporting this issue in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version?

If you can reproduce the issue, please change the status to "REPORTED" when replying. Thank you!
Comment 15 Bug Janitor Service 2022-11-18 05:15:16 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 16 Łukasz 2022-11-23 06:56:32 UTC
(In reply to Justin Zobel from comment #14)
> Thank you for reporting this issue in KDE software. As it has been a while
> since this issue was reported, can we please ask you to see if you can
> reproduce the issue with a recent software version?
> 
> If you can reproduce the issue, please change the status to "REPORTED" when
> replying. Thank you!

The issue doesn't exist with the Kontact version I currenlty use, which is 5.7.3.