| Summary: | Deleted emails stay, but greyed out | ||
|---|---|---|---|
| Product: | [Frameworks and Libraries] Akonadi | Reporter: | Alex Merry <alex.merry> |
| Component: | IMAP resource | Assignee: | Christian Mollekopf <chrigi_1> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | alberto, ami.nuscule, anmeldungen, bugs, christoph.ruessler, cjlapao, code, dmitry, dvratil, e-anima, eduard, henriet.quentin, huleo, ilovekde, info, jranke, jsardid, justin, kdepim-bugs, kill, luigi.toscano, maarten.bezemer, Martin, massimiliano.torromeo, naworski98, paolo.pedroni, piral, rafaelalcantaraperez, rdieter, rf-kde, saleck, sgruener, stephan, steve, stuart, tim.eberhardt, vkrause, yanp.bugz |
| Priority: | HI | Keywords: | regression |
| Version: | GIT (master) | ||
| Target Milestone: | --- | ||
| Platform: | Compiled Sources | ||
| OS: | Linux | ||
| Latest Commit: | http://commits.kde.org/kdepim-runtime/6f2026f657ba7b68b4022ca969b3b6ff052609d1 | Version Fixed In: | 16.04.0 |
| Sentry Crash Report: | |||
| Attachments: |
Log of imap delete on Courier and Dovecot
akonadiconsole debug output while deleting akonadconsole debug output while reentering directory after "deletion" |
||
|
Description
Alex Merry
2015-10-18 18:46:43 UTC
Possibly related: Bug 299422 (although this is about the SEEN flag, rather than deleting) Bug 321357 (although my issue doesn't resolve itself by using roundcube to view my email, or by restarting anything) Bug 324135 (likewise). The server greeting is * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. I can confirm this bug using KMail 5.0.2 from Kubuntu Wily. Deleting doesn't really work (as described). And it seems all emails are fetched every time on sync. I can confirm this bug. It happens since I upgraded to the new KF5 based pim apps (Gentoo and openSUSE Tumbleweed). Mail is only marked as deleted but not expunged. This happens to my second Account only. Same thing here in Kubuntu Wily I have the same problem with a fresh install of Kubuntu Wily. I am using filters to move my e-mails in my local folders, and as the mails are not really deleted from the IMAP server, I have a new copy of the mail in my local folder every 5 minutes (every sync) but these messages are greyed, marked unread, and unselectable. Same thing here in Kubuntu Wily, kmail 15.08.2 I can confirm the same on kubuntu Willy (15.10) kmai 5.0.2, Moving or deleting a mail in a IMAP folder marks the original as deleted and creates a copy in the destination folder, but the source folder does not seem to be "expunged". this a real problem, it is no longer possible to run any filters, since moved mails are no longer beeing purged from original folder, resulting in the orignal folder growing, and the destination folder getting dublicates. Specially bad on accounts that get a large amount of spam. I confirm both problems. My Courier IMAP server do not recognize/get the Compact/purge/Expunge message that Akonadi/kmail is supposed to send. It used to work so I'd say the problem is within kmail/Akonadi. I'd like to stress the second problem said here: Reading incoming messages is far too slow and I only would understand it if kmail/akonadi/baloo is downloading again and again all the messages from inbox. Now the third problem is: there is not option I know to filter out deleted messages from kmail lists. If kmail were able not to show greyed out deleted messages, the problem would remain but we would be able to continue using kmail. Because so many greyed out deleted messages make the software unusable. One last point: I changed my server to a new one: Ubuntu 14.04 with plesk and CourierIMAP. The problem remains. The messages are not purged by kmail/akonadi. Thunderbird 38 has no problem purging. Same thing with Kubuntu 15.10 and KMail 5.0.2 I'm afraid i must switch to another mail client, as KMail is unuseable as is... What a pity ! I really like (and i'm used to) KMail Please feel free to ask for additional information, as i'm really looking forward to seeing Kmail as sexy as before :) Same to me, sadly, ever since I moved to KF5 based akonadi and kmail2 Created attachment 95789 [details] Log of imap delete on Courier and Dovecot The tar file contains separate sets of log files from two different delete cases, one for a courier server , and the other for a dovecot server. the settup was from a freshly added user, with a virgin Kmail config. System: Kubuntu 15.10 (Wily) and Kmail 5.0.2 , Kde Frameworks 5.10.0, QT 5.4.2. New email accounts on both servers, one mail in inbox, nothing else. Imap accounts identically configured, ie, only change was on "General" tab for server and login and on "Advanced" tab to enable tls ("Auto test") Observe that the capability report from each are different: Courier: OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information Dovecot: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED] Cyclops ready And that the way the akonadi imap client goes about deleting mail is also quite different. but the greatest difference are the lack of EXPUNGE commands in the Courier case: Courier delete: C: A000016 UID FETCH 1 (BODY.PEEK[] UID) S: * 1 FETCH ( BODY[] Return-Path: <kill@itr.no> ... Mail content deleted for brevity UID 1 ) S: A000016 OK FETCH completed. C: A000017 UID STORE 1 +FLAGS (\Seen) S: * 1 FETCH ( UID 1 FLAGS (\Seen) ) S: A000017 OK STORE completed. C: A000018 UID STORE 1 +FLAGS (\Deleted) S: * 1 FETCH ( UID 1 FLAGS (\Seen \Deleted) ) S: A000018 OK STORE completed. C: A000019 SELECT "INBOX.Trash" S: * FLAGS ( \Draft \Answered \Flagged \Deleted \Seen \Recent ) -------------------------------------------------------------------------------------------------------- Dovecot C: A000005 UID FETCH 1 (BODY.PEEK[] UID) S: * 1 FETCH ( UID 1 BODY[] ... mail content deleted for brevity ) S: A000005 OK Fetch completed. C: A000006 UID STORE 1 +FLAGS (\Seen) S: * 1 FETCH ( UID 1 FLAGS (\Seen) ) S: A000006 OK Store completed. C: A000007 EXPUNGE S: A000007 OK Expunge completed. C: A000008 SELECT "INBOX" (CONDSTORE) S: * OK Previous mailbox closed. [ CLOSED ] S: * FLAGS ( \Answered \Flagged \Deleted \Seen \Draft ) S: * OK Flags permitted. [ PERMANENTFLAGS ( \Answered \Flagged \Deleted \Seen \Draft \* ) ] S: * 1 EXISTS S: * 0 RECENT S: * OK UIDs valid [ UIDVALIDITY 1448727510 ] S: * OK Predicted next UID [ UIDNEXT 2 ] S: * OK Highest [ HIGHESTMODSEQ 3 ] S: A000008 OK Select completed ( 0.000 secs ) . [ READ-WRITE ] C: A000009 UID FETCH 1:2 (FLAGS UID) (CHANGEDSINCE 2) S: * 1 FETCH ( UID 1 FLAGS (\Seen) MODSEQ (3) ) S: A000009 OK Fetch completed. C: A000010 UID STORE 1 +FLAGS (\Deleted) S: * 1 FETCH ( UID 1 MODSEQ (4) FLAGS (\Deleted \Seen) ) S: A000010 OK Store completed. C: A000011 EXPUNGE S: * 1 EXPUNGE S: A000011 OK Expunge completed. C: A000012 SELECT "INBOX" (CONDSTORE) *** Bug 316153 has been marked as a duplicate of this bug. *** Hello !! How can such a critical bug be remaining after 2 months ? This bug is REALLY making impossible using KMail2. I may not be the only one to have switched to another mail client ? I know you guys may have big loads of work and that seasonal issues may make it difficult to schedule. But please keep in mind that KMail2 also makes it difficult these days too... However, keep on the good work, and please make us feel like there is really someone concerned about this critical issue :) Thx for all, we're (really) looking forward to seing this bug erradicated. This also happens when connecting do an MDaemon IMAP server [1] (not my choice of IMAP server but it is what is used at work) [1] http://www.altn.com/Products/MDaemon-Email-Server-Windows/ This is still present: Akonadi 5.0.51 kmail2 5.0.3 Hi ! Is KMail dead ?? If so, please tell us, so that we can switch (and cry) to another email client ? What a pity, kmail was really the best choice for mail client !! But unfortunately it is unusable (as it is right now)... Can you guys try and give us a timeline approximation ?? (yes, i'm really fed up with sylpheed..) Thanks for the goooooooooooooood work anyway !! I can confirm this behaviour as well on Kubuntu Wily (KMail 5.0.2, Aconadi 5.0.51) when communicating with Courier IMAP server (both non deleting messages and excessive amount of transmitted data). Other accounts connecting to Dovecot are not affected. I can confirm this on Kmail 5.1.2, Akonadi 5.1.51, KDE Framework 5.19, Qt 5.5.1 (Packages from Opensuse Tumbleweed) I encounter this bug with 2 of my 3 imap-accounts, unfortunately I don't know which software they are using. This bug is reproducible even after deleting akonadi-databases[1] If I add the accounts again, Akonadi even builds it's indexes with greyed-out e-mails. They seem to be deleted in my webmail-interface. These emails can't be accessed through kmail (they're just showing up). I can't access them via browser-tab in akonadiconsole either (because of this bug: https://bugs.kde.org/show_bug.cgi?id=355229) BUT: In akonadiconsole DB-Browser-Tab "pimitemtable" there are entries with remoteID-values that are not shown/indexed by Thunderbird [2] . The big question now is, why is Akonadi indexing mails, that Thunderbird/$Webmail isn't? [1] https://docs.kde.org/trunk5/en/kdepim/kmail/clean-start-after-a-failed-migration.html [2] In Thunderbird you can sort mails and show "Order Recieved", which I think is the same number as is the remoteID in akonadi. Created attachment 97371 [details]
akonadiconsole debug output while deleting
Created attachment 97372 [details]
akonadconsole debug output while reentering directory after "deletion"
Bug 351571(In reply to Sebastian Gruener from comment #20) > I encounter this bug with 2 of my 3 imap-accounts, unfortunately I don't > know which software they are using. That's interesting. Could you do me a favour and check if you're also affected by Bug 351571 with the 2 buggy accounts? In my case there is some kind of correlation with this Bug here with 1 of my 4 imap-accounts. I can confirm this bug since my distribution upgrade. The to me latest known working version is kmail 4.14.7 with akonadi 1.13.0.
Like with Thunderbird, syncing the affected imap account parallel with the old/working kmail/akonadi version makes the ghost/shadowed/grayed ("undead") mails disappear forever.
Maybe its trivial to say, that the undead mails appear also if moving them somewhere, not only by deleting them. This is really annoying if you use Filters to move incoming mails around, because these undead mails are returning tagged as unread.
Speaking about tags: When the bug started to affect me, I was able to make the undead mails appear as "normal" in by changing the status from unread to read. So I could delete them (again), but after refreshing it just reappeared with an additional undead mail. Changing the Status is now (kmail 5.0.3, akonadi 5.0.51) not possible anymore.
(In reply to naworski98 from comment #24) > Speaking about tags: When the bug started to affect me, I was able to make > the undead mails appear as "normal" in by changing the status from unread to > read. So I could delete them (again), but after refreshing it just > reappeared with an additional undead mail. Changing the Status is now (kmail > 5.0.3, akonadi 5.0.51) not possible anymore. O.k. an update for this behaviour: "reanimating" the "undead" mails by changing the read/unread status works as long there is at least one normal ("living") mail present in the directory. However, the "reanimated" mails still just generates additional "undead" copys of itself by moving/deleting them. i have this problem too. i have the felling that importing old mail killed something. even recreating accounts doesnt solve it. in my case imap is somehow completely broken now because of these greyed out ghost mails. kmail 5.1.2 Same Problem in kmail5-15.12.2-39.10.x86_64 You can select "Automatically compact folders" in the Advanced Tab of the IMAP Ressource. This will instruct KMail to send the EXPUNGE command and you will get rid of the greyed out messages. Why you cannot tell kmail to not display messages marked as deleted is beyond my understanding though. That should be standard. Also, boo to the kmail developers, who failed to respond properly to this bug report in weeks. And it is not the only report I noticed this behaviour. Is there any active development in kmail? (In reply to Ben from comment #29) > You can select "Automatically compact folders" in the Advanced Tab of the > IMAP Ressource. > > This will instruct KMail to send the EXPUNGE command and you will get rid of > the greyed out messages. > > Why you cannot tell kmail to not display messages marked as deleted is > beyond my understanding though. That should be standard. > > Also, boo to the kmail developers, who failed to respond properly to this > bug report in weeks. And it is not the only report I noticed this behaviour. > Is there any active development in kmail? Ben, that doesn't work, that is the problem. When KMail connects to Courier IMAP server it simply doesn't expunge deleted messages no matter how you set KMail up. Older versions had been ok. (In reply to Ben from comment #29) > You can select "Automatically compact folders" in the Advanced Tab of the > IMAP Ressource. > > This will instruct KMail to send the EXPUNGE command and you will get rid of > the greyed out messages. Sorry, this checkbox appears to have no function. The greyed out messages are still there. Same problem here with two different systems: Akonadi 5.1.51 KMail2 5.1.3 QT 5.5.1 und 5.6 Imap Server: Kolab with Cyrus Imap backend No problems when using Thunderbird or Roundcube - the emails don't show up in these. Git commit 6f2026f657ba7b68b4022ca969b3b6ff052609d1 by Daniel Vrátil. Committed on 21/03/2016 at 22:27. Pushed by dvratil into branch 'Applications/16.04'. IMAP: handle deprecated Delete right when checking ACL before expunge Due to ambiguity in RFC 2086 the new RFC 4314 introduced two virtual rights, one of them being "d" (delete). When "d" right is present, presence of "e", "t" and "x" rights must be assumed by the clients. Courier IMAP server uses this virtual rights but does not explicitly list the "e" (expunge) right like most other servers, which caused RetrieveItemsTask to not trigger EXPUNGE before syncing the mailbox. FIXED-IN: 16.04.0 M +3 -2 resources/imap/retrieveitemstask.cpp http://commits.kde.org/kdepim-runtime/6f2026f657ba7b68b4022ca969b3b6ff052609d1 I confirm, it works now. Thanks a lot! :-) Still present in kubuntu 16.04 with kmail 5.1.3. Deleted mails first disapear but reapears again after the next update of the mailbox. suse-leap 42.1 kmail 4.14.10 google-imap confirm greyedout mails after delete or move well IMAP via GMAIL ist kindo special .. however : for me i was able to get rid of the greyout-emails by checking the option in Accounts > $myGMAILaccount > advanced > automatically compress folders it took ages for akonadi to cleanup but it worked out well FYI This has been resolved in the latest kmail version (5.1.3) and kde framework (5.18.0) (In reply to Kim Lilliestierna from comment #38) > This has been resolved in the latest kmail version (5.1.3) and kde > framework (5.18.0) For me the problem persists in Kubuntu 16.04. Versions: kmail2/kontact 5.13 Akonadi 5.1.51 KDE Frameworks: 5.18.0 (In reply to Kim Lilliestierna from comment #38) > This has been resolved in the latest kmail version (5.1.3) and kde > framework (5.18.0) The problem persists for me too. Kubuntu 16.04 kmail2/kontact 5.13 Akonadi 5.1.51 KDE Frameworks: 5.18.0 Server: Ubtuntu 12.04 / Plesk / courier-imap This bug persists for me in kmail 5.1.3 and frameworks 5.22.0 (kubuntu 16.04 + backports ppa), should not be marked as fixed IMHO Daniel, I reopen this. Please you advice whether users shall open a new bug report about the remaining deleted mails stay but greyed out issues. If you think it is better to open a new bug report, feel free to close this one again. Also to all the users of KDEPIM + Akonadi 16.04 who it still doesn´t work, did you try: "Accounts > Your account > advanced > automatically compress folders". Same versions and symptoms at those above. Have automatic compact on but it does not work. Expiring manually through Thunderbird works. (In reply to Martin Steigerwald from comment #42) > Also to all the users of KDEPIM + Akonadi 16.04 who it still doesn´t work, > did you try: "Accounts > Your account > advanced > automatically compress > folders". Is checked for every account. Cached imap is off (hopefully it's called that way in english). I do not know if this is related though. For everyone still reporting the issue: please make sure that you have PIM components from KDE Applications 16.04 (which has nothing to do with *Ubuntu 16.04), which means pim (kontact and akonadi) version 5.2, not 5.1. Closing again as those "problems persist" reports are against a version that does not contain the bugfix. Having the same issue with kmail 5.7.3 , kubuntu 18.04 Is there a way to clear this akonandi cache? I've found what "fixes" my inbox folder - starting and exiting from thunderbird. When thunderbird exits all greyed mails disappear. Why did you close? I just spent several hours hunting this down. It turned out I just had to enable "compress folders" in the advanced view of my account configuration as mentioned earlier in this thread. Can't believe it was so easy... |