Bug 340931 - Read status of emails not updated when changed outside KMail, but only for GMail accounts via IMAP
Summary: Read status of emails not updated when changed outside KMail, but only for GM...
Status: CONFIRMED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: IMAP resource (show other bugs)
Version: 1.13.0
Platform: Debian testing Linux
: NOR normal with 160 votes (vote)
Target Milestone: ---
Assignee: Christian Mollekopf
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-11-13 16:57 UTC by Brendon Higgins
Modified: 2022-01-21 19:49 UTC (History)
16 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brendon Higgins 2014-11-13 16:57:50 UTC
I have several email accounts set up in Akonadi (which I access with KMail 2). One for work that uses some kind of Exchange IMAP bridge (I don't know the details), and a couple of GMail accounts set to allow IMAP. I use several computers all accessing these accounts in the same way.

When I read an unread email in my work account on one computer, then re-sync on another computer, the email status changes to "read" on the second computer, just like you would expect. Conversely, if I then mark a read email as unread, it's status is correctly updated on the other computer.

But if I do the same for an email in either of my GMail accounts, nothing changes on the other computer. I can confirm using GMail's web interface that the change has propagated to the server. I can also do testing via that interface, and still no changes are making it back into KMail. So it seems to be the case that Akonaid is failing to notice this status has changed - somehow the IMAP client is not being told about the change, or doesn't ask the right question.

The problem appears on both of my GMail accounts, and creating a new IMAP resource does not make the problem go away (the initial sync is correct, but subsequent syncs still don't update statuses correctly).

Unfortunately I can't comment specifically on when this bug arose because it's been happening probably for a few months.

Reproducible: Always

Steps to Reproduce:
1. Change the read status of a message in a GMail account (using a different computer or the web interface).
2. Sync the account.

Actual Results:  
The status is not updated.

Expected Results:  
The status is updated.
Comment 1 John Baldwin 2014-11-19 18:24:05 UTC
I am seeing a similar problem against an IMAP server running cyrus.  I only started seeing this recently when I upgraded to KDE 4.14.2 from 4.12.5.  When mails are marked as read on any other device (another computer running kmail, my phone or tablet, etc.) while my desktop running KDE 4.14.2 is up and running (so akonadi still running), kmail doesn't notice that the mails are marked read.  I've tried restarting akonadi to try to force it to re-sync its state with the server but that doesn't fix it.
Comment 2 Adriano Vilela 2014-12-08 23:29:29 UTC
I'm seeing the exact same problem here (accessing a Gmail account through IMAP). I'm using Kmail 4.14.2 on Debian Testing.
Comment 3 Erik Quaeghebeur 2014-12-13 19:10:08 UTC
This just now started happening to me for a Cyrus IMAP server after updating to KDE SC 4.14.3 from 4.12.5. I assume there must be some connection to KMail, because akonadi-server had been updated already to 1.13.0 some weeks before.

My investigation turned up the following:

* When opening the mailbox with kmail, they appeared as unread.

* When opening with akonadi-console, the messages in the mailbox had no flags (or keywords) set at all.

* When opening the mailbox using Thunderbird, everything worked fine.

* When opening with Trojita, its IMAP implementation complained that there were non-parseable flags (or did it say keywords? that would be more correct), namely, "[EXPUNGED]", and "[ARCHIVED]". It is not clear to me whether the '[' and ']' characters are disallowed by the IMAP RFCs (they seem valid in modified UTF-7, I think.)

I therefore did the following: I used Firefox to remove those keywords, after which all the other clients behaved as they should: picking up all flags and keywords. (KMail does not show the keywords, of course, see Bug 283020.)

My conclusion is therefore that the IMAP resource chokes on such keywords. If these are not invalid according to the RFC, the resource's flag/keyword parser should be modified to be able to deal with them. If these are invalid, upstream bugs should be filed to the generators of those keywords and perhaps workaround code for know invalid keywords could be included.

To verify my conclusion, another affected user should install Trojita and sees what it lists. (They can of course also use akonadi-console or Thunderbird for further tests.)

If verified somebody with sufficient IMAP RFC erudition should shine a light on whether such keywords are valid or not. Then a course of action can be taken. (Modifying the parser should not be that big of a modification, no?)
Comment 4 John Baldwin 2015-01-05 15:41:03 UTC
RFC 3501 contains a BNF at the end that specifies this for a flag name:

flag-keyword    = atom

atom            = 1*ATOM-CHAR

ATOM-CHAR       = <any CHAR except atom-specials>

atom-specials   = "(" / ")" / "{" / SP / CTL / list-wildcards /
                  quoted-specials / resp-specials

list-wildcards  = "%" / "*"

quoted-specials = DQUOTE / "\"

resp-specials   = "]"

So it does seem that at least ']' is invalid though '[' may be valid.

However, I wonder if those brackets are just in the dialog your test program showed you and not in the actual flags on your server?  I did a manual IMAP connection to my Cyrus IMAP server and examined one of the e-mails in question.  I don't see any flags with brackets in their name:

4 select "INBOX.njvine"
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $Forwarded KMAILFORWARDED KMAILTODO KMAILWATCHED KMAILIGNORED $TODO $WATCHED $IGNORED Junk NonJunk $REPLIED)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $Forwarded KMAILFORWARDED KMAILTODO KMAILWATCHED KMAILIGNORED $TODO $WATCHED $IGNORED Junk NonJunk $REPLIED \*)]  
* 763 EXISTS
* 0 RECENT
...
4 OK [READ-WRITE] Completed
5 fetch 763 all
* 763 FETCH (FLAGS (\Seen) INTERNALDATE "23-Dec-2014 16:04:26 -0500" RFC822.SIZE 29456 ENVELOPE ( <redacted> ))
5 OK Completed (0.000 sec)

As you can see, the only flag on this message is \SEEN, and yet kmail refuses to consider it read.  In akonadiconsole I can confirm that I do not see \SEEN as a flag for this message even though the server reports \SEEN as set above.  Interestingly, the modification time of the message in akonadiconsole is right now (Jan 5, 15:32:48 UTC) though it reports a revision of 1 (and the message itself was marked read via a different mail client (probably Mail on iOS) about a week ago).

With the volume of e-mail I get each day on various FreeBSD lists, this bug has made kmail useless for me since things are only counted as read reliably in kmail on my desktop at home if I only ever read e-mail on my desktop at home.  Entirely defeats the point of using an IMAP server for mail.
Comment 5 Erik Quaeghebeur 2015-01-06 10:18:48 UTC
(In reply to John Baldwin from comment #4)
>
> However, I wonder if those brackets are just in the dialog your test program
> showed you and not in the actual flags on your server?  I did a manual IMAP
> connection to my Cyrus IMAP server and examined one of the e-mails in
> question.  I don't see any flags with brackets in their name: [...]

No, the brackets were indeed there: KMail and Trojita choked on them, but Thunderbird accepted them. I removed them using Thunderbird (and not Firefox, of course, as I stated in my earlier report) and then everything went fine.

Trojita's <http://trojita.flaska.net/> IMAP implementation is completely different from KMail's (Akonadi's) and quite good. I can advise it as a good program to debug any IMAP related problem. I believe you when you say that you do not have brackets in the keywords of the problematic mails; it would be useful for solving this bug in finding out what is causing the trouble for you. Having a look at the mailbox concerned with Trojita may help in finding out.
Comment 6 Hans de Raad 2015-09-07 13:46:37 UTC
*** This bug has been confirmed by popular vote. ***
Comment 7 Jesús 2016-05-31 22:02:06 UTC
I have the same bug on Manjaro with kmail 5.2.1
Comment 8 John Baldwin 2018-06-14 23:01:50 UTC
I am still seeing this after upgrading to KMail 5.8.1 (18.04.1) against a Cyrus IMAP server.
Comment 9 Martin Tlustos 2018-09-03 06:13:10 UTC
Confirmed on kmail 5.9.0 on KDE Neon User Edition
Comment 10 Doug 2021-08-17 03:44:25 UTC
Just want to confirm that this is still an issue for me using the latest Kontact/Akonadi 21.08.0 on Manjaro Testing.
Comment 11 Meggs 2022-01-21 19:49:36 UTC
I can confirm this bug with reference on gmail (IMAP) on kubuntu 20.04, KDE-Version 5.18
Please fix this silly bug.

Another bug is similar.  if I set in kmail  a new message in the "inbox" to read, the 
email in folder "all mail" shows the email as unread.  

Excursion:
Messages in gmail folders (for instance inbox) are virtualed (labeled) they are only stored in the folder "all mails". 
If i receive an email it appears in the folders "inbox" and "all mails".