Bug 338186 - Since updating to the latest Kubuntu packages Kmail is not picking up imap mail.
Summary: Since updating to the latest Kubuntu packages Kmail is not picking up imap mail.
Status: RESOLVED FIXED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: IMAP resource (show other bugs)
Version: unspecified
Platform: Ubuntu Linux
: VHI grave
Target Milestone: ---
Assignee: Christian Mollekopf
URL:
Keywords:
: 338250 338557 338567 338572 338733 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-08-11 08:25 UTC by Peter Nunn
Modified: 2016-07-15 22:37 UTC (History)
47 users (show)

See Also:
Latest Commit:
Version Fixed In: 16.04


Attachments
logs (14.19 KB, text/plain)
2014-08-25 07:18 UTC, Olivier Goffart
Details
debug logs (3.85 KB, text/plain)
2014-08-25 15:01 UTC, Marcel Naziri
Details
IMAP log comparison Dovecot and Courier (3.17 KB, text/plain)
2014-08-26 15:26 UTC, Marcel Naziri
Details
Courier IMAP session log (2.45 KB, text/plain)
2014-08-26 20:54 UTC, Marcel Naziri
Details
let ImapInterval handle negative value (1.17 KB, patch)
2014-08-26 22:11 UTC, Marcel Naziri
Details
patch for fallback with invalid uidnext (4.61 KB, patch)
2014-08-27 10:46 UTC, Christian Mollekopf
Details
version 2 of patch for fallback with invalid uidnext (8.59 KB, patch)
2014-08-29 08:07 UTC, Christian Mollekopf
Details
version 3 of patch for fallback with invalid uidnext (8.81 KB, patch)
2014-09-09 19:14 UTC, Christian Mollekopf
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Nunn 2014-08-11 08:25:31 UTC
Since updating to the latest kontact packages this morning, kmail is not picking up any email during a sync. Originially I thought it was due to it not picking up anything since the update, but removal of the account and re-adding it shows that it is infact not picking up any mail.

I've tried tls and ssl (on two different ports) with no success. No errors, it shows that it is connecting OK but the scan never moves off 0%. Other mail clients are having no issues connecting to the server (courier imap if that matters).

Akonadi console shows the sync working, but it is admittedly slower than normal it seems.

The kontact version is 4.14rc.
Comment 1 pipapo 2014-08-12 12:24:36 UTC
I confirm this bug.

It's happening with an imap-account on imap.arcor.de.
This behaviour also occurres when setting up the mail-account with a new user on a new and clean desktop.
Thunderbird and web interface are working fine. Also mail-clients on other devices and platforms.
It's possible to send mails via the smpt server via the same provider. The sended mails will be sync with kmail and appears in kmail and also on the sever.
It seems that the sync works only in one direction.

Dist: Ubuntu 14.04.1 LTS
Kmail version: 4.13.97

The only suspicous messages are found in the job tracker window of the akonadi console.

----------
d851a0	14:09:20.752	00:00:00.000	00:00:00.089	Akonadi::ItemSync	Failed: Abbruch durch Benutzer	Collection 1034 (foo)
		d639d0	14:09:20.839	00:00:00.001	00:00:00.002	Akonadi::TransactionSequence	Ended	
			akonadi_imap_resource_0						
			c848e0	14:09:21.522	00:00:00.001	00:00:00.001	Akonadi::ItemFetchJob	Ended	All items from collection 1029
			d810c0	14:09:22.020	00:00:00.000	00:00:00.001	Akonadi::TransactionCommitJob	Ended
------------

With guidance I can provide further information. The imap-bugtracking article on kde thechbase is too advanced. I'm failing on the first sentence. This bug has essential influence on my daily working process. Using Thunderbird is current my workaround.
Comment 2 avlas 2014-08-12 17:16:27 UTC
Same thing here, I also confirm it
Comment 3 Bernd Weigelt 2014-08-13 15:22:51 UTC
Same problem with opensuse's new factory distri, which uses KDE 4.13.90.
roll back to to openSUSE 13.1 with KDE 4.13.3 solve the problem for me.

IMAP is used with my own local server.
Comment 4 Thomas Gideon 2014-08-14 15:45:20 UTC
*** Bug 338250 has been marked as a duplicate of this bug. ***
Comment 5 Thomas Gideon 2014-08-14 15:47:34 UTC
I can also confirm. I will add that I have another IMAP resource configured that does *not* show this behavior. The resource that is failing in this way is also connected to Courier as Peter Nunn noted in the original description. The resource that *is* working connects to a Dovecot server. I can attach more info if needed and helpful.
Comment 6 Bernd Weigelt 2014-08-14 15:56:45 UTC
BTW: I'm using Courier on my server, see comment #3
Comment 7 Christoph Feck 2014-08-15 10:41:14 UTC
Is someone able to test if the regression is caused by the Akonadi update? In other words, keep newest Akonadi, but revert kontact/kmail2 to 4.13.3 version?
Comment 8 Thomas Gideon 2014-08-20 18:18:18 UTC
I have tried downgrading Kontact with no success, using the "apt-get install kontact=<version>" method, so cannot test the regression case suggested.

Also, I just received the 4.14 update and the account that was working, against a server running Dovecot, now falls in the same way. So as of 4.14, it falls for servers running both Courier and Dovecot.
Comment 9 Christoph Feck 2014-08-20 18:45:43 UTC
Adding more people who might know where to look for regressions. It's still unclear if kmail2 or akonadi regressed.
Comment 10 Robert G. Siebeck 2014-08-22 11:50:41 UTC
Same problem here since upgrading to Kmail 4.14.0 and Akonadi 1.13.0. I'm using Courier IMAP.
Comment 11 Thomas Gideon 2014-08-22 14:38:47 UTC
An update on the fresh problems since receiving the 4.14 ugrade. Kontact/Akonadi went from not picking up email on my Dovecot supported account to Akonadi crashing and refusing to start. I backed up my .local/akonadi directorty and ran "akonadictl start". That rebuilt my database from scratch, effectively fixing the start up problems. I can now at least get my email from my account on a Dovecot server, again. However, now I am having a problem with my calendars. Only one of my three calendars from the same Davical server are being picked up. I looked in the object browser in akonadiconsole and indeed it only seems to have objects for the entries in that one calendar. I tried re-fetching from the calendar properties. I even deleted the CalDAV entries in the calendar properties and re-added. Those calendars continue to work fine in an un-upgraded instance of Kontact/Akonadi and on my Android phone plus the entry count in the server's admin interface says those calendars are not empty so this seems to be a similar issue to the original email behavior but now with a different protocol, caldav. Maybe that helps narrow where the regression occurred?
Comment 12 Bernd Weigelt 2014-08-23 08:33:35 UTC
I've backuped ~/.local/share/akonadi and restart akonadi, now my folders in kmail are completly empty  and i get a lot of messages like this:

akonadi_imap_resource_0(2192) BatchFetcher::onUidSearchDone: Search job failed:  "Suche fehlgeschlagen, die Serverantwort lautet: A000656 NO Error in IMAP command received by server. " 
akonadi_imap_resource_0(2192) RetrieveItemsTask::onRetrievalDone: "" 

At the moment kmail is unusable for me :(
Comment 13 Bernd Weigelt 2014-08-23 08:38:36 UTC
This is the output on the console, if i use my old akonadi database:

QDBusObjectPath Akonadi::Server::NotificationManager::subscribeV2(const QString&, bool) Akonadi::Server::NotificationManager(0x1b11a00) "kontact_2268_nbmI9b" true 
QDBusObjectPath Akonadi::Server::NotificationManager::subscribeV3(const QString&, bool, bool) Akonadi::Server::NotificationManager(0x1b11a00) "kontact_2268_nbmI9b" true false 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onFinalSelectDone: Detected inconsistency in local cache, we're missing some messages. Server:  143  Local:  124 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onFinalSelectDone: Refetching complete mailbox. 
akonadi_imap_resource_0(2667) BatchFetcher::onUidSearchDone: Search job failed:  "Suche fehlgeschlagen, die Serverantwort lautet: A000021 NO Error in IMAP command received by server. " 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onRetrievalDone: "" 
posting retrieval request for item 6975  there are  1  queues and  0  items in mine 
request for item 6975 still pending - waiting 
processing retrieval request for item 6975  parts: ("ENVELOPE")  of resource: "akonadi_imap_resource_0" 
continuing 
request for item 6975 "27372" failed: "Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>" 
ItemRetrieverException :  Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>
akonadi_imap_resource_0(2667) RetrieveItemsTask::onFinalSelectDone: Detected inconsistency in local cache, we're missing some messages. Server:  143  Local:  124 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onFinalSelectDone: Refetching complete mailbox. 
akonadi_imap_resource_0(2667) BatchFetcher::onUidSearchDone: Search job failed:  "Suche fehlgeschlagen, die Serverantwort lautet: A000028 NO Error in IMAP command received by server. " 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onRetrievalDone: "" 
posting retrieval request for item 7051  there are  1  queues and  0  items in mine 
request for item 7051 still pending - waiting 
processing retrieval request for item 7051  parts: ("RFC822")  of resource: "akonadi_imap_resource_0" 
continuing 
request for item 7051 "27404" failed: "Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>" 
ItemRetrieverException :  Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>
posting retrieval request for item 7051  there are  1  queues and  0  items in mine 
request for item 7051 still pending - waiting 
processing retrieval request for item 7051  parts: ("RFC822")  of resource: "akonadi_imap_resource_0" 
continuing 
request for item 7051 "27404" failed: "Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>" 
ItemRetrieverException :  Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>
akonadi_imap_resource_0(2667) RetrieveItemsTask::onFinalSelectDone: Detected inconsistency in local cache, we're missing some messages. Server:  11  Local:  9 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onFinalSelectDone: Refetching complete mailbox. 
akonadi_imap_resource_0(2667) BatchFetcher::onUidSearchDone: Search job failed:  "Suche fehlgeschlagen, die Serverantwort lautet: A000037 NO Error in IMAP command received by server. " 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onRetrievalDone: "" 
posting retrieval request for item 7051  there are  1  queues and  0  items in mine 
request for item 7051 still pending - waiting 
processing retrieval request for item 7051  parts: ("RFC822")  of resource: "akonadi_imap_resource_0" 
continuing 
request for item 7051 "27404" failed: "Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>" 
ItemRetrieverException :  Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>
akonadi_imap_resource_0(2667) RetrieveItemsTask::onFinalSelectDone: Detected inconsistency in local cache, we're missing some messages. Server:  11  Local:  9 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onFinalSelectDone: Refetching complete mailbox. 
akonadi_imap_resource_0(2667) BatchFetcher::onUidSearchDone: Search job failed:  "Suche fehlgeschlagen, die Serverantwort lautet: A000046 NO Error in IMAP command received by server. " 
akonadi_imap_resource_0(2667) RetrieveItemsTask::onRetrievalDone: "" 
Database "akonadi" opened using driver "QMYSQL" 
SEARCH: 
        Query: "{ "limit" : -1, "negated" : false, "rel" : 1, "subTerms" : [ { "cond" : 0, "key" : "email", "negated" : false, "value" : "mmmm@1234.de" } ] }" 
        MimeTypes: ("text/directory") 
        Collections: QVector(115, 115) 
        Remote: false 
        Recursive true 
Executing search "kontact-196458353-SearchSession" 
akonadiserver(2616)/kdecore (KSycoca) KSycocaPrivate::openDatabase: Trying to open ksycoca from "/var/tmp/kdecache-bernd/ksycoca4"
Search done "kontact-196458353-SearchSession" (without remote search) 
        Result: 0 matches
Comment 14 goliath 2014-08-24 07:09:42 UTC
*** This bug has been confirmed by popular vote. ***
Comment 15 Daniel Wendler 2014-08-24 11:37:49 UTC
(In reply to Christoph Feck from comment #7)
> Is someone able to test if the regression is caused by the Akonadi update?
> In other words, keep newest Akonadi, but revert kontact/kmail2 to 4.13.3
> version?

I do have the same behavior on my arch system after upgrade to kde 4.14.
So i tried a few things:
* downgrade akonadi -> kmail doesn't start anymore (fatal error)
* downgrade kdepim packages but leave akonadi latest -> same behavior as with all on 4.14.
* downgrade akonadi -> things work again

(In reply to Christoph Feck from comment #9)
> Adding more people who might know where to look for regressions. It's still
> unclear if kmail2 or akonadi regressed.

Based on my findings, i think its an akonadi regression.
Comment 16 Christian Mollekopf 2014-08-24 15:43:31 UTC
We'll need an imap log (https://techbase.kde.org/Projects/PIM/Akonadi/Debug_IMAP)
and what imap server this is failing with.
Comment 17 Olivier Goffart 2014-08-25 07:18:59 UTC
Created attachment 88412 [details]
logs
Comment 18 Olivier Goffart 2014-08-25 07:20:10 UTC
I suffer from the same problem for one of my imap account. I have attached the log output and hope this helps.
Comment 19 Marcel Naziri 2014-08-25 14:28:43 UTC
I do have the same problem since upgrading to kde 4.14 on Arch connecting to a Courier IMAP Server (4.10.0 on Debian wheezy).
As a workaround I sync my mailboxes with offlineimap to a locally installed Dovecot server (2.1.7 on Debain wheezy) that kmail can play with perfectly.
Comment 20 Marcel Naziri 2014-08-25 15:01:14 UTC
Created attachment 88420 [details]
debug logs

My debug output is quite similar to Olivers. The error on the "UID SEARCH UID 1:-1" command looks remarkable.
Comment 21 s.h. 2014-08-25 17:45:18 UTC
Although I cannot provide any logs at the moment I noticed that all mails since the upgrade are corrupted on the imap server ! (checked via webfrontend).
The mail header does not correspond to the mail body anymore. It's a big mess and a lot of important mails are lost :(
Comment 22 Marcel Naziri 2014-08-25 19:32:01 UTC
No corruption for my by now (hopefully). I downgraded the kdepim-runtime package to version 4.13.3 with that I can sync with Courier again. The older runtime uses FETCH command instead of UID SEARCH which gives output like that

C: A000004 SELECT "INBOX"
S: * FLAGS ( $ATTACHMENT $Labelimportant \Draft \Answered \Flagged \Deleted \Seen \Recent )
S: * OK Limited [ PERMANENTFLAGS ( $ATTACHMENT $Labelimportant \* \Draft \Answered \Flagged \Deleted \Seen )  ]
S: * 421 EXISTS
S: * 0 RECENT
S: * OK Ok [ UIDVALIDITY 1178485368  ]
S: * OK ACL [ MYRIGHTS acdilrsw  ]
S: A000004 OK Ok [ READ-WRITE  ]
C: A000005 EXPUNGE
S: A000005 OK EXPUNGE completed
C: A000006 SELECT "INBOX"
S: * FLAGS ( $ATTACHMENT $Labelimportant \Draft \Answered \Flagged \Deleted \Seen \Recent )
S: * OK Limited [ PERMANENTFLAGS ( $ATTACHMENT $Labelimportant \* \Draft \Answered \Flagged \Deleted \Seen )  ]
S: * 421 EXISTS
S: * 0 RECENT
S: * OK Ok [ UIDVALIDITY 1178485368  ]
S: * OK ACL [ MYRIGHTS acdilrsw  ]
S: A000006 OK Ok [ READ-WRITE  ]
C: A000007 FETCH 1:421 (RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER] FLAGS UID)
S: * 1 FETCH ( RFC822.SIZE 5329 INTERNALDATE 06-May-2007 23:28:32 +0200 BODY[HEADER] Return-Path: <MAILER-DAEMON>
[...]

So kdepim-runtime seems to be affected. RetrieveItemsTask::onFinalSelectDone gets -1 values for nextUid which are not correctly mapped to an IMAP interval like 1:* instead it results in 1:-1

I hope someone can figure it out ...
Comment 23 Christoph Feck 2014-08-25 20:50:40 UTC
*** Bug 338557 has been marked as a duplicate of this bug. ***
Comment 24 Thomas Tanghus 2014-08-25 21:06:14 UTC
Same problem here using Courier IMAP. Do you need more debug info?
Comment 25 buddle1313 2014-08-26 08:07:14 UTC
I confirm this bug also, with Server:

"[CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc."

If I switch back to kdepimruntime-4:4.13.3, everything works on Kubuntu 4.14.0!
Comment 26 Thomas Tanghus 2014-08-26 10:25:08 UTC
(In reply to buddle1313 from comment #25)
> If I switch back to kdepimruntime-4:4.13.3, everything works on Kubuntu
> 4.14.0!

How exactly do you do that?
Comment 27 Christian Mollekopf 2014-08-26 13:17:42 UTC
> So kdepim-runtime seems to be affected. RetrieveItemsTask::onFinalSelectDone
> gets -1 values for nextUid which are not correctly mapped to an IMAP
> interval like 1:* instead it results in 1:-1

That does sound like a server bug, or at least I don't know when UIDNEXT < 0 would make any sense.

I suppose we can catch this case and just fallback a very large UIDNEXT value.

Can other's confirm this is indeed the problem?
Comment 28 Christoph Feck 2014-08-26 14:05:46 UTC
*** Bug 338572 has been marked as a duplicate of this bug. ***
Comment 29 Christoph Feck 2014-08-26 14:06:40 UTC
*** Bug 338567 has been marked as a duplicate of this bug. ***
Comment 30 Thomas Gideon 2014-08-26 14:29:05 UTC
Downgrading kdepim-runtime to 4.13.3 also worked for me, restoring access to my Courier IMAP account.

For Thomas Tanghus, the command I used is:

sudo apt-get install kdepim-runtime=4:4.13.3-0ubuntu0.1~ubuntu14.04.1~ppa1

You'll probably want to run "apt-cache madison kdepim-runtime" since depending on what sources you have, the 4.13.3 package may have a slightly different version.
Comment 31 Wolfgang Schindler 2014-08-26 14:35:49 UTC
Did the same via synaptic on debian 3.14 with an courier IMAP Server.
Worked like a charm. Thanks

On 26.08.2014 16:29, Thomas Gideon wrote:
> https://bugs.kde.org/show_bug.cgi?id=338186
>
> --- Comment #30 from Thomas Gideon <cmdln@thecommandline.net> ---
> Downgrading kdepim-runtime to 4.13.3 also worked for me, restoring access to my
> Courier IMAP account.
>
> For Thomas Tanghus, the command I used is:
>
> sudo apt-get install kdepim-runtime=4:4.13.3-0ubuntu0.1~ubuntu14.04.1~ppa1
>
> You'll probably want to run "apt-cache madison kdepim-runtime" since depending
> on what sources you have, the 4.13.3 package may have a slightly different
> version.
>
Comment 32 Thomas Tanghus 2014-08-26 15:13:08 UTC
(In reply to Thomas Gideon from comment #30)
> For Thomas Tanghus, the command I used is:
> 
> sudo apt-get install kdepim-runtime=4:4.13.3-0ubuntu0.1~ubuntu14.04.1~ppa1

Thanks a bunch - I've now got a working IMAP account \o/
Comment 33 Marcel Naziri 2014-08-26 15:26:16 UTC
Created attachment 88438 [details]
IMAP log comparison Dovecot and Courier
Comment 34 Marcel Naziri 2014-08-26 15:42:15 UTC
(In reply to Christian Mollekopf from comment #27)
> > So kdepim-runtime seems to be affected. RetrieveItemsTask::onFinalSelectDone
> > gets -1 values for nextUid which are not correctly mapped to an IMAP
> > interval like 1:* instead it results in 1:-1
> 
> That does sound like a server bug, or at least I don't know when UIDNEXT < 0
> would make any sense.

As you can see in the formerly attached log the Courier imap server does not give a nextuid prediction, so the value -1 comes from the default value of SelectJob. I don't know if Courier breaks any rules not to answer with UIDNEXT.
 
> I suppose we can catch this case and just fallback a very large UIDNEXT
> value.

I made a quick patch to set the nextuid value to 0 in RetrieveItemsTask::onFinalSelectDone in case it's set to -1 before the FetchJob is build. This way the ImapInterval will result not in "1:-1" but in "1:*". According to RFC 3501 "*" represents the largest number in use.

This way the sync with kdepim 4.14 and Courier worked again. I hope this gives a good starting point for a proper fix.
Comment 35 Søren Holm 2014-08-26 19:35:11 UTC
Downgrading to this version works for me on kubuntu 14.10

http://launchpadlibrarian.net/172530639/kdepim-runtime_4.13.0-0ubuntu1_amd64.deb
Comment 36 Thomas Gideon 2014-08-26 19:44:39 UTC
I apologize if this comment isn't strictly appropriate for the bug, but another thing I found that will help folks continue to test other parts of current development without breaking the workaround of downgrading kdepim-runtime is to use

"sudo apt-mark hold kdepim-runtime"

Once a fixed package is available,

"sudo apt-mark unhold kdepim-runtime"

followed by an update and upgrade should pick up the latest, again.
Comment 37 Marcel Naziri 2014-08-26 20:54:33 UTC
Created attachment 88443 [details]
Courier IMAP session log

Played a bit with the Courier IMAP server. Looks like UIDNEXT value can be gathered by the STATUS command.
Comment 38 Marcel Naziri 2014-08-26 22:11:13 UTC
Created attachment 88445 [details]
let ImapInterval handle negative value

I applied the attached patch to kdepimlibs which make kdepim 4.14.0 work for me with Courier again. It lets ImapInterval interpret also negative values for begin and end as unset. This way the missing UIDNEXT value will at least result in a valid UID SEARCH command with parameter "1:*"

It does not resolve the problem itself that no proper nextuid value can be stored for further optimization. So on every check of the folder all message headers are fetched. But at least the sync is not broken completely.
Comment 39 Christian Mollekopf 2014-08-27 10:46:54 UTC
Created attachment 88453 [details]
patch for fallback with invalid uidnext

Can somebody try this patch for kdepim-runtime?
Comment 40 Thomas Tanghus 2014-08-28 15:17:22 UTC
(In reply to Christian Mollekopf from comment #39)
> Created attachment 88453 [details]
> patch for fallback with invalid uidnext
> 
> Can somebody try this patch for kdepim-runtime?

What are the steps for testing it?
Comment 41 Carsten Pfeiffer 2014-08-28 20:01:24 UTC
I just tried the patch on Debian with Courier 4.10, but now kmail doesn't show any mails at all anymore (except for one strange mail in one particular folder).

I get the following output:

akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::LoginJob::LoginJob: KIMAP::LoginJob(0x26e1ac0)
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::SessionThread::reconnect: connectToHost "hosti.leonde.de" 993
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::LoginJob::doStart: KIMAP::LoginJob(0x26e1ac0)
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::SessionThread::sslConnected: TLS negotiation done.
SEARCH: 
        Query: "{ "limit" : -1, "negated" : false, "rel" : 1, "subTerms" : [ { "cond" : 0, "key" : "email", "negated" : false, "value" : "my@emailaddress.de" } ] }" 
        MimeTypes: ("text/directory") 
        Collections: QVector(131, 131) 
        Remote: false 
        Recursive true 
Executing search "kontact-1492973798-SearchSession" 
Search done "kontact-1492973798-SearchSession" (without remote search) 
        Result: 0 matches 
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::LoginJob::handleResponse: "Capability" "*"
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::LoginJob::handleResponse: Capabilities updated:  ("IMAP4rev1", "UIDPLUS", "CHILDREN", "NAMESPACE", "THREAD=ORDEREDSUBJECT", "THREAD=REFERENCES", "SORT", "QUOTA", "IDLE", "AUTH=PLAIN", "ACL", "ACL2=UNION")
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::LoginJob::handleResponse: "Capability" "A000001"
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::LoginJob::handleResponse: "Login" "A000002"
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::startRetrievalTasks: Starting retrieval for  "INBOX.uni"
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) KIMAP::LoginJob::~LoginJob: KIMAP::LoginJob(0x26e1ac0)
akonadi_imap_resource_2(23669) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value 
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: Starting message retrieval. Elapsed(ms):  195
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: MessageCount:  4891 Local message count:  1
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: UidNext:  0 Local UidNext:  0
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: HighestModSeq:  0 Local HighestModSeq:  0
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: Invalid UIDNEXT
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: Fetching complete mailbox  "INBOX.uni"
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) BatchFetcher::fetchNextBatch: Fetching  1  intervals
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::taskComplete: Retrieval complete. Elapsed(ms):  351
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveCollectionMetadataTask::doStart: ".uni"
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::startRetrievalTasks: Starting retrieval for  "INBOX.uni"
akonadi_imap_resource_2(23669) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value 
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: Starting message retrieval. Elapsed(ms):  135
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: MessageCount:  4891 Local message count:  1
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: UidNext:  0 Local UidNext:  0
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: HighestModSeq:  0 Local HighestModSeq:  0
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: Invalid UIDNEXT
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::onFinalSelectDone: Fetching complete mailbox  "INBOX.uni"
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) BatchFetcher::fetchNextBatch: Fetching  1  intervals
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveItemsTask::taskComplete: Retrieval complete. Elapsed(ms):  291
akonadi_imap_resource_2(23669)/kdepimlibs (kimap) RetrieveCollectionMetadataTask::doStart: ".uni
Comment 42 Christian Mollekopf 2014-08-29 08:07:59 UTC
Created attachment 88482 [details]
version 2 of patch for fallback with invalid uidnext

The last patch was indeed incomplete, sorry about that. This one should do better.
Comment 43 Carsten Pfeiffer 2014-08-29 10:26:17 UTC
Thanks Christian, that one works. kmail shows all mails again, but everytime I switch folders, it will synchronize the entire folder.

I guess that was to be expected from the discussion here, so I hope that this is a quick fix and not the solution.
Comment 44 Christian Mollekopf 2014-08-29 11:10:50 UTC
(In reply to Carsten Pfeiffer from comment #43)
> Thanks Christian, that one works. 

Great, thanks for testing.

> kmail shows all mails again, but everytime
> I switch folders, it will synchronize the entire folder.
> 

Yes, that is expected.

> I guess that was to be expected from the discussion here, so I hope that
> this is a quick fix and not the solution.

I don't plan on working on this any further. It could be improved, but courier should just send UIDNEXT which is what is specified in RFC 3501 (IMAP). Note that this simply reestablishes what was done before (You just didn't have any debug messages telling you that a full sync was executed).
Comment 45 Carsten Pfeiffer 2014-08-29 12:02:10 UTC
I don't think the behavior is the same as before. Previously it would switch folders and display mails immediately. Now it only shows the preselected mail immediately. When selecting another mail, I have to wait for the full sync to finish before the contents of the next mail is displayed. The full sync can take minutes, depending on the size of the folder and the connection.

So 
- either it did not make a full sync previously,
- or the full sync was way faster so the delay was not noticeable
- or the displaying of selected mails would not need for the full sync to finish

In any case, this makes kmail with courier and not-so-tiny folders hardly usable, even if courier is to blame for the lack of UIDNEXT support.
Comment 46 Thomas Tanghus 2014-08-29 12:11:29 UTC
(In reply to Carsten Pfeiffer from comment #45)
> In any case, this makes kmail with courier and not-so-tiny folders hardly
> usable, even if courier is to blame for the lack of UIDNEXT support.

Isn't it possible to see what has changed between 4.13.3(4?) and 4.14? 4.13.x managed Courier servers without this problem.
Comment 47 didi.debian 2014-08-29 12:13:20 UTC
(In reply to Thomas Tanghus from comment #46)
> Isn't it possible to see what has changed between 4.13.3(4?) and 4.14?
> 4.13.x managed Courier servers without this problem.

+ 1
Comment 48 Marcel Naziri 2014-08-29 12:39:38 UTC
I don't see it really as Bug in Courier. RFC 3501 says
     "Note that earlier versions of this protocol only required the
      FLAGS, EXISTS, and RECENT untagged data; consequently, client
      implementations SHOULD implement default behavior for missing data
      as discussed with the individual item."

I don't expect to see the requested feature in old rusty Courier Mail Server soon - if at all. So this should be handled by kmail with appropriate action. Isn't it possible to fallback to the behaviour in 4.13.x in this case?
Comment 49 Marcel Naziri 2014-08-29 13:15:31 UTC
Anyone tested my patch for ImapInterval in kdepimlibs? For me it's sufficient to work with courier again using the actual implementation of kdelibs-runtime.
With the patch I can't see a complete fetching of the mail headers when just switching to another message or changing folders. It only happens when the mailbox has changed, e.g. new mail arrived.
Comment 50 Carsten Pfeiffer 2014-08-29 14:04:22 UTC
(In reply to Marcel Naziri from comment #49)
> Anyone tested my patch for ImapInterval in kdepimlibs?

Will do so tonight or tomorrow night. Completely missed that the patch is attached.
Comment 51 Marcel Naziri 2014-08-29 15:17:21 UTC
(In reply to Carsten Pfeiffer from comment #50)
> (In reply to Marcel Naziri from comment #49)
> > Anyone tested my patch for ImapInterval in kdepimlibs?
> 
> Will do so tonight or tomorrow night. Completely missed that the patch is
> attached.

I realized that with my patch flags (e.g. read/unread) changed by another client aren't reflected in kmail when refreshing the folder. not until mailbox content is changed (new mail arrived or something moved) the flags are updated.

Is this also happening with Christians patch v2?
Comment 52 Carsten Pfeiffer 2014-08-29 15:38:36 UTC
(In reply to Marcel Naziri from comment #51)
> I realized that with my patch flags (e.g. read/unread) changed by another
> client aren't reflected in kmail when refreshing the folder. not until
> mailbox content is changed (new mail arrived or something moved) the flags
> are updated.
> 
> Is this also happening with Christians patch v2?

No, this works fine with Christian's patch.
Comment 53 Christian Mollekopf 2014-08-29 15:53:26 UTC
(In reply to Marcel Naziri from comment #51)
> (In reply to Carsten Pfeiffer from comment #50)
> > (In reply to Marcel Naziri from comment #49)
> > > Anyone tested my patch for ImapInterval in kdepimlibs?
> > 
> > Will do so tonight or tomorrow night. Completely missed that the patch is
> > attached.
> 
> I realized that with my patch flags (e.g. read/unread) changed by another
> client aren't reflected in kmail when refreshing the folder. not until
> mailbox content is changed (new mail arrived or something moved) the flags
> are updated.
> 
> Is this also happening with Christians patch v2?

Please note that your patch is really rather a workaround that cannot be upstreamed. Thanks a lot for your analysis though as it pointed me in the right direction.

What's currently missing from my patch is that we are not using the optimization that avoid downloading all messages on every sync. This is because we usually rely on a combination of message-count and uidnext to detect if new messages are available or old ones have been deleted. Whitout uidnext we could still avoid a sync if the messagecount remains the same, but then we would run into the problem that we for instance would never notice if i.e. one message has been removed and another one added (which is nasty because it's very unpredictable).
Comment 54 Marcel Naziri 2014-08-29 16:34:13 UTC
(In reply to Christian Mollekopf from comment #53)
> (In reply to Marcel Naziri from comment #51)
> > (In reply to Carsten Pfeiffer from comment #50)
> What's currently missing from my patch is that we are not using the
> optimization that avoid downloading all messages on every sync. This is
> because we usually rely on a combination of message-count and uidnext to
> detect if new messages are available or old ones have been deleted. Whitout
> uidnext we could still avoid a sync if the messagecount remains the same,
> but then we would run into the problem that we for instance would never
> notice if i.e. one message has been removed and another one added (which is
> nasty because it's very unpredictable).

Is the nextuid always monotonic increasing value or is it also lowered by the mail server in some circumstances?

I will test your patch tongiht or tomorrow.
Comment 55 Carsten Pfeiffer 2014-08-30 11:48:18 UTC
(In reply to Christian Mollekopf from comment #53)
> Whitout
> uidnext we could still avoid a sync if the messagecount remains the same,
> but then we would run into the problem that we for instance would never
> notice if i.e. one message has been removed and another one added (which is
> nasty because it's very unpredictable).

Marcel said that Courier would report a proper uidnext value when issuing the STATUS command. Wouldn't it be a good workaround to issue the STATUS command in case UIDNEXT is otherwise not available?
Comment 56 Carsten Pfeiffer 2014-09-01 07:34:58 UTC
> > Anyone tested my patch for ImapInterval in kdepimlibs?
> 
> Will do so tonight or tomorrow night. Completely missed that the patch is
> attached.

I tested it and *for me* it is a better workaround than Christian's patch v2, because of the lack of full sync. I mostly use just one client anyway, so missing external status changes is not really a problem for me.
Comment 57 Marcel Naziri 2014-09-01 13:12:43 UTC
I've tested Christians patch and with it a refresh of just my inbox folder take about 1 minute 40 seconds. So kmail would get unusable for me. :(

I dumped a imap debug log of thunderbird. It utilizes a "UID FETCH <ImapSet> FLAGS" command that returns a list of mails with UIDs and corresponding flags. seems to be a quick way to sync flags and detect deleted mails without having all the mail headers downloaded.
also thunderbird doesn't always use an interval like 1:* for this command. It seems that it takes the highest seen uid + 1 as prediction, so I've seen a command like "UID FETCH 443:* FLAGS" when 442 was the highest uid in the list before.

Here is a little netcat chat log with a Courier Imap 4.15 (Debian testing)

a01 CAPABILITY
* CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION
a01 OK CAPABILITY completed
a02 SELECT INBOX
* FLAGS (Junk $ATTACHMENT NonJunk \Draft \Answered \Flagged \Deleted \Seen \Recent)
* OK [PERMANENTFLAGS (Junk $ATTACHMENT NonJunk \* \Draft \Answered \Flagged \Deleted \Seen)] Limited
* 4 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 409534394] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
a02 OK [READ-WRITE] Ok
a03 UID FETCH 1:* FLAGS
* 1 FETCH (UID 1 FLAGS (\Flagged))
* 2 FETCH (UID 2 FLAGS (\Seen))
* 3 FETCH (UID 4 FLAGS ($ATTACHMENT NonJunk))
* 4 FETCH (UID 5 FLAGS (Junk))
a03 OK FETCH completed.
a04 UID FETCH 6:* FLAGS
a04 OK FETCH completed.
Comment 58 Rigo Wenning 2014-09-01 14:39:55 UTC
I installed the patch v2 via some OpenSuse package. It re-fetched all messages from the Server (for me this was much). But it is also very slow. I have a large inbox of about 5500 messages. It takes minutes on a 1.2MB/s line to synchronize for each synchronization when it should only synchronize the diff. kdepim-runtime 4.13.3 was so much faster. 
And yes, my imap server is Courier
Comment 59 Rigo Wenning 2014-09-01 15:32:15 UTC
I have backup folders per month for my email. With the patch, kmail downloaded those backup folders now for the third time while it is guaranteed that there was no change. This took seconds in kdepim-runtime 4.13.3 So the patch really needs improvement. It doesn't work like this for folders with many emails
Comment 60 Laurent Montel 2014-09-01 17:19:36 UTC
*** Bug 338733 has been marked as a duplicate of this bug. ***
Comment 61 Rigo Wenning 2014-09-05 08:04:44 UTC
(In reply to Rigo Wenning from comment #59)
as recommended, I downgraded to kdepimruntime-4.13.3. But this messed up akonadi. I use expiry and Akonadi now has 6 duplicates of each message in the folder where the expired messages are filed into. But on the local disc, I only see a fraction of that email. Note sure even all messages are there. So using  kdepimruntime-4.13.3. with KDE 4.14 may expose you to data loss. Other comment still counts, the patch is using all bandwidth as it downloads the entire imap content with every poll. For me, this is Gigabytes. Doesn't work on ADSL. This to say that users who upgraded to 4.14 and have Courier are mostly stuck. Either use another email client or downgrade all KDE to 4.13.3
Comment 62 Vadim Dyadkin 2014-09-06 08:36:14 UTC
Do not apply the patch v2! Through the last night kmail filled all my hard drive with mail duplicates.
Comment 63 Wolfgang Schindler 2014-09-08 07:44:45 UTC
(In reply to Rigo Wenning from comment #61)
> (In reply to Rigo Wenning from comment #59)
> as recommended, I downgraded to kdepimruntime-4.13.3. But this messed up
> akonadi. I use expiry and Akonadi now has 6 duplicates of each message in
> the folder where the expired messages are filed into. But on the local disc,
> I only see a fraction of that email. Note sure even all messages are there.
> So using  kdepimruntime-4.13.3. with KDE 4.14 may expose you to data loss.
> Other comment still counts, the patch is using all bandwidth as it downloads
> the entire imap content with every poll. For me, this is Gigabytes. Doesn't
> work on ADSL. This to say that users who upgraded to 4.14 and have Courier
> are mostly stuck. Either use another email client or downgrade all KDE to
> 4.13.3

+1
Comment 64 Freek de Kruijf 2014-09-08 08:46:17 UTC
I would suggest to have two IMAP resources to choose from when adding an account to KMail. One with the new capability and one with which is a copy of the one in the previous version of akonadi. Obviously with some comment about which RFC is supported or with a comment on the new one that if that one does not have the expected behavior one can try the other one.
Comment 65 Christian Mollekopf 2014-09-09 19:14:46 UTC
Created attachment 88636 [details]
version 3 of patch for fallback with invalid uidnext

I think this should now restore functionality as before. Can somebody try if it works?
A proper fix would be to implement the STATUS command. I won't do that but you're welcome to supply patches.
Comment 66 Christoph Feck 2014-09-09 20:03:49 UTC
Thanks Christian for the updated patch. I hope someone is able to stress test it and give feedback before the 4.14.1 tagging on Thursday.
Comment 67 Marcel Naziri 2014-09-10 02:45:02 UTC
(In reply to Christian Mollekopf from comment #65)
> A proper fix would be to implement the STATUS command. I won't do that but
> you're welcome to supply patches.

After reading in RFC 3501 about the STATUS command I doubt that it would be the right way. :( It says that STATUS should not be used on the currently selected mailbox. Also this command can be quite slow.

On the other hand I found by chatting with a Courier IMAP server via netcat that the STATUS result values like MESSAGES, RECENT, UIDNEXT, UIDVALIDITY and UNSEEN are only updated if prior to this a SELECT or EXAMINE command was executed. So Courier seems not to be conform to the RFC.
Comment 68 Vadim Dyadkin 2014-09-10 07:43:01 UTC
I have been testing the patch v3. Everything works as it was before in 4.13.3. Thanks Christian for it.
Comment 69 Christian Mollekopf 2014-09-10 08:22:26 UTC
Git commit ff02785839e9b41b587fac5ca9bc2768a2729eec by Christian Mollekopf.
Committed on 10/09/2014 at 08:21.
Pushed by cmollekopf into branch 'KDE/4.14'.

IMAP-Resource: Detect invalid UIDNEXT values and fall-back to sequence number based fetches.

We run into this codepath with courier imap.

M  +7    -0    resources/imap/batchfetcher.cpp
M  +35   -11   resources/imap/retrieveitemstask.cpp
M  +36   -1    resources/imap/tests/testretrieveitemstask.cpp

http://commits.kde.org/kdepim-runtime/ff02785839e9b41b587fac5ca9bc2768a2729eec
Comment 70 Christian Mollekopf 2014-09-10 08:48:23 UTC
(In reply to Marcel Naziri from comment #67)
> (In reply to Christian Mollekopf from comment #65)
> > A proper fix would be to implement the STATUS command. I won't do that but
> > you're welcome to supply patches.
> 
> After reading in RFC 3501 about the STATUS command I doubt that it would be
> the right way. :( It says that STATUS should not be used on the currently
> selected mailbox. 

This could be worked around by deselecting first.

> Also this command can be quite slow.

Yeah, I guess it depends on the definition on slow. If you want something fast you better use another IMAP server.

> 
> On the other hand I found by chatting with a Courier IMAP server via netcat
> that the STATUS result values like MESSAGES, RECENT, UIDNEXT, UIDVALIDITY
> and UNSEEN are only updated if prior to this a SELECT or EXAMINE command was
> executed. So Courier seems not to be conform to the RFC.

Indeed, but in that case we could run STATUS after a SELECT that didn't return UIDNEXT.
Anways, I'm not going to work on this any further.
Comment 71 Rigo Wenning 2014-09-10 12:10:17 UTC
(In reply to Christian Mollekopf from comment #70)
Christian, if I change to another imap-server than Courier, the performance improvements would still work, right? You haven't just reverted to the 4.13.3 state, I hope.
Comment 72 Christian Mollekopf 2014-09-10 12:18:27 UTC
(In reply to Rigo Wenning from comment #71)
> (In reply to Christian Mollekopf from comment #70)
> Christian, if I change to another imap-server than Courier, the performance
> improvements would still work, right? You haven't just reverted to the
> 4.13.3 state, I hope.

No, I simply added an additional codepath that is only used if UIDNEXT is not delivered on SELECT. So other imap servers should not be affected at all by this change.
Comment 73 Marcel Naziri 2014-09-10 13:16:17 UTC
Big thanks to Christian for his work!
Sooner or later I want to migrate to Dovecot anyway. So I'm happy with the solution that kmail works with Courier like it did in 4.13.
Comment 74 Søren Holm 2014-09-10 15:10:19 UTC
I use Dovecot 2.2.9 and have this problem also, so that wont help you I guess.
Comment 75 Alexander Elbs 2014-09-11 06:09:24 UTC
I had the same trouble with an exchange imap server (* OK Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.7638.1).
The patch fixes this. Thanks.
Comment 76 Marcel Naziri 2014-09-24 21:45:42 UTC
I updated to kde 4.14.1 on Arch (testing repository) and kmail is still not usable for me. A refresh of my inbox takes about 1 minute. Even marking an email as read (manually or automatically) will trigger such an long-lasting refresh. selecting another email in the list will show up in the preview not before the refresh is done.
Wasn't the patch applied for 4.14.1 or doesn't it bring back the behaviour of 4.13.x when UIDNEXT is missing?
Comment 77 Thomas Tanghus 2014-10-14 18:46:29 UTC
(In reply to Marcel Naziri from comment #76)
> I updated to kde 4.14.1 on Arch (testing repository) and kmail is still not
> usable for me. A refresh of my inbox takes about 1 minute. Even marking an
> email as read (manually or automatically) will trigger such an long-lasting
> refresh. selecting another email in the list will show up in the preview not
> before the refresh is done.
> Wasn't the patch applied for 4.14.1 or doesn't it bring back the behaviour
> of 4.13.x when UIDNEXT is missing?

Anybody else experiencing this? I'm not that eager to update to a half-functioning kmail ;)
Comment 78 Søren Holm 2014-10-14 19:46:46 UTC
Current 14.10 does not suffer from this problem anymore
Comment 79 Thomas Tanghus 2014-10-14 21:01:08 UTC
(In reply to Søren Holm from comment #78)
> Current 14.10 does not suffer from this problem anymore

Kubuntu 14.10 is still in RC state, right? Been on vacation, so haven't been following ;)
Is that shipping with 4.14.1?
Comment 80 Vladislav Vorobiev 2014-11-07 11:26:08 UTC
Any progress on this BUG?
Version Fixed In:	4.14.1 ??

I have here 4:4.14.2-2 and still have this bug

posting retrieval request for item 86502  there are  1  queues and  0  items in mine 
request for item 86502 still pending - waiting 
processing retrieval request for item 86502  parts: ("RFC822")  of resource: "akonadi_imap_resource_11" 
akonadi_imap_resource_11(24819) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
QDBusObjectPath Akonadi::Server::NotificationManager::subscribeV2(const QString&, bool) Akonadi::Server::NotificationManager(0x1a19910) "akonadi_newmailnotifier_agent_24852_ECGjdZ" true 
QDBusObjectPath Akonadi::Server::NotificationManager::subscribeV3(const QString&, bool, bool) Akonadi::Server::NotificationManager(0x1a19910) "akonadi_newmailnotifier_agent_24852_ECGjdZ" true false 
continuing 
request for item 86502 succeeded
Comment 81 Wolfgang Schindler 2014-11-10 12:36:30 UTC
Same here on Debian

On 07.11.2014 12:26, Vladislav Vorobiev wrote:
> https://bugs.kde.org/show_bug.cgi?id=338186
>
> Vladislav Vorobiev <v@piratweb.com> changed:
>
>             What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                   CC|                            |v@piratweb.com
>
> --- Comment #80 from Vladislav Vorobiev <v@piratweb.com> ---
> Any progress on this BUG?
> Version Fixed In:    4.14.1 ??
>
> I have here 4:4.14.2-2 and still have this bug
>
> posting retrieval request for item 86502  there are  1  queues and  0  items in
> mine
> request for item 86502 still pending - waiting
> processing retrieval request for item 86502  parts: ("RFC822")  of resource:
> "akonadi_imap_resource_11"
> akonadi_imap_resource_11(24819) RetrieveItemsTask::onFinalSelectDone: Server
> bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known
> problem with Courier IMAP.
> QDBusObjectPath Akonadi::Server::NotificationManager::subscribeV2(const
> QString&, bool) Akonadi::Server::NotificationManager(0x1a19910)
> "akonadi_newmailnotifier_agent_24852_ECGjdZ" true
> QDBusObjectPath Akonadi::Server::NotificationManager::subscribeV3(const
> QString&, bool, bool) Akonadi::Server::NotificationManager(0x1a19910)
> "akonadi_newmailnotifier_agent_24852_ECGjdZ" true false
> continuing
> request for item 86502 succeeded
>
Comment 82 eaThiethaeroo1aemaun2i 2015-03-06 14:21:50 UTC
Same with kubuntu-backports (4.14.2) - looks like it is still present.
Comment 83 eaThiethaeroo1aemaun2i 2015-03-06 14:53:09 UTC
I've created a new bug #344904 regarding 4.14.2, maybe you can verify if this covers your problem too.
Comment 84 Colin J Thomson 2015-04-27 20:10:28 UTC
I've just been hit with this after changing my Email service provider :-(  F21 - 4.14.7
Comment 85 Michael Fuchs 2015-04-29 16:47:52 UTC
WORKAROUND:
One of my providers also uses courier.
After creating a new account, only messages from subfolders got fetched but not the ones from INBOX. After enabling disconnected imap ("Download all messages for offline use") also mails from INBOX come in.
Comment 86 Colin J Thomson 2015-04-29 20:16:07 UTC
I can confirm Michaels WORKAROUND is working for me, Thanks.
Comment 87 Michael Fuchs 2015-11-17 11:26:48 UTC
This is defenitly not Fixed!
Kmail 5.03 still has this problem and even more, it doesnt work with dimap.

I don't understand the difficulties. Thunderbird, Outlook and Trojita have no Problem with Courier-IMAP. They are even much faster.

Please also note, that this problem will *not* be solved at the server-side:

http://sourceforge.net/p/courier/mailman/message/33686723/
Comment 88 Freek de Kruijf 2015-11-17 11:50:25 UTC
Op dinsdag 17 november 2015 11:26:48 schreef u:
> https://bugs.kde.org/show_bug.cgi?id=338186
> 
> --- Comment #87 from Michael Fuchs <michfu@gmx.at> ---
> This is defenitly not Fixed!
> Kmail 5.03 still has this problem and even more, it doesnt work with dimap.
> 
> I don't understand the difficulties. Thunderbird, Outlook and Trojita have
> no Problem with Courier-IMAP. They are even much faster.
> 
> Please also note, that this problem will *not* be solved at the server-side:
> 
> http://sourceforge.net/p/courier/mailman/message/33686723/

I solved the problem at the server-side by replacing courier with dovecot. It 
turned out that this was quite easy. I looked at this change as a mountain, 
but it appeared to be a hill.
Comment 89 Michael Fuchs 2015-11-28 12:08:30 UTC
Unaccpetable Solution.

Not everybody has the choice, which server hosts their emails and which software is running on it.
Comment 90 Michael Fuchs 2015-12-21 15:31:27 UTC
With Kmail 5.1 (kmail 15.12 on ArchLinux) I found another workaround:

This time you got to disable "Enable server-side subscriptions" then the mails in inbox also appear. Changing the "Download all meeages for offline use." option doesn't have a positive nor negative effect.
Comment 91 Vladislav Vorobiev 2015-12-28 23:04:40 UTC
Why we see Status: RESOLVED FIXED
Version Fixed In:	4.14.1

Nothing is fixed, I have here 4.14.10-2 and akonadi-server 1.13.0-8

I't happends every 3 minutes on my PC.

is a known problem with Courier IMAP. 
akonadi_imap_resource_11(14518) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_32(14536) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_30(14532) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_26(14523) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_35(14540) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_11(14518) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_25(14522) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_30(14532) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_26(14523) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_11(14518) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_25(14522) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_26(14523) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_11(14518) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_25(14522) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_26(14523) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_11(14518) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_25(14522) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
213098 "2377" "" 
213099 "2377" "" 
akonadi_imap_resource_26(14523) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
akonadi_imap_resource_34(14539) RetrieveItemsTask::onFinalSelectDone: Server bug: Your IMAP Server delivered an invalid UIDNEXT value. This is a known problem with Courier IMAP. 
continuing 
request for item 213275 "19546" failed: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken." 
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
posting retrieval request for item 213251  there are  1  queues and  0  items in mine 
request for item 213251 still pending - waiting 
processing retrieval request for item 213251  parts: ("RFC822")  of resource: "akonadi_imap_resource_28" 

I fix it with 
$ akonadictl stop
$ akonadictl start

every time and i have this probleme more or less progressive about one year.
Currently I think to drop kontact from my system and migrate to thunderbirth....
Kontact is Unusable, this issue is so important and kde-team don't take care to fix it.
Comment 92 Vladislav Vorobiev 2015-12-28 23:11:40 UTC
It seems that 	Christian Mollekopf isn't active. Any Idea how to reasignt this bug to somebody else who can fix it?
Comment 93 Rigo Wenning 2015-12-30 10:11:56 UTC
I guess this has been set to fixed because it isn't a bug of kmail or akonadi, but rather a problem with the courier imap server that does not implement uidnext. I have solved this by using offlineimap until I get our BOFH to switch servers. I then run a local instance of dovecot, which is rather easy to install. Perhaps you have more luck encouraging the courier folks to fix their old software.
Comment 94 Michael Fuchs 2015-12-30 11:38:50 UTC
As I wrote before the courier devs won't solve this bug, because they don't think it actually *is* a bug.

http://sourceforge.net/p/courier/mailman/message/33686723/

KMail is the only IMAP-Client i know, which has such massive problems with courier servers.
Comment 95 Vladislav Vorobiev 2016-01-03 10:06:18 UTC
Currently i migrated from courier to dovecot (seems to work) but
this is also defenetly massive an akonadi/kmail bug because.

No external service like courier-imap should produce so massive problems by working with Kontact (Suite). After some minutes akonadi  stop to work and makes kontact including calendar/todos unusable and need to be restarted like killall -9 kontact; akonadictl stop; akonadictl start.
KMail (akonadi_imap) should don't influence other services.

Status: RESOLVED FIXED is not correct.
Comment 96 Alvin 2016-01-03 10:09:30 UTC
On does not simply ask their IMAP provider to switch software. This is a regression and marking it as RESOLVED FIXED is not correct.
Comment 97 Vladislav Vorobiev 2016-01-03 10:37:41 UTC
Many reason to fix it in other way.
Everybody how try to use kontact, kmain, akondi with broken imap (like courier) and don't find this tread, will think kmail (kontact) is broken and buggy software. Note that kontact is a important part of KDE.

Thats a reputation of KDE depends also on this small komponent.
Comment 98 Grishin 2016-02-02 18:48:41 UTC
(In reply to Vladislav Vorobiev from comment #97)
> Many reason to fix it in other way.
> Everybody how try to use kontact, kmain, akondi with broken imap (like
> courier) and don't find this tread, will think kmail (kontact) is broken and
> buggy software. Note that kontact is a important part of KDE.
> 
> Thats a reputation of KDE depends also on this small komponent.

В таком случае, можно рекомендовать разработчикам указать данное замечание в документации или, лучше, в самом kmail, чтобы пользователи не были смущены проблемой. 
Сам же считаю kmail лучшей программой электронной почты, но печально, что возникают такие нерешаемые проблемы..
Comment 99 Stephan Lahl 2016-02-17 20:43:54 UTC
Not resolved, not fixed.
KMail 4.14.3 on KDE 4.14.13 (Ubuntu 14.04 LTS) is completely broken.
Every 4 or 5 mails one has to yell some akondactl restart at a shell.
If you bugfix is to install other servers - this is unacceptable. 
The real bugfix in this case is to switch from KMail to Thunderbird.
Comment 100 Stephan Lahl 2016-02-17 20:44:59 UTC
Not resolved, not fixed.
KMail 4.14.3 on KDE 4.14.13 (Ubuntu 14.04 LTS) is completely broken.
Every 4 or 5 mails one has to yell some akondactl restart at a shell.
If you bugfix is to install other servers - this is unacceptable. 
The real bugfix in this case is to switch from KMail to Thunderbird.
Comment 101 Giovanni Beltrame 2016-02-25 02:25:59 UTC
I agree on the not resolved/not fixed. This is a major issue since many people don't have a choice on the software used by their email provider (especially work ones).
Comment 102 Daniel Vrátil 2016-03-22 13:24:35 UTC
Git commit 21ba8042d8f73992d18f9282743312d9ce84c22d by Daniel Vrátil.
Committed on 22/03/2016 at 13:18.
Pushed by dvratil into branch 'Applications/16.04'.

IMAP: Better handling of missing UIDNEXT response in RetrieveItemsTask

This improves experience with Courier IMAP servers which does not return UIDNEXT in
SELECT response. That forced us to always refetch the entire mailbox. Courier however
does return UIDNEXT in response to STATUS, so we now issue a STATUS command in case
SELECT gives us empty UIDNEXT.

REVIEW: 127457
FIXED-IN: 16.04

M  +7    -4    resources/imap/autotests/testretrieveitemstask.cpp
M  +115  -75   resources/imap/retrieveitemstask.cpp
M  +11   -1    resources/imap/retrieveitemstask.h

http://commits.kde.org/kdepim-runtime/21ba8042d8f73992d18f9282743312d9ce84c22d
Comment 103 Michael Fuchs 2016-03-22 16:42:10 UTC
Thanks for taking care of this.