Bug 338370

Summary: Kmail2 (4.14rc) with gmail too many simultaneous imap connections
Product: [Frameworks and Libraries] Akonadi Reporter: RJVB <rjvbertin>
Component: IMAP resourceAssignee: Christian Mollekopf <chrigi_1>
Status: RESOLVED UNMAINTAINED    
Severity: major CC: kdepim-bugs, montel, rjvbertin, vkrause
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Kubuntu   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description RJVB 2014-08-19 09:45:41 UTC
With the latest (4.13.97) packages in the Kubuntu 14.04 backport PPA, kmail2 has become (almost) unusable with my main gmail account. I've always had the occasional failure due to gmail's limit on the number of imap connections (though I *think* it started with 4.13) but now this has become a serious blocker. There is some voodoo involving various combinations of offlining the account and restarting the akonadi server and/or kmail, but it never takes long before I get bitten again.

I do not only have my Linux workstation on which I handle email, but also an Android tablet, an iPhone and a Mac workstation, so the gmail account has a number of filters that sorts incoming mail in folders on the server. Because of that and gmail's 15 connection limit, it is extremely important that each client behaves and doesn't just multiply the number of concurrent connections (for scanning all those folders in parallel, for instance) without ever closing them immediately when done.

Thunderbird has the same issue with its default of 5 simultaneous connections max. per server, but it allows to dial that value down. With 1, I can avoid the issue.

I have thus gone back to using Thunderbird and removed all my kmail/akonadi imap accounts (since they cannot be deactivated permanently otherwise) in hope and waiting for better days.

NB: the issue doesn't appear to occur (or MUCH less frequently) when just letting the akonadi resources run without kmail running. To me this suggests that it's something in the way kmail talks to the akonadi resources that triggers the issue.

Is there a way to limit the number of simultaneous imap connections, either in kmail or in akonadi?
Comment 1 RJVB 2014-08-20 14:17:22 UTC
I downgraded to KDE 4.13.3, and removed,recreated the gmail account. The issue has become much less frequent. Here's some output from netstat:

With the "too many simultaneous connections, enterpassword/try again/cancel" alert still open:

> netstat -aeepW | fgrep imap
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 patux:57979             wi-in-f109.1e100.net:imaps ESTABLISHED bertin     5617520     18352/akonadi_imap_
tcp        0      0 patux:57526             wi-in-f109.1e100.net:imaps ESTABLISHED bertin     5394945     18352/akonadi_imap_
unix  3      [ ]         STREAM     CONNECTED     5392495  18352/akonadi_imap_ 
unix  3      [ ]         STREAM     CONNECTED     5392357  18352/akonadi_imap_ 
unix  3      [ ]         STREAM     CONNECTED     5392742  18352/akonadi_imap_ 
unix  3      [ ]         STREAM     CONNECTED     5392270  18352/akonadi_imap_ 
unix  3      [ ]         STREAM     CONNECTED     5392192  18352/akonadi_imap_ 

After hitting cancel, causing kmail to take the account offline:

netstat -aeepW | fgrep imap
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 patux:57526             wi-in-f109.1e100.net:imaps TIME_WAIT   root       0           -               
unix  3      [ ]         STREAM     CONNECTED     5392495  18352/akonadi_imap_ 
unix  3      [ ]         STREAM     CONNECTED     5392357  18352/akonadi_imap_ 
unix  3      [ ]         STREAM     CONNECTED     5392742  18352/akonadi_imap_ 
unix  3      [ ]         STREAM     CONNECTED     5392270  18352/akonadi_imap_ 
unix  3      [ ]         STREAM     CONNECTED     5392192  18352/akonadi_imap_ 

In other words, taking the account offline does not cause it to close all its connections to the server, something I'd expect.

Sending a SIGHUP to 18352 causes it to exit, and immediately thereafter I can online the account in kmail with success.That would appear to offer at least a suggestion for a workaround solution. The restart command in the akonadiconsole does NOT have that effect (in fact, it rarely appears to do anything with imap accounts ... unless they're working properly?!) but it'd be much more convenient NOT to have to launch a separate application to correct an issue that's apparent in kmail.
Comment 2 RJVB 2014-08-20 15:27:22 UTC
Something else: given how gmail and kmail/akonadi all support IMAP IDLE, is it still necessary to poll explicitly at regular intervals - and would NOT doing so improve things?

Googling for akonadi imap IdleRidPath gave multiple hits for multiple IDLE paths (which I guess I'd need) but also mentions a GUI for configuring those ... and I don't seem to have that. Has that feature ever been implemented and committed?
Comment 3 RJVB 2014-08-22 07:39:46 UTC
Some feedback:
I am currently testing a configuration without explicit, timed check for new mail in my gmail inbox, relying instead on the IDLE feature for being notified about incoming mail in that mailbox (since the IDLE feature cannot be deactivated anyway).

Since configuring things that way, I have had no more errors about too many simultaneous connections on that account. I do get kwallet unlock request for reading the credentials to the account from time to time (the wallet is set to close after 2min). (Personally I'd store an encrypted copy of the password in RAM to prevent that, but that's a different issue.)

Note that this is after downgrading to KDE 4.13.3, which reduced but did not eliminate the simult. connections lock-out issue.

Curiously, I also no longer get a dialog for entering the actual gmail password anew. I used to get that after the kwallet dialog had gone undetected for too long (?), as if there's a timeout on the request. I haven't had time to confirm, but it seems that this (always?) preceded the lock-out issue.
I think I still observed that behaviour in KDE 4.13.3, so after downgrading but before removing the timed poll from the gmail inbox.
Comment 4 Daniel Vrátil 2014-08-25 16:09:25 UTC
Reassigning to IMAP resource which this bug report is against (the native GMail resource was not shipped in stable yet).

Note that IDLE only works on the INBOX folder: it won't update your other folders, if GMail automatically files an email there (due to filters for instance).

For performance reasons the IMAP resource keeps the connections open as long as possible, so it should not matter whether you enable/disable interval check.
Comment 5 RJVB 2014-08-25 16:25:34 UTC
> Note that IDLE only works on the INBOX folder: it won't update your other
> folders, if GMail automatically files an email there (due to filters for
> instance).

I saw work somewhere online that would add IDLE support for other folders, but it's possible that'd be at the cost of 1 connection per folder ...

> For performance reasons the IMAP resource keeps the connections open as long
> as possible, so it should not matter whether you enable/disable interval
> check.

I have a hunch that performance is more hampered by the fact that messages are downloaded systematically even when right-clicked to be deleted or when part of a multiple selection (I am not downloading offline copies) than by reconnecting to the server. Provided each reconnect isn't going to present a wallet unlock request, of course.

Would it be a thought to close the connections if the server replies with the (unambiguous) error message?
Comment 6 Denis Kurz 2016-09-24 20:41:21 UTC
This bug has only been reported for versions older than KDEPIM 4.14 (at most akonadi-1.3). Can anyone tell if this bug still present?

If noone confirms this bug for a recent version of akonadi (part of KDE Applications 15.08 or later), it gets closed in about three months.
Comment 7 Denis Kurz 2017-01-07 21:35:36 UTC
Just as announced in my last comment, I close this bug. If you encounter it again in a recent version (at least 5.0 aka 15.08), please open a new one unless it already exists. Thank you for all your input.