Bug 335795 - IMAP namespace support is incomplete/insufficient on large imap accounts
Summary: IMAP namespace support is incomplete/insufficient on large imap accounts
Status: CLOSED INTENTIONAL
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: IMAP resource (show other bugs)
Version: 4.13
Platform: unspecified All
: NOR grave
Target Milestone: ---
Assignee: Christian Mollekopf
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-06-04 16:14 UTC by RJVB
Modified: 2016-05-27 10:24 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description RJVB 2014-06-04 16:14:04 UTC
I run an imap server on my main workstation, allowing me to archive email no matter from what computer I'm doing that. Those archives live under ~/Mail/Archives, and an imap client not ignoring everything above that path will spend a very long time indexing ~ (I had to kill and delete the resource when it hogged the whole computer after I let it run for about 2h).

I've skimmed through a few discussions on kmail's approach to the "imap prefix path" most email readers support, and while I tend to agree that if namespaces are the standards compliant way to do things they should be used, I also think the implementation isn't without flaws.

Let's ignore the fact that the current imap configuration dialog doesn't even mention namespaces (presumably replaced by server-side subscriptions?). The real gripe I have is that I apparently have to let the imap process index the full server-side contents before I can point it to the appropriate root folder (Mail/Archives). And it's not just an omission in the GUI: I tried simulating the required actions with a small imap account, and observed that apparently all mailboxes (folders) under the root are listed explicitly in the configuration file, rather than only the root folder. Which in my book is not using a namespace as any addition (new folder) will have to be added manually to the selection (requiring another lengthy indexing step).

I'd suggest reinstating a simple "prefix path" feature that can be specified (= typed in) even before the first connection to a new imap account. It's only a single text entry field, and internally it could perfectly well be translated into the appropriate namespace magic.

Reproducible: Always

Steps to Reproduce:
1. set up an imap server on a workstation
2. Use Thunderbird, Apple's Mail.app, sylpheed, pine, just about any MUA to set up an email archive on that imap server, connecting with your work/home account on that, for instance under Mail/Archives (= ~/Mail/Archives)
3. Define an akonadi imap resource pointing to that same IMAP server, using the same login credentials
4. Sit back and wait ...
Actual Results:  
A synchronisation step will be triggered that can take hours to complete if the user home directory (the "~" namespace) contains a large amount of files and folders, of which only 1 is relevant for the imap resource (Mail/Archives, and the mbox files contained therein).

Expected Results:  
Given additional configuration options, either to specify a path prefix or to alter the namespace queried, or both (as Thunderbird does). 

I don't have direct experience using imap namespaces, but if they don't allow to specify the path to where the mailboxes are (to be) stored, an additional mechanism should be implemented. Again, just about any other email client allows to do that. Following standards nice, but it should not become a case of form over function.

I'm marking this bug as "grave", because it requires serious workarounds if one is obliged to work with an imap server as outlined above.
Comment 1 Christian Mollekopf 2014-07-23 19:46:29 UTC
subscriptions allow you to not download what you don't want. I disagree that specifing a "prefix path" adds anything useful. Therefore I have to close this, sorry.

Thanks for the elaborate suggestion though (no irony intended).
Comment 2 RJVB 2014-07-23 20:01:16 UTC
Have you actually tried to use subscriptions in a use case like the one I described? Your claim about subscriptions is true only in theory, in practice one has to wait longer for a possibility to subscribe to only the desired mailboxes than it takes to  decide to install another email client, install, configure and start using it. Much longer.

So yeah, if speed and ease of use (= not having to subscribe to ALL folders inside/under the mailbox repository folder) aren't useful then I guess the majority of other readers that do provide a "prefix path" setting are just plain wrong to do so. No irony intended either (sarcasm and cynicism are a different matter) ...

BTW, nothing has been resolved here, so I'm setting a more appropriate status.
Comment 3 Christian Mollekopf 2014-07-23 20:14:19 UTC
(In reply to RJVB from comment #2)
> Have you actually tried to use subscriptions in a use case like the one I
> described? Your claim about subscriptions is true only in theory, in
> practice one has to wait longer for a possibility to subscribe to only the
> desired mailboxes than it takes to  decide to install another email client,
> install, configure and start using it. Much longer.
> 

Erm, what? So is the problem that the syncing starts before you have the chance to adjust subscriptions? If that is so, you have an entierly valid point, and we can fix it whenever I get around to it. Please open an according bugreport in that case.

We could also add the subscription dialog as part of the setup-dialog, which would probably improve the visibility of the feature (I know the subscription support in the UI is currently pretty bad, and I'm already working on that).

> So yeah, if speed and ease of use (= not having to subscribe to ALL folders
> inside/under the mailbox repository folder) aren't useful then I guess the
> majority of other readers that do provide a "prefix path" setting are just
> plain wrong to do so. No irony intended either (sarcasm and cynicism are a
> different matter) ...
> 

I pretty sure subscriptions do in fact solve the problem. It's entierly possible that the implementation isn't perfect and that it requires some adjustments, but that's the solution then, and not adding yet another workaround.

> BTW, nothing has been resolved here, so I'm setting a more appropriate
> status.

Please don't. I'm the one that has to deal with the tickets, and it helps noone keeping a tickets open that has been declined already. There is only "Resolved" and if you set it back to "unconfirmed" there is no way for me to distinguish it from other tickets that I haven't processed yet.
Even if it's set to "resolved" we can keep the discussion going here if you like.
Comment 4 RJVB 2014-07-23 20:47:40 UTC
Yes, there are 2 things which make (made) the current implementation unworkable for me. Most importantly, the fact that there is no way to obtain the list of folders (not) to subscribe to before the whole account is scanned. (And again, on a typical workstation also running an imapd that amounts to transferring all files under your account, possibly even trying to parse them.) The other thing is that there is nothing recursive to a subscription, i.e. there is no way I found to select all mailboxes in the directory where one keeps them.
One could argue that that's a once-only action, but in reality that would go against the fact that IMAP has always been designed to allow access from multiple locations (i.e., mailboxes might be created or deleted while connecting from a different location).

Isn't there a "closed - wontfix" status?
Comment 5 Christian Mollekopf 2014-07-23 21:55:05 UTC
(In reply to RJVB from comment #4)
> Yes, there are 2 things which make (made) the current implementation
> unworkable for me. Most importantly, the fact that there is no way to obtain
> the list of folders (not) to subscribe to before the whole account is
> scanned. (And again, on a typical workstation also running an imapd that
> amounts to transferring all files under your account, possibly even trying
> to parse them.)

Agreed, it should be possible to make that selection before the sync starts.

https://bugs.kde.org/show_bug.cgi?id=337744

> The other thing is that there is nothing recursive to a
> subscription, i.e. there is no way I found to select all mailboxes in the
> directory where one keeps them.

It would be nice if the UI allowed recursive actions indeed.

I'm aware the UI side isn't pretty in general, and I plan to improve that.

> One could argue that that's a once-only action, but in reality that would go
> against the fact that IMAP has always been designed to allow access from
> multiple locations (i.e., mailboxes might be created or deleted while
> connecting from a different location).
> 

I don't get that part. Server-side subscriptions are for all devices which you may or may not desire, but we allow to override that setting on a per-folder basis using local subscriptions, which should cover all use-cases, no?

> Isn't there a "closed - wontfix" status?

Indeed there is, once the status was resolved (it's a workflow thingy I suppose...).
Comment 6 Cristiano Guadagnino 2016-05-27 10:20:27 UTC
Hi,
I don't know if it is correct to add discussion to such an old thread, but I have found nothing else regarding the problem I am having.

Laos I think my input may change your mind regarding the usefulness of having an "imap path prefix" configuration option.

The point is that if you want to access any recent Exchange server from Linux you are very limited in options: either your administrators allow IMAP access to your mailbox (which most Exchange administrators won't allow) or you use DavMail.
DavMail is a very good option and can be used with just about any MUA.

BUT the situation changes greatly if you want to access a shared mailbox (which is a very common feature in enterprise accounts). This is a very common need, based on the number of requests one can find on the DavMail support forums.

Since both the server and the login/password is the same as your primary mailbox, there's no other way to access a shared mailbox than changing the IMAP path prefix.

This can easily be done with e.g. Thunderbird, but if one prefers KMail there's absolutely NO WAY to get this done.

I am using KMail 5.1.3.

Thank you for reading.
Comment 7 Cristiano Guadagnino 2016-05-27 10:24:50 UTC
"Laos I think my input..." should have been "Also I think my input..."