Bug 275418 - IMAP (sub-)subfolders empty in kmail2
Summary: IMAP (sub-)subfolders empty in kmail2
Status: RESOLVED UNMAINTAINED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: Migration (show other bugs)
Version: 4.7
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-11 19:00 UTC by Richard Homonnai
Modified: 2017-01-07 22:27 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Homonnai 2011-06-11 19:00:13 UTC
Version:           1.99.0 (using KDE 4.6.4) 
OS:                Linux

I am using the new kmail2, version 2.1.0. Everything seems to work fine - but all my subfolders (and their subfolders) seem empty (which they aren't). Trash seems to be an exception, I see messages there.

I tried to regenerate the index, but I cannot find that option in kmail2 (in 1 it helped with such problems).

So how can I do this?

Reproducible: Always

Steps to Reproduce:
Start KMail2
Look at a subfolder

Actual Results:  
The folder is empty.

Expected Results:  
There should be mails visible.

OS: Linux (i686) release 2.6.38-gentoo-r6-richMini
Compiler: i686-pc-linux-gnu-gcc
Comment 1 Arne K. Haaje 2011-06-27 09:08:00 UTC
I can confirm this with Kmail 2.1.0 on KDE 4.6.4. IMAP server in use is Dovecot 1.2.12

All subfolders appear empty. The containing folder appear to have a correct message count.

Possibly important, the subfolders are not under the inbox (which is named INBOX). The containign folders are all on the same level as INBOX, like this;

Afolder
Bfolder
Container1
         - some messages
         Subfolder1
         Subfolder2
Dfolder
Efolder
INBOX
Ffolder
ListcontainerFolder
         List1
         List2
OneMoreFolder

and so on. All folders on the first level are correct. The subfolders appear empty
Comment 2 Modestas Vainius 2011-06-27 09:52:23 UTC
In short the cause of this bug is that migrator assumes (or misdetects) that a IMAP hierarchy separator. It assumes it to be '/' (GMail default) while dovecot uses '.' by default. If you're using akonadi mysql backend (if not, you can still adapt the instructions), read the IRC conversation below to fix this for your setup. Please note that these instructions deal with consequences, not the cause.

[Wednesday 15 June 2011] [01:07:42] <MoDaX>	do have imap folders like INBOX/something ?
[Wednesday 15 June 2011] [01:07:55] <MoDaX>	or rather INBOX.something
[Wednesday 15 June 2011] [01:08:51] <otherperson>	On my private accout all folders are subfolders of INBOX
[Wednesday 15 June 2011] [01:09:19] <MoDaX>	and they open fine?
[Wednesday 15 June 2011] [01:09:24] <otherperson>	yes
[Wednesday 15 June 2011] [01:09:35] <otherperson>	MoDaX: wait
[Wednesday 15 June 2011] [01:09:38] <otherperson>	no they don't
[Wednesday 15 June 2011] [01:09:41] <MoDaX>	heh
[Wednesday 15 June 2011] [01:09:44] <otherperson>	:S
[Wednesday 15 June 2011] [01:10:39] <otherperson>	Did I lose all my mails in INBOX/sent ? 
[Wednesday 15 June 2011] [01:11:10] <MoDaX>	otherperson: not yet
[Wednesday 15 June 2011] [01:11:19] <otherperson>	promising words :D
[Wednesday 15 June 2011] [01:11:19] <MoDaX>	kmail2 does not show them
[Wednesday 15 June 2011] [01:11:57] <otherperson>	MoDaX: strange is, that I sent a test mail, and it (and only it) is visibile in IBOX/sent
[Wednesday 15 June 2011] [01:12:43] <MoDaX>	otherperson: akonadi cached it, it's not on the imap server though
[Wednesday 15 June 2011] [01:12:54] <otherperson>	ok
[Wednesday 15 June 2011] [01:13:01] <MoDaX>	you better delete it
[Wednesday 15 June 2011] [01:13:08] <otherperson>	so how do I make subfolders visibile
[Wednesday 15 June 2011] [01:13:12] <otherperson>	ok
[Wednesday 15 June 2011] [01:13:27] <MoDaX>	otherperson: scroll back to my backlog from yesterday
[Wednesday 15 June 2011] [01:13:53] <MoDaX>	I hope you use mysql as akonadi backend, don't you?
[Wednesday 15 June 2011] [01:14:39] <otherperson>	MoDaX: yes
[Wednesday 15 June 2011] [01:16:10] <MoDaX>	otherperson: this is going to be a bit tricky. connect to mysql server
[Wednesday 15 June 2011] [01:16:12] <MoDaX>	mysql --socket ~/.local/share/akonadi/socket-`hostname`/mysql.socket
[Wednesday 15 June 2011] [01:16:47] <MoDaX>	there do 'use akonadi'
[Wednesday 15 June 2011] [01:17:14] <otherperson>	database changed
[Wednesday 15 June 2011] [01:18:02] <MoDaX>	otherperson: select id, remoteid, name, parentid from collectiontable;
[Wednesday 15 June 2011] [01:18:16] <MoDaX>	remember IDs of your imap folders
[Wednesday 15 June 2011] [01:19:17] <MoDaX>	stop kmail2
[Wednesday 15 June 2011] [01:19:23] <otherperson>	where to find the ID's
[Wednesday 15 June 2011] [01:19:29] <otherperson>	IDs
[Wednesday 15 June 2011] [01:19:43] <MoDaX>	otherperson: select gave them to you
[Wednesday 15 June 2011] [01:19:51] <MoDaX>	first column
[Wednesday 15 June 2011] [01:20:10] <otherperson>	MoDaX: ahh, overread that line
[Wednesday 15 June 2011] [01:20:12] <otherperson>	ok
[Wednesday 15 June 2011] [01:20:30] <otherperson>	kmail stoped
[Wednesday 15 June 2011] [01:21:25] <MoDaX>	your mailbox subfolders should be sequential and fit into min <= id AND id <= max . find those min and max from the output
[Wednesday 15 June 2011] [01:22:33] <otherperson>	so there is a line: 7->imap:... and  all in between until 27->/Trash
[Wednesday 15 June 2011] [01:22:51] <otherperson>	is then 8-27 ?
[Wednesday 15 June 2011] [01:23:00] <MoDaX>	yeah
[Wednesday 15 June 2011] [01:23:13] <MoDaX>	what you need to do here is to change the first symbol of remoteid column from '/' to '.'
[Wednesday 15 June 2011] [01:23:14] <otherperson>	ok
[Wednesday 15 June 2011] [01:23:37] <MoDaX>	update collectiontable set remoteid = concat('.', substr(remoteid from 2)) where id >= 80 AND id <= 85
[Wednesday 15 June 2011] [01:23:43] <MoDaX>	replace min and max
[Wednesday 15 June 2011] [01:25:47] <MoDaX>	then in another console restart akonadi (akonadictl restart). mysql server will restart, reconnect to it and recheck with select if remoteid where actually changed
[Wednesday 15 June 2011] [01:27:37] <otherperson>	ok, went fine I guess
[Wednesday 15 June 2011] [01:27:42] <otherperson>	start kmail2 ?
[Wednesday 15 June 2011] [01:28:08] <otherperson>	really annoying bug
[Wednesday 15 June 2011] [01:28:24] <otherperson>	imagine explaining this to old users at a university
[Wednesday 15 June 2011] [01:28:29] <MoDaX>	yeah, start it
[Wednesday 15 June 2011] [01:28:46] <MoDaX>	folder icons should be back too now
[Wednesday 15 June 2011] [01:28:53] <otherperson>	MoDaX: Yippie
[Wednesday 15 June 2011] [01:29:00] <otherperson>	thanks a lot
[Wednesday 15 June 2011] [01:29:55] <MoDaX>	but kmail2 might still eat your mail. better disable automatic expurging in account settings. might help
[Wednesday 15 June 2011] [01:30:20] <MoDaX>	I think I have already "saved" 5 mails with this today
[Wednesday 15 June 2011] [01:30:33] <MoDaX>	they are both 'deleted' and 'unread' now :)
Comment 3 Richard Homonnai 2011-06-27 09:59:32 UTC
I am using Dovecot on one of my servers, the others have Courier-IMAP I think.
Is this one also affected by this bug? Because on both of them I don't see any subfolder contents.

Can I edit the db with normal SQLite tools if using SQLite? Or is there something to keep in mind?
Comment 4 Arne K. Haaje 2011-06-27 10:01:43 UTC
I deleted the account in Kmail2 (which as imported by the migrator), then recreated it manually. That fixed it for me.
Comment 5 Modestas Vainius 2011-06-27 10:07:11 UTC
AFAIK '.' is more common than '/' among IMAP servers.
Comment 6 Modestas Vainius 2011-06-27 10:08:37 UTC
Richard, of course you can edit using SQLite tools. You might just need to adapt SQL syntax a bit.
Comment 7 Richard Homonnai 2011-06-28 09:13:31 UTC
Well, this fixed it for me. If anybody also uses SQlite, here's the query:

update collectiontable set
remoteid = '.' || substr(remoteid,2) where id >= 80 AND id <= 85

of course, replace 80 and 85 accordingly. I for myself just updated all rows and then fixed a few (3, I think) calendar entries by hand. Was much less effort than the 114 broken subdirs on four mail servers ;)

The most safe approach might be:

update collectiontable set
remoteid = '.' || substr(remoteid,2) where remoteid NOT LIKE '/home/username/%'

of course, assuming all your local calendars are on your home.
Comment 8 Markus Rennings 2011-07-08 20:40:02 UTC
Same here (Arch Linux, KMail 2.1.1), the IRC snippet helped
Comment 9 FiNeX 2011-07-27 20:57:24 UTC
I confirm the bug. I've migrated an IMAP account (dovecot) with about 50000 email and hundred of subfolders and all looks empty :-(

Unfortunatly I don't have time to play with scripts and configuration files. So I will downgrade to KMail 1 until KMail 2 is an alpha-quality version :-(

bye :-)
Comment 10 FiNeX 2011-07-27 21:22:57 UTC
Humm... I've deleted the account from Kmail and I've recreated a new one and now it shows all my emails... :-/
Comment 11 Ferdinand Gassauer 2011-08-03 08:52:50 UTC
same problem with
mailstore: cyrus imap

deleting and recreating fixes the ssue
Comment 12 Christophe Marin 2011-09-13 15:27:53 UTC
that *only* happens when migrating accounts, right ?
Comment 13 Arne K. Haaje 2011-09-13 18:03:28 UTC
Yes, it is the migration that fails. I've tested on two different machines, and if account is created from scratch, then the subfolders are populated. If the accounts are migrated, then the subfolders are empty
Comment 14 Christophe Marin 2011-09-13 21:14:34 UTC
Thanks
Comment 15 Modestas Vainius 2012-06-23 10:17:56 UTC
I did not experience this when I migrated my kmail 1 accounts with kdepim(libs) 4.8.4 today. It has been fixed probably.
Comment 16 Denis Kurz 2016-09-24 20:39:08 UTC
This bug has only been reported for versions older than KDEPIM 4.14 (at most akonadi-1.3). Can anyone tell if this bug still present?

If noone confirms this bug for a recent version of akonadi (part of KDE Applications 15.08 or later), it gets closed in about three months.
Comment 17 Denis Kurz 2017-01-07 22:27:18 UTC
Just as announced in my last comment, I close this bug. If you encounter it again in a recent version (at least 5.0 aka 15.08), please open a new one unless it already exists. Thank you for all your input.