Bug 319175

After going through:

1) Create one maildir resource for each maildir main directory of each POP3 account, importing old filter rules, having the migrator migrate the POP3 accounts and other stuff from old configuration. => Mail losses (was on BTRFS as well)

2) Have the migrator do everything with ~/Mail in place. Strange folder ordering in KMail, scary popups (I didn´t note these, went away to quickly)

 3) This morning: Import old mails via KMailCVT. KMailCVT completed within a hour. But then Akonadi got stuck what seemed to be a write loop into changes.dat file. I restarted it after it has written more than 450 GB(!) which is about 1,5 times the amount of the capacity of the SSD. Mail unread/read status not imported correctly.

(bugs reported here)

I tried:


 4) Now: Import via pointing a mixed maildir resource at ~/Mail, moving over folders
[15:06:23] <helios21> to automatically pre-created pure Local Folders maildir resource


After moving a small folder with two subfolders from the mixedmaildir pointing to ~/Mail to the local folders pure maildir successfully I tried a bigger one with about 168000 mails. It took at least half a hour, maybe a hour till Akonadi actually created the target maildir folder. Then more and more mail file appeared there. More than in the source. But most were empty one-byte sized files just containing a linefeed.

Reproducible: Didn't try

Steps to Reproduce:
1. Setup KMail 2 from scratch
2. Have a mixedmaildir resource point to an old (maildir only) KMail-1 ~/Mail directory.
3. Let it "Synchronize All"
4. Move over a bigger folder with some subfolders to the pre-created pure maildir local folders resource
Actual Results:  
Source:
martin@merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory> find -type f | wc -l
168299
martin@merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory> find -type d | wc -l
154

martin@merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory/kernel-ml> find cur -type f | wc -l
42071
martin@merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory/kernel-ml> find cur -type f ! -size 1 | wc -l
42071
martin@merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory/kernel-ml> find cur -type f -size 1 | wc -l 
0




(including cur/new and so one folders)


Target:

martin@merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory> find -type f | wc -l
304006
martin@merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory> find -type d | wc -l
150

martin@merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory#1> find -type f ! -size 1 | grep -v xfs-ml
martin@merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory#1> find -size 1 | wc -l          
292456

martin@merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory> find -type f ! -size 1 -ls | head
1475802    4 -rw-r--r--   1 martin   martin       3114 Mai  1 15:04 ./xfs-ml/new/1367413494.R571.merkaba
1482572    4 -rw-r--r--   1 martin   martin       3700 Mai  1 15:06 ./xfs-ml/new/1367413573.R557.merkaba
1480531    8 -rw-r--r--   1 martin   martin       6771 Mai  1 15:05 ./xfs-ml/new/1367413552.R539.merkaba
1479024   12 -rw-r--r--   1 martin   martin       8950 Mai  1 15:05 ./xfs-ml/new/1367413535.R248.merkaba
1477745   12 -rw-r--r--   1 martin   martin      11310 Mai  1 15:05 ./xfs-ml/new/1367413518.R446.merkaba
1483894   16 -rw-r--r--   1 martin   martin      13906 Mai  1 15:06 ./xfs-ml/new/1367413594.R276.merkaba
1479049    8 -rw-r--r--   1 martin   martin       7755 Mai  1 15:05 ./xfs-ml/new/1367413535.R726.merkaba
1484993    8 -rw-r--r--   1 martin   martin       4452 Mai  1 15:06 ./xfs-ml/new/1367413613.R453.merkaba
1485314    8 -rw-r--r--   1 martin   martin       6755 Mai  1 15:06 ./xfs-ml/new/1367413619.R134.merkaba
1486609    8 -rw-r--r--   1 martin   martin       6410 Mai  1 15:07 ./xfs-ml/new/1367413651.R689.merkaba


martin@merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory/kernel-ml/new> hd 1367412411.R104.merkaba
00000000  0a                                                |.|
00000001

martin@merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory/kernel-ml/new> find -type f| wc -l 
84142
martin@merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory/kernel-ml/new> find -size 1 | wc -l
84142


Expected Results:  
An *exact* copy of the mail files from the source (albeit with different filenames)

Debian Sid with:

martin@merkaba:~> apt-show-versions | egrep "(kmail|akonadi|mysql)"
akonadi-backend-mysql/experimental uptodate 1.9.1-2
akonadi-backend-postgresql/experimental uptodate 1.9.1-2
akonadi-backend-sqlite/experimental uptodate 1.9.1-2
akonadi-server/experimental uptodate 1.9.1-2
akonadiconsole/experimental uptodate 4:4.10.2-2
kmail/experimental uptodate 4:4.10.2-2
libakonadi-calendar4/experimental uptodate 4:4.10.2-2
libakonadi-contact4/experimental uptodate 4:4.10.2-2
libakonadi-kabc4/experimental uptodate 4:4.10.2-2
libakonadi-kcal4/experimental uptodate 4:4.10.2-2
libakonadi-kde4/experimental uptodate 4:4.10.2-2
libakonadi-kmime4/experimental uptodate 4:4.10.2-2
libakonadi-notes4/experimental uptodate 4:4.10.2-2
libakonadiprotocolinternals1/experimental uptodate 1.9.1-2
libdbd-mysql-perl/sid uptodate 4.021-1+b1
libmysqlclient-dev/sid uptodate 5.5.30+dfsg-1.1
libmysqlclient18/sid uptodate 5.5.30+dfsg-1.1
libqt4-sql-mysql/sid uptodate 4:4.8.2+dfsg-11
mysql-client/sid uptodate 5.5.30+dfsg-1.1
mysql-client-5.5/sid uptodate 5.5.30+dfsg-1.1
mysql-common/sid uptodate 5.5.30+dfsg-1.1
mysql-server-core-5.5/sid uptodate 5.5.30+dfsg-1.1
python-mysqldb/sid uptodate 1.2.3-2


All Akonadi data located on Ext4.