Bug 114985

Confirmed. Serverity raised.

Created attachment 13593 [details]
fix moving of folders containing subfolders from local to dIMAP accounts

The patch fixes the moving of folders containing subfolders from local to dIMAP
accounts. No messages are lost any longer.

It doesn't solve moving from dIMAP to local accounts.

Johannes, can you tell me how you managed to move nested folders from cachedimap to local accounts or within the cachedimap account area?

Nested folders shouldn't have the Move Folder To popup menu item. How can you have lost messages except when moving from the local account to a cachedimap account?
(There is a bug which shows this menu item for nested folders which have just been moved to a cachedimap account. However you stated that you could only move one folder and the child folders where lost. So how could you have a child folder then? In fact, you need some additional code like in my patch to see the mentioned bug.)

Created attachment 13613 [details]
fix for moving around nested folders from a local account

Okay, here is a patch which allows moving of nested folders from a local
account to a cachedimap account. The opposite direction would work as well but
is allowed only for leaf folders.
I made sure that fresh copies of nested folders have the correct state. So
AFAICT it's not possible to move nested folders from within a cachedimap
account. In case I've overseen a constellation, the nested folders are moved to
a local account w/o data loss though.

> Johannes, can you tell me how you managed to move nested folders from
> cachedimap to local accounts or within the cachedimap account area? 

I used context menue for the toplevel folder (Test1).


Today I played again with the moving:

I used context menues to create locally a.m. folders. Then I copied some mails into them and again used the context menue item for the toplevel (Test1) folder "Move Folder To" to move the folder to the DIMAP account (placing toplevel).

2nd test was the same but moving to the IMAP account (pointing to the same mailbox as the DIMAP one).

3rd test was to create the folders using a IMAP account. Creation using the DIMAP failed: the DIMAP showed Test2 as subfolder of Test1. But IMAP and subscriptions tell of them to be in parallel.
BTW, the IMAP and the DIMAP seem to share the subscriptions! When having created those folders after some time the DIMAP account showed those folders automatically. When unsubscribing using the DIMAP account, the folders first (!) disappeared from the IMAP account and later (!) from the DIMAP one. I do not feel this to be intended behaviour ... ?
Due to this there was no need to subscribe separately the DIMAP account for the new folders. Within the DIMAP account I then could use context menue item to move the folders (within the IMAP account the menue item does not exist) to the local folders (toplevel).
I just saw a new detail about this last move: sometime after I moved, the IMAP account complained about the missing Test1 folder. Having a look to the subscriptions, Test1 is not subscribed (not in IMAP nor DIMAP) but the Test2 subfolder is (in both). The IMAP account shows both, Test1 (now empty) and below Test2 (still containing the mails), the DIMAP shows none of them. For the DIMAP I then re-subscribed both (unsubscribe Test1, subscribe Test1 and Test2; every step followed by cache refresh from the DIMAP account troubleshooting context menue). Afterwards DIMAP showed the same as the IMAP.
And: I was able to move the remaining subfolder seperately to the local folders.

Summary:
- when moving from local to DIMAP, subfolders and mails contained are lost
- when moving from local to IMAP, subfolders and mails contained are lost
- when moving from DIMAP to local or to another folder within the DIMAP, no folders or mails are lost but
    - display is confused
    - unexpectedly subfolders and their mails are not moved, only mails of the toplevel folder are moved
- connecting to an IMAP account both ways, IMAP and DIMAP, in parallel, subscriptions are shared. And the display of one gets confused on changes to the other.
- creation of subfolder succeeds only with the parent being synced to the server. When creating within a short time both, parent and subfolder, this leads to two folders in parallel after having synced with the server.
 - moving from IMAP to local or DIMAP is not possible, no menue entry for doing so

Am Montag, 28. November 2005 09:23 schrieb Johannes Stamminger:
> Summary:


Hi Johannes, thanks for your effort.

> - when moving from local to DIMAP, subfolders and mails contained are lost
> - when moving from local to IMAP, subfolders and mails contained are lost
> - when moving from DIMAP to local or to another folder within the DIMAP, no
> folders or mails are lost but 
>    - display is confused 
>    - unexpectedly subfolders and their mails are not moved, only mails of
>       the toplevel folder are moved


Well, actually you should only be able to move folders without child folders 
in an DIMAP account. But this may fail when subscription comes into play. I 
haven't tested with online IMAP so far.

> - connecting to an IMAP account both ways, IMAP and DIMAP, in parallel,
> subscriptions are shared. And the display of one gets confused on changes 
> to the other. 


This is the way subscription works. The subscription data is stored on the 
server per account. All clients share this data when accessing an account 
which is intended.

> - creation of subfolder succeeds only with the parent being synced to the
> server. When creating within a short time both, parent and subfolder, this
> leads to two folders in parallel after having synced with the server. 


I need to check this, but it will take a while as currently I have no access 
to a server.

> - moving from IMAP to local or DIMAP is not possible, no menue entry 
> for doing so 


That's intended AFAIK.

I decided to try online IMAP today. Never done that before.
Of course I lost ALL OF MY MAIL to my friends 3 years back, because of trying to move several folders at the same time (Kmail willingly allowed it no Hour-glass or anything.)

this is moving from local folders to online IMAP folders.

The effect:
The first folder was large and took about 15 minutes to transfer. I started transferring 5 other folders while waiting.
And when activity ended I had the folders I moved on the IMAP server but they were all empty but the first big one.

If this is not relevant to this exact bug I'll be happy to report it in a new bug using all caps. 
:(

Forgot to say this is kde 3.5 from svn some days ago, with kmail 1.9

Jonas, your problem of loosing the content of the child folders is exactly the one which the bug report is about.
If you're running a self-compiled KDE, you can apply the patch attached to this report to minimize the chance for a message loss.

Thanks,
The 13613 patch worked great.

kdedev@jw ~/src/kde-svn-3.5/kdepim/kmail $ patch <movenestedfolderspatch.diff
patching file renamejob.cpp
patching file renamejob.h
patching file kmfolder.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file kmfolder.cpp.rej
kdedev@jw ~/src/kde-svn-3.5/kdepim/kmail $

I just followed the defaults that patch suggested regarding kmfolder.cpp. Never applied a patch before so I didn't completely understand -R when doing it, but it compiled fine against Revision: 486913 anyway.

Big thanks. Is this one of those patches that change too much to get committed to the branch? no? It's IMHO such a serious bug I think it should be committed. (haven't looked at the code though)

Thanks.

> Big thanks. Is this one of those patches that change too much to get
> committed to the branch? no? It's IMHO such a serious bug I think it should
> be committed. (haven't looked at the code though)


Each patch is a risk of it's own. This one has been tested only by me. I 
haven't got feedback from other people so far (except from you).

Anyway, the core developers prefer disabling the move function for folders to 
be on the safe side. But I haven't got around with that change because I've 
spent my time on report 113730 which is critical as well.

SVN commit 487310 by gungl:

Disable folders in IMAP and cached IMAP accounts as
destination for a folder move to avoid loosing all
messages in the subfolders when moving nested folders.

Moving to a local destination folder works correct.

This is only a workaround for the real problem which is
addressed by a patch. It was aggreed on not applying the
patch to the KDE 3.5 branch before it's been tested by
more people.
BUG:114985


 M  +12 -0     kmfoldertree.cpp  


--- branches/KDE/3.5/kdepim/kmail/kmfoldertree.cpp #487309:487310
@@ -1780,6 +1780,18 @@
       item = item->nextSibling();
       continue;
     }
+    if ( action == MoveFolder ) {
+      // FIXME remove in KDE 4
+      // skip all but local folders if a folder is to be moved
+      // because moving of nested folders to IMAP and DIMAP
+      // looses all messages in the subfolders
+      if ( fti->protocol() != KFolderTreeItem::Local
+        && fti->protocol() != KFolderTreeItem::NONE )
+      {
+        item = item->nextSibling();
+        continue;
+      }
+    }
     QString label = fti->text( 0 );
     label.replace( "&","&&" );
     if ( fti->firstChild() )

All mail lost after moving folder to cached IMAP folder. After syncronization with server it shows that all folders is syncronized but after system proper shutdown and start kmail lost all mail that is moved to IMAP folder. Worse yet it does not show any mail in pop folder too- with other mail client opera I can see pop mail account mails. Moved cached IMAP folder is empty on server side. 

KDE Version  1.9.3 (KDE 3.5.3, compiled sources)
Application  E-Mail Client
Operating System  FreeBSD (i386) release 6.1-RELEASE-p2
Compiler  gcc version 3.4.4 [FreeBSD] 20050518