Bug 115254 - Cannot create new folders with IMAP using Dovecot-imapd, regression from 3.4.X
Summary: Cannot create new folders with IMAP using Dovecot-imapd, regression from 3.4.X
Status: RESOLVED FIXED
Alias: None
Product: kmail
Classification: Applications
Component: IMAP (show other bugs)
Version: 1.9
Platform: Compiled Sources Linux
: NOR major
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-10-28 14:22 UTC by Alex Sidorenko
Modified: 2007-09-14 12:17 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Sidorenko 2005-10-28 14:22:20 UTC
Version:            (using KDE KDE 3.4.92)
Installed from:    Compiled From Sources
Compiler:          gcc version 4.0.2 20050808 Ubuntu 4.0.1-4ubuntu9
OS:                Linux

Everything works perfectly with Kmail from KDE-3.4.x, but with 3.5-beta2 (Ubuntu) and SVN (compiled from branches/KDE/3.5/kdepim on 27 Oct 2005) it always fails with a message:
=======================================================================
Error while creating a folder.
Could Not Create Folder
An attempt to create the requested folder failed.
Details of the request:
URL: (unknown)
Date and time: October 28, 2005 08:13 am
Additional information: imap://asid@cats:143.Test2//;INFO=ASKUSER
Possible causes:
Your access permissions may be inadequate to perform the requested operation on this resource.
The location where the folder was to be created may not exist.
A protocol error or incompatibility may have occurred.
Possible solutions:
Retry the request.
Check your access permissions on this resource.
=======================================================================

The problem can be reproduced just by installing dovecot-imapd, enabling imap imaps in dovecot.conf and trying to create a new folder

I can see it both with dovecot-imapd installed on Debian/testing and Ubuntu/breezy. This is a regression - Kmail from KDE-3.4.{1,3} creates new folders/subfolders without any problems
Comment 1 Alex Sidorenko 2005-10-31 14:04:58 UTC
Using ethereal I can see that while trying to create a mailbox/folder with the name 'Alex1' Kmail from 3.5 prepends a dot:

IMAP Request: 20 CREATE ".Alex1"

but Kmail from 3.4 doesn't:

IMAP Request: 9 CREATE "Alex1"

It seems that a prepended character is always dot and does not depend on hierarchy separator in namespace, I tried with two namespaces
IMAP Response: * NAMESPACE (("" ".")) NIL NIL
  and 
IMAP Response: * NAMESPACE (("" "/")) NIL NIL

I think that it is a function implementing 'New Subfolder' that prepends a dot for IMAP, not kio_imap4 slave. I'll continue to visually inspect the sources to see where it is done but it would be probably faster if one of KMail developers looks at that - I consider this bug a show-stopper for KDE-3.5 release
Comment 2 Casey Allen Shobe 2005-12-01 10:08:27 UTC
Confirm this problem in KDE 3.5 RC1 on Gentoo against dovecot-0.99.14 server.

Folder deletions also fail, and are a more hairy problem.

Agreed that this should be a showstopper for the 3.5 release, but that's not my decision to make, unfortunately.
Comment 3 Malte S. Stretz 2005-12-07 12:49:25 UTC
I already had the same problem with KDE 3.4 with dovecot 0.99.14.  Creating folders eg. with Squirrelmail works fine.
Comment 4 Carsten Burghardt 2005-12-07 14:39:31 UTC
Is there any dovecot test server that I can use to debug this?
Comment 5 Jan de Visser 2005-12-07 15:27:10 UTC
On Wednesday 07 December 2005 08:39, you wrote:
[bugs.kde.org quoted mail]

Carsten,

I can get you an account on my dovecot FC4 server. Let me know if you need it.

JdV!!


> _______________________________________________
> KMail developers mailing list
> KMail-devel@kde.org
> https://mail.kde.org/mailman/listinfo/kmail-devel

Comment 6 Alex Sidorenko 2005-12-07 20:33:24 UTC
Hello Carsten,

I cannot give you an account on my testing host (I work for HP and my hosts are behind the firewall). But I think I know what is wrong - I have found two problems while experimenting with kmail compiled with debugging enabled.

1. In the initial bug report I was wrong that the problem exists even with '/' hierarchy separator - I did not refresh account properties. Everything works fine with '/'. 

2. To reproduce the problem you need 'dovecot' to use '.' as separator. This is a default behaviour for Debian/Ubuntu version but you can control it using something like

namespace private {
  separator = .		# Here
  prefix =
  inbox = yes
}

In this case attempting to create a top-level folder N3 is rejected with a message 

imap://asid@cats:143.N3//;INFO=ASKUSER

Please note that this URL is incorrect - we add a dot (our separator) after port 143

3. Here are two problems I have found in imapaccountbase.cpp

a) 

QString ImapAccountBase::createImapPath( const QString& parent,
                                           const QString& folderName )
  {
    QString newName = parent;
    // strip / at the end
    if ( newName.endsWith("/") ) {
      newName = newName.left( newName.length() - 1 );
    }
    // add correct delimiter
    QString delim = delimiterForNamespace( newName );
    // should not happen...
    if ( delim.isEmpty() ) {
      delim = "/";
    }
/**/    if ( !newName.endsWith( delim ) && !folderName.startsWith( delim ) ) {
      newName = newName + delim;
    }
    newName = newName + folderName;
    // add / at the end
    if ( !newName.endsWith("/") ) {
      newName = newName + "/";
    }
    return newName;
  }

Let us assume that we want to create a toplevel folder N3 and use '.' delimiter (like I did in my test). In the beginning we set 

newName = parent;

and later at the line I marked with /**/ the test 
    if ( !newName.endsWith( delim ) && !folderName.startsWith( delim ) ) {

succeeds - as a result we have newName = '.'
Later we do newName = newName + folderName;
so we'll have newname = '.N3'

This is wrong ! Why does it work with '/' delimiter? I am not 100% sure but I think that IMAP slave will strip the leading / while parsing URL as a standard URL-separator

b) In QString ImapAccountBase::delimiterForNamespace( const QString& prefix )

we are sometimes unable to find the delimiter (I was able to see this in debugging output). As far as I understand the problem is due to the fact that two values can be used for an empty prefix - QString::null and "". I am not sure how this happens but in my experiments I was able to see both of them. This probably depends on how namespace info is obtained from IMAP-server, how it is stored in kmailrc. I can see in this file

Namespace:=/

which means no namespace (I have not defined it) and '/' delimiter. Depending on how this file is parsed we can get either QString::null or "". 

Anyway, I have found that the last lines

    // see if we have an empty namespace
    if ( mNamespaceToDelimiter.contains( QString::null ) ) {
      return mNamespaceToDelimiter[QString::null];
    }

do not always work. But when I added another test 

    if ( mNamespaceToDelimiter.contains("") ) {
      return mNamespaceToDelimiter[""];
    }

(using "" index instead of QString::null) it started working for me. I am not quite sure about this fix - probably it is better to find how empty namespace is parsed and set.

Finally, here is a patch that made things work for me:

{root 14:25:44} svn diff
Index: imapaccountbase.cpp
===================================================================
--- imapaccountbase.cpp (revision 486403)
+++ imapaccountbase.cpp (working copy)
@@ -785,6 +785,12 @@
     if ( mNamespaceToDelimiter.contains( QString::null ) ) {
       return mNamespaceToDelimiter[QString::null];
     }
+
+    // see if we have an empty namespace as ""
+    if ( mNamespaceToDelimiter.contains("") ) {
+      return mNamespaceToDelimiter[""];
+    }
+
     // well, we tried
     kdDebug(5006) << "delimiterForNamespace - not found" << endl;
     return QString::null;
@@ -1284,7 +1290,8 @@
     if ( delim.isEmpty() ) {
       delim = "/";
     }
-    if ( !newName.endsWith( delim ) && !folderName.startsWith( delim ) ) {
+    if (!newName.isEmpty() && \
+       !newName.endsWith( delim ) && !folderName.startsWith( delim ) ) {
       newName = newName + delim;
     }
     newName = newName + folderName;

With this fix everything works fine both with '/' and '.' delimiters. 

Regards,
Alex
Comment 7 Casey Allen Shobe 2005-12-07 21:09:40 UTC
Carsten,

I will be happy to set up an account on our production systems for you (we are a hosting provider).  I'm heading out at the moment, but will send you an E-mail with account details in a few hours when I get back.
Comment 8 Carsten Burghardt 2005-12-13 16:12:51 UTC
SVN commit 488193 by burghard:

Empty strings are read from the config as empty strings and not
QString::null
BUGS: 115254


 M  +16 -20    imapaccountbase.cpp  


--- branches/KDE/3.5/kdepim/kmail/imapaccountbase.cpp #488192:488193
@@ -202,10 +202,8 @@
     namespaceDelim entries = config.entryMap( config.group() );
     namespaceDelim namespaceToDelimiter;
     for ( namespaceDelim::ConstIterator it = entries.begin(); 
-          it != entries.end(); ++it )
-    {
-      if ( it.key().startsWith( "Namespace:" ) )
-      {
+          it != entries.end(); ++it ) {
+      if ( it.key().startsWith( "Namespace:" ) ) {
         QString key = it.key().right( it.key().length() - 10 );
         namespaceToDelimiter[key] = it.data();
       }
@@ -226,18 +224,15 @@
     config.writeEntry( "loadondemand", loadOnDemand() );
     config.writeEntry( "listOnlyOpenFolders", listOnlyOpenFolders() );
     QString data;
-    for ( nsMap::Iterator it = mNamespaces.begin(); it != mNamespaces.end(); ++it )
-    {
-      if ( !it.data().isEmpty() )
-      {
+    for ( nsMap::Iterator it = mNamespaces.begin(); it != mNamespaces.end(); ++it ) {
+      if ( !it.data().isEmpty() ) {
         data = "\"" + it.data().join("\",\"") + "\"";
         config.writeEntry( QString::number( it.key() ), data );
       }
     }
     QString key;
     for ( namespaceDelim::ConstIterator it = mNamespaceToDelimiter.begin(); 
-          it != mNamespaceToDelimiter.end(); ++it )
-    {
+          it != mNamespaceToDelimiter.end(); ++it ) {
       key = "Namespace:" + it.key();
       config.writeEntry( key, it.data() );
     }
@@ -688,14 +683,14 @@
         QString msg = i18n("KMail has detected a prefix entry in the "
             "configuration of the account \"%1\" which is obsolete with the "
             "support of IMAP namespaces.").arg( name() );
-        if ( list.contains( QString::null ) ) {
+        if ( list.contains( "" ) ) {
           // replace empty entry with the old prefix
-          list.remove( QString::null );
+          list.remove( "" );
           list += mOldPrefix;
           mNamespaces[PersonalNS] = list;
-          if ( mNamespaceToDelimiter.contains( QString::null ) ) {
-            QString delim = mNamespaceToDelimiter[QString::null];
-            mNamespaceToDelimiter.remove( QString::null );
+          if ( mNamespaceToDelimiter.contains( "" ) ) {
+            QString delim = mNamespaceToDelimiter[""];
+            mNamespaceToDelimiter.remove( "" );
             mNamespaceToDelimiter[mOldPrefix] = delim;
           }
           kdDebug(5006) << "migratePrefix - replaced empty with " << mOldPrefix << endl;
@@ -726,7 +721,7 @@
     {
       kdDebug(5006) << "migratePrefix - no migration needed" << endl;
     }
-    mOldPrefix = QString::null;
+    mOldPrefix = "";
   }
 
   //-----------------------------------------------------------------------------
@@ -771,8 +766,7 @@
     // then try if the prefix is part of a namespace
     // exclude empty namespace
     for ( namespaceDelim::ConstIterator it = mNamespaceToDelimiter.begin(); 
-          it != mNamespaceToDelimiter.end(); ++it )
-    {
+          it != mNamespaceToDelimiter.end(); ++it ) {
       // the namespace definition sometimes contains the delimiter
       // make sure we also match this version
       QString stripped = it.key().left( it.key().length() - 1 );
@@ -782,8 +776,9 @@
       }
     }
     // see if we have an empty namespace
-    if ( mNamespaceToDelimiter.contains( QString::null ) ) {
-      return mNamespaceToDelimiter[QString::null];
+    // this should always be the case
+    if ( mNamespaceToDelimiter.contains( "" ) ) {
+      return mNamespaceToDelimiter[""];
     }
     // well, we tried
     kdDebug(5006) << "delimiterForNamespace - not found" << endl;
@@ -1273,6 +1268,7 @@
   QString ImapAccountBase::createImapPath( const QString& parent, 
                                            const QString& folderName )
   {
+    kdDebug(5006) << "createImapPath parent="<<parent<<", folderName="<<folderName<<endl;  
     QString newName = parent;
     // strip / at the end
     if ( newName.endsWith("/") ) {