Version: (using KDE KDE 3.3.0) Installed from: Unspecified Linux OS: Linux It's impossible connect to any IMAP server when the LC_ALL and LANG envs are set to Turkish, neither with kmail nor with konqueror. The problem occurs with both tr_TR and tr_TR.UTF-8 locales. When I start KDE with any other locale - like en_US, en_US.UTF-8, fr_FR*, de_DE or C - it seems to work fine. The problem is somewhat critical for Turkish KDE users. What I get as a err. message is pasted below. ----------- Turkish error message -------------- mail.metin.org sunucusu IMAP4 ya da IMAP4rev1 desteklemiyor. Sunucu kendisini bu şekilde tanımladı: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2004 Double Precision, Inc. See COPYING for distribution information. ----------- Turkish error message -------------- Message simply reports that, the server does not support neither IMAP4 nor IMAP4rev1. But, as you guess it does :).
The problem araise just in Turkish locale, cause in Turkish character lowercasing ascii character I produces a non-ascii (multibyte) character "i without the dot above" (0131;LATIN SMALL LETTER DOTLESS I). In kdepim/kioslaves/imap4/imapparser.cc imapParser::hasCapability function first lowercase the string (QString c = cap.lower();) then checks the existence of the protocol.
Created attachment 8023 [details] kdepim/kioslaves/imap4/imapparser.cc locale patch. Attached patch solves the imap connection problem. I know it is a trival solution but can show the problems source...
This happens when one uses Kontact (Kmail component) with Turkish locale, too.
> Created an attachment (id=8023) > --> (http://bugs.kde.org/attachment.cgi?id=8023&action=view) > kdepim/kioslaves/imap4/imapparser.cc locale patch. I like this solution. In fact, setting LC_CTYPE to "C" in the constructor of kio_imap would seem right to me, since nothing in there should depend on the current user's locale. Well the i18n calls do, of course, but those don't use LC_CTYPE anyway. Carsten, what do you think?
Raising severity to make sure it's fixed before 3.4
From talking to Trolltech and looking at Qt's source code, it seems that QCString::lower() is indeed locale-dependent, but QString::lower() is not (it uses Qt's unicode tables). In the patch you submitted, was the last part necessary? i.e. can you test if the first 2 hunks of the patch are enough? The cstr().lower() call certainly needs to be done with C locale, but the cap.lower() call *should* be fine as far as I can see from the QString code. This is especially important in order to fully understand the problem and get it fixed correctly in Qt4.
The Unicode conversion tables contain some exceptions for locale-dependent upper/lowercasing. One such example is 'I' in tr_TR. I am not sure if Qt implements it, but it is possible it will be in the future. Therefore, I don't recommend using QString::lower() either. Instead, in environments that are pure ASCII, we need a pure ASCII case-conversion function. This same bug has happened before for Konversation.
Currently Qt has some serious issues with Turkish Charset ( sorting, case convertiong problems etc. ). I think QString::lower() part is not necessary for now. Because it behaves buggy in TR locales and this buggy behaviour working for IMAP->imap conversion. See http://bugs.kde.org/show_bug.cgi?id=101211, funny! By the way bug #101211 reported to Trolltech, if they will change the QString::lower()/upper() functions to locale-dependent ( Trolltech must solve this locale problems, because of these problems KDE also behaves buggy for 70million Turkish people ) , then 2nd part of patch or pure-ascii conversion function will be needed in order to close this bug. Because there is also a Turkish "I"<->"i" maddness ( see http://www.i18nguy.com/unicode/turkish-i18n.html )
As Thiago mentioned there are exceptions in the case-conversion functions. Currently Qt's unicode table is broken and the bug is reported. Currently we do not need the second part of the patch but if Trolltech fix the problem with case-conversion tables we'll need it. So in an ideal (true) case-conversion we need the whole patch. Infact, can we benefit from newly added kasciistrcmp (and friends) functions? I think we can.
> Currently Qt's unicode table is broken and the bug is reported. How so? We don't want QString::lower() to be locale-dependent by default. OK there should probably be a localeAwareLower() or something, like there is a localeAwareCompare(). The bugs with the imap code are because of QCString::lower() being locale-dependent, but NOT because of QString that you say "should" be locale-dependent. The imap code is all ascii stuff, no locale-dependent functions needed nor wanted. Do you agree, or do I miss something?
Sure, in imap code we do not need any locale-dependent function. In this case you are right, we don't need the the second part of the patch. OTOH not having a locale-dependent QString::lower() is sad, cause there are some programs in KDE that depend on that kind of functionality. KWrite (as described in the bug:#101211 is one of them :(. So does this mean we have a different kind of challenge, fixing the programs that assume QString::lower() is locale-dependent, now?
On Wednesday 06 April 2005 13:04, Baris Metin wrote: > So does this mean we have a different kind of challenge, fixing the programs that assume QString::lower() is locale-dependent, now? Yes but I wonder how, if Qt doesn't offer anything for that. I'll ask on the qt4-preview-feedback list.
OK. I've also joined the list. Thanks.
> OK there should probably be a localeAwareLower() or something, like there is > a localeAwareCompare(). IMHO, I don't think that's a good idea. We'd get tons of subtle bugs of people using the wrong function in code.
On Thursday 07 April 2005 06:08, Thiago Macieira wrote: > > OK there should probably be a localeAwareLower() or something, like there is > > a localeAwareCompare(). > > IMHO, I don't think that's a good idea. We'd get tons of subtle bugs of people using the wrong function in code. ? But then we can fix it. Right now, some code wants locale-aware lower and some code wants ascii-based lower, so we're stuck: whatever Qt does, it breaks some code.
In my point of view, what we need is a consistent and well documented behaviour. I don't care if QString::lower() stay being locale-independent, I (and some others) can be a volunteer to fix the existing bugs in current KDE code. But (in time) the newly added code must not suffer from this problem. That's why I'm saying that we need a consistent and well documented beheviour, hoping this will be a solution.
Created attachment 11025 [details] kdepim/kioslaves/imap4/imapparser.cc patch, using libemailfunctions As kasciistricmp() is in kdepim/libemailfunctions now, we can use it without changing locale.
Applied the patch with approval from David Faure. Committed revision 413105.
SVN commit 428066 by kloecker: Fix another problem with case-conversion in tr_TR (ISO) locale. Based on patch by Barış Metin. CCBUG:91089 M +14 -12 imapparser.cc --- trunk/KDE/kdepim/kioslaves/imap4/imapparser.cc #428065:428066 @@ -57,6 +57,7 @@ #include <kurl.h> #include <kasciistricmp.h> +#include <kasciistringtools.h> imapParser::imapParser () { @@ -185,7 +186,7 @@ interact->len = strlen( (const char *) interact->result ); break; default: - interact->result = 0; + interact->result = 0; interact->len = 0; break; } @@ -216,7 +217,7 @@ return false; // result = sasl_client_new( isSSL ? "imaps" : "imap", - result = sasl_client_new( "imap", /* FIXME: with cyrus-imapd, even imaps' digest-uri + result = sasl_client_new( "imap", /* FIXME: with cyrus-imapd, even imaps' digest-uri must be 'imap'. I don't know if it's good or bad. */ aFQDN.latin1(), 0, 0, 0, 0, &conn ); @@ -252,7 +253,7 @@ tmp.resetRawData( out, outlen ); // then lets try it QString firstCommand = aAuth; - if ( !challenge.isEmpty() ) { + if ( !challenge.isEmpty() ) { firstCommand += " "; firstCommand += QString::fromLatin1( challenge.data(), challenge.size() ); } @@ -488,7 +489,7 @@ imapParser::parseResult (QByteArray & result, parseString & rest, const QString & command) { - if (command == "SELECT") + if (command == "SELECT") selectInfo.setReadWrite(true); if (rest[0] == '[') @@ -631,7 +632,8 @@ void imapParser::parseCapability (parseString & result) { - imapCapabilities = QStringList::split (' ', result.cstr().lower()); + QCString temp( result.cstr() ); + imapCapabilities = QStringList::split ( ' ', KPIM::kAsciiToLower( temp.data() ) ); } void imapParser::parseFlags (parseString & result) @@ -649,7 +651,7 @@ result.pos++; // tie off ( this_one.parseAttributes( result ); - + result.pos++; // tie off ) skipWS (result); @@ -1363,7 +1365,7 @@ if (lastHandled) envelope = lastHandled->getHeader (); - else + else lastHandled = new imapCache(); if (envelope && !envelope->getMessageId ().isEmpty ()) @@ -1422,7 +1424,7 @@ mailHeader *envelope = 0; if (lastHandled) envelope = lastHandled->getHeader (); - else + else lastHandled = new imapCache(); if (seenUid.isEmpty ()) @@ -1576,12 +1578,12 @@ QCString delim = parseOneWordC( result ); kdDebug(7116) << "imapParser::parseNamespace ns='" << prefix << "',delim='" << delim << "'" << endl; - if ( ns == 0 ) + if ( ns == 0 ) { // at least one personal ns personalAvailable = true; } - QString nsentry = QString::number( ns ) + "=" + QString(prefix) + + QString nsentry = QString::number( ns ) + "=" + QString(prefix) + "=" + QString(delim); imapNamespaces.append( nsentry ); if ( prefix.right( 1 ) == delim ) { @@ -1589,7 +1591,7 @@ prefix.resize( prefix.length() ); } namespaceToDelimiter[prefix] = delim; - + result.pos++; // tie off ) skipWS( result ); } else if ( result[0] == ')' ) @@ -1893,7 +1895,7 @@ inWords.takeLeftNoResize(retVal, i); retValSize = i; inWords.pos += i; - + if (retVal == "NIL") { retVal.truncate(0); retValSize = 0;
SVN commit 429639 by baris: Fix more case-conversion problems in tr_TR (ISO) locale. Based on the patch by Recai Oktas. CCBUG:91089 M +8 -5 configuredialog.cpp M +19 -5 kmmsgbase.cpp --- trunk/KDE/kdepim/kmail/configuredialog.cpp #429638:429639 @@ -74,6 +74,7 @@ #include <klocale.h> #include <kapplication.h> #include <kcharsets.h> +#include <kasciistringtools.h> #include <kdebug.h> #include <knuminput.h> #include <kfontdialog.h> @@ -715,7 +716,7 @@ QListViewItem *cur = mTransportList->selectedItem(); mModifyTransportButton->setEnabled( cur ); mRemoveTransportButton->setEnabled( cur ); - mSetDefaultTransportButton->setEnabled( cur ); + mSetDefaultTransportButton->setEnabled( cur ); } // adds a number to @p name to make the name unique @@ -914,7 +915,7 @@ } } } else { - GlobalSettings::setDefaultTransport( QString::null ); + GlobalSettings::setDefaultTransport( QString::null ); } delete item; @@ -3000,9 +3001,11 @@ QStringList charsets = composer.readListEntry( "pref-charsets" ); for ( QStringList::Iterator it = charsets.begin() ; it != charsets.end() ; ++it ) - if ( (*it) == QString::fromLatin1("locale") ) - (*it) = QString("%1 (locale)") - .arg( QCString( kmkernel->networkCodec()->mimeName() ).lower() ); + if ( (*it) == QString::fromLatin1("locale") ) { + QCString cset = kmkernel->networkCodec()->mimeName(); + KPIM::kAsciiToLower( cset.data() ); + (*it) = QString("%1 (locale)").arg( cset ); + } mCharsetListEditor->setStringList( charsets ); mKeepReplyCharsetCheck->setChecked( !composer.readBoolEntry( "force-reply-charset", false ) ); --- trunk/KDE/kdepim/kmail/kmmsgbase.cpp #429638:429639 @@ -14,6 +14,7 @@ #include <kdebug.h> #include <kglobal.h> #include <kcharsets.h> +#include <kasciistringtools.h> #include <kmdcodec.h> #include <krfcdate.h> @@ -570,7 +571,9 @@ const QTextCodec* KMMsgBase::codecForName(const QCString& _str) { if (_str.isEmpty()) return 0; - return KGlobal::charsets()->codecForName(_str.lower()); + QCString codec = _str; + KPIM::kAsciiToLower(codec.data()); + return KGlobal::charsets()->codecForName(codec); } @@ -762,8 +765,13 @@ if (charset == "us-ascii") return toUsAscii(_str); QCString cset; - if (charset.isEmpty()) cset = QCString(kmkernel->networkCodec()->mimeName()).lower(); - else cset = charset; + if (charset.isEmpty()) + { + cset = kmkernel->networkCodec()->mimeName(); + KPIM::kAsciiToLower(cset.data()); + } + else cset = charset; + const QTextCodec *codec = codecForName(cset); if (!codec) codec = kmkernel->networkCodec(); @@ -849,7 +857,10 @@ QCString cset; if ( charset.isEmpty() ) - cset = QCString( kmkernel->networkCodec()->mimeName() ).lower(); + { + cset = kmkernel->networkCodec()->mimeName(); + KPIM::kAsciiToLower( cset.data() ); + } else cset = charset; const QTextCodec *codec = codecForName( cset ); @@ -968,7 +979,10 @@ { QCString encoding = (*it).latin1(); if (encoding == "locale") - encoding = QCString(kmkernel->networkCodec()->mimeName()).lower(); + { + encoding = kmkernel->networkCodec()->mimeName(); + KPIM::kAsciiToLower(encoding.data()); + } if (text.isEmpty()) return encoding; if (encoding == "us-ascii") {