Bug 126837 - KMail does not correctly detect UTF-8 encoding
Summary: KMail does not correctly detect UTF-8 encoding
Status: RESOLVED FIXED
Alias: None
Product: kmail
Classification: Applications
Component: messageviewer (show other bugs)
Version: 1.9.1
Platform: unspecified Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-05-06 09:47 UTC by Ronny Standtke
Modified: 2007-09-14 12:17 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
sample message demonstrating the bug. (3.30 KB, text/plain)
2006-05-21 19:58 UTC, Ronny Standtke
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ronny Standtke 2006-05-06 09:47:16 UTC
Version:           1.9.1 (using KDE 3.5.2, Debian Package 4:3.5.2-2+b1 (testing/unstable))
Compiler:          Target: i486-linux-gnu
OS:                Linux (i686) release 2.6.15-1-686

I receive many emails where KMail's automatic charset detection fails. 

It may have to do with the fact that in many headers the Content-Type definition has several lines, like that:
------------
Content-Type: text/plain;
  charset=UTF-8
------------

I had to manually set the encoding to UTF-8 (Ansicht-> Kodierung festlegen) to fix this.

I am not attaching my personal emails here but I can send it to anyone who is working on a fix...
Comment 1 Thiago Macieira 2006-05-14 11:12:39 UTC
Does it happen in ALL the emails with such a header? Or just some?
Comment 2 Ronny Standtke 2006-05-14 23:02:31 UTC
I don't know. I didn't check ALL emails. After checking some of the last messages in my folders I would say it happens always...
Comment 3 Ingo Klöcker 2006-05-15 00:43:55 UTC
Please make sure that you didn't accidentally force KMail to use another encoding, i.e. check that the "Override character encoding" in Settings->Configure KMail->Appearance->Message Window is set the "Auto". If it is set to "Auto" then I'll need a sample message demonstrating the bug.
Comment 4 Ronny Standtke 2006-05-21 19:58:42 UTC
Created attachment 16208 [details]
sample message demonstrating the bug.

"Override character encoding" is set to Auto, therefore I attach this example
message.
Comment 5 David Martínez Moreno 2006-07-28 22:35:39 UTC
Hello, Ronny.  I had the same problem since several months ago, until I sit down today with KMail sources determined to fix this bug.  It was then, trying to reproduce the bug with your mail, when I noticed that in my domestic Kmail I can read UTF-8 as was usual.  A fast diff showed the problem:

In .kde/share/config/kmailrc, section [Reader], I had these two statements:
[...]
autodetect-encoding=false
[...]
encoding=ISO 8859-1
[...]

    I bet that if you remove them, your problem will be fixed.

    Best regards,


        Ender.
Comment 6 Ronny Standtke 2006-07-29 08:43:18 UTC
Yes, that did the trick! But how can it come that KMail stumbles over its very own configuration file!?
Comment 7 Ronny Standtke 2006-07-29 08:45:12 UTC
An additional comment: In my config file it read before removal:
[...] 
autodetect-encoding=true
[...] 
encoding=ISO 8859-1 
[...] 
Comment 8 Ingo Klöcker 2006-07-29 23:26:07 UTC
You must have set the Override character encoding (that's what "encoding" refers to) to latin1 (iso 8859-1) some time in the past.

"autodetect-encoding" is no longer used by KMail. It must come from an older version of KMail.

Closing as INVALID because KMail did what it was told to do, i.e. always to use latin1 encoding.
Comment 9 Ronny Standtke 2006-07-30 13:01:13 UTC
I think you are a little bit too overhasty with marking this bug as "INVALID". 
KMail definately has a problem. If you have "encoding=ISO 8859-1" in kmailrc it does very stupid things:
First problem is that if you open "Settings -> Configure KMail... -> Appearance -> Message Window -> Override character encoding" the combobox shows "Auto".
Second problem is that even when closing the configuration dialog with "OK" the encoding is not set to "Auto" and in kmailrc it still reads "encoding=ISO 8859-1".
Comment 10 Stefan Gehn 2006-07-30 13:20:31 UTC
It's probably just and old leftover from prior kmail version. You have to switch from "Auto" to another encoding, hit "Ok", reopen prefs and set it back to "Auto". This is probably one of the rare cases where kmail fails to properly update kmailrc when a newer version is started.
Comment 11 Ingo Klöcker 2006-07-31 00:20:38 UTC
SVN commit 568026 by kloecker:

Fix bug 126837 (KMail does not correctly detect UTF-8 encoding)
Fall back to automatic encoding if we encounter an unknown encoding.

BUG: 126837



 M  +7 -0      configuredialog.cpp  
 M  +7 -0      kmreaderwin.cpp  


--- branches/KDE/3.5/kdepim/kmail/configuredialog.cpp #568025:568026
@@ -2309,6 +2309,13 @@
     }
     i++;
   }
+  if ( i == encodings.size() ) {
+    // the current value of overrideCharacterEncoding is an unknown encoding => reset to Auto
+    kdWarning(5006) << "Unknown override character encoding \"" << currentOverrideEncoding
+                    << "\". Resetting to Auto." << endl;
+    mOverrideCharsetCombo->setCurrentItem( 0 );
+    GlobalSettings::self()->setOverrideCharacterEncoding( QString::null );
+  }
 }
 
 void AppearancePage::ReaderTab::doLoadFromGlobalSettings()
--- branches/KDE/3.5/kdepim/kmail/kmreaderwin.cpp #568025:568026
@@ -1103,6 +1103,13 @@
           break;
         }
       }
+      if ( i == encodings.size() ) {
+        // the value of encoding is unknown => use Auto
+        kdWarning(5006) << "Unknown override character encoding \"" << encoding
+                        << "\". Using Auto instead." << endl;
+        mSelectEncodingAction->setCurrentItem( 0 );
+        mOverrideEncoding = QString::null;
+      }
     }
   }
   update( true );
Comment 12 Juha Tuomala 2006-08-01 11:06:09 UTC
Additional Comment #10 From Stefan Gehn 2006-07-30 13:20 ------- 
> It's probably just and old leftover from prior kmail 
> version. You have to switch from "Auto" to another 
> encoding, hit "Ok", reopen prefs and set it back 
> to "Auto". This is probably one of the rare cases 
> where kmail fails to properly update kmailrc when 
> a newer version is started. 
 
What if the organization has multiple versions of KDE in different
workstations with shared home dir? 

I think it would be better that programs would safely ignore
old settings and let them be there, so users would be able to
mix KDE/kmail versions, new and old, constantly.
Comment 13 BORGULYA Gábor 2006-11-09 11:49:03 UTC
My ~/.kde/share/config/kmailrc contains

[...]
[Reader]
[...]
FallbackCharacterEncoding=iso-8859-2
[...]
encoding=automatikus
[...]

And I have the same problem. I never edited kmailrc by hand. I agree with Juha that kmail should not get confused by the settings written out by its older versions.