Bug 166222

Version:           4.0.85 (using Devel)
Installed from:    Compiled sources
Compiler:          gcc 4.1.2 --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-languages=c,c++,objc,obj-c++,java,fortran --disable-libgcj --with-cpu=generic --host=i686-pc-linux
OS:                Linux

Hi, i have port firefox's charset detection and add some patches to kde4.0.85, to make universal charset autodetection works in kwrite(kate) and konqueror(all apps use KEncodingDetector).

ftp://orafy:public@public.sjtu.edu.cn/mozilla-chardet-0.1.tar.bz2
ftp://orafy:public@public.sjtu.edu.cn/kdelibs-4.0.85-cmake.patch
ftp://orafy:public@public.sjtu.edu.cn/kdelibs-4.0.85-katedialogs.patch
ftp://orafy:public@public.sjtu.edu.cn/kdelibs-4.0.85-kcodecaction.patch
ftp://orafy:public@public.sjtu.edu.cn/kdelibs-4.0.85-kencoding.patch

Screenshot: kwrite's config dialog which shows the new added "Universal" option is
charset detection combox.
http://img61.imageshack.us/my.php?image=80961170fw7.png

untar mozilla-chardet-0.1.tar.bz2 and cmake && make && make install
mozilla-chardet depends on nothing so it's also easy to include it in the source branch.

I'm a chinese kde user,
 after I apply these patches, i have tested it by using kwrite to open big5/gb18030/enc-jp encoded documents, correctness is nearly 100%.
The encoding detection algorithm is very complex compared to kdecore/localization/*
A paper describe their methods:
 http://www.mozilla.org/projects/intl/UniversalCharsetDetection.html
Mozilla's related sourcecode:
http://lxr.mozilla.org/seamonkey/source/extensions/universalchardet/src/