Bug 226960

Summary: error with mysql 5.1.43: Last database error: "Can't initialize character set latin1
Product: [Frameworks and Libraries] Akonadi Reporter: Christophe Marin <christophe>
Component: serverAssignee: Volker Krause <vkrause>
Status: RESOLVED UPSTREAM    
Severity: normal CC: arekm, bjoern.ricks, budinero, christophe, fabo, freebsd, kde, kdepim-bugs, rdieter, robbat2
Priority: NOR    
Version: 4.5   
Target Milestone: ---   
Platform: Unlisted Binaries   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Christophe Marin 2010-02-15 11:19:53 UTC
This was reported several times and seem to affect the 5.1.43 users:

http://forum.kde.org/viewtopic.php?p=147126

from bug 225333#c5:

Here mysql 5.1.43, too and akonadiserver dies this way:

$ akonadiserver start
search paths:  ("/home/users/arekm/.vpython/bin", "/sbin", "/usr/sbin",
"/usr/local/sbin", "/home/users/arekm/bin", "/usr/lib64/qt4/bin",
"/usr/local/bin", "/usr/bin", "/bin", "/usr/games", "/usr/X11R6/bin",
"/home/users/arekm/bin", "/usr/sbin", "/usr/local/sbin", "/usr/local/libexec",
"/usr/libexec", "/opt/mysql/libexec", "/opt/local/lib/mysql5/bin")              
Database "akonadi" opened using driver "QMYSQL"                                 
DbInitializer::run()                                                            
checking table  "SchemaVersionTable"                                            
checking table  "ResourceTable"                                                 
checking table  "CollectionTable"                                               
checking table  "MimeTypeTable"                                                 
checking table  "PimItemTable"                                                  
checking table  "FlagTable"                                                     
checking table  "PartTable"                                                     
checking table  "CollectionAttributeTable"                                      
checking relation  "PimItemFlagRelation"
checking relation  "CollectionMimeTypeRelation"
checking relation  "CollectionPimItemRelation"
DbInitializer::run() done
skipping update 2
skipping update 3
skipping update 4
skipping update 8
skipping update 10
skipping update 12
skipping update 13
skipping update 14
Character set 'latin1' is not a compiled character set and is not specified in
the '/usr/share/mysql/charsets/Index.xml' file
Nepomuk QueryServer interface not available!
Connecting to deprecated signal
QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
DataStore::unhideAllPimItems()
Character set 'latin1' is not a compiled character set and is not specified in
the '/usr/share/mysql/charsets/Index.xml' file
Database error: Cannot open database.
Last driver error: "QMYSQL: Unable to connect"
Last database error: "Can't initialize character set latin1 (path:
/usr/share/mysql/charsets/)"
Database error: Cannot open database.
Last driver error: "QMYSQL: Unable to connect"
Last database error: "Can't initialize character set latin1 (path:
/usr/share/mysql/charsets/)"
Unable to connect to dbus service:  ""
"[
0: akonadiserver(_Z11akBacktracev+0x39) [0x40ad19]
1: akonadiserver() [0x40b262]
2: /lib64/libc.so.6(+0x32520) [0x7fbd34dd1520]
3: /lib64/libc.so.6(gsignal+0x35) [0x7fbd34dd14a5]
4: /lib64/libc.so.6(abort+0x180) [0x7fbd34dd2970]
5: /usr/lib64/libQtCore.so.4(_Z17qt_message_output9QtMsgTypePKc+0x74)
[0x7fbd35f7d7c4]
6: akonadiserver(_ZN15FileDebugStream9writeDataEPKcx+0xa8) [0x40c328]
7: /usr/lib64/libQtCore.so.4(_ZN9QIODevice5writeEPKcx+0x77) [0x7fbd3600c197]
8: /usr/lib64/libQtCore.so.4(+0x10b529) [0x7fbd3601d529]
9: /usr/lib64/libQtCore.so.4(_ZN11QTextStreamD1Ev+0x39) [0x7fbd3601e729]
10: akonadiserver(main+0x542) [0x406022]
11: /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fbd34dbdc2d]
12: akonadiserver() [0x4059e9]
]
"

but /usr/share/mysql/charsets/Index.xml contains latin1 definition:
<charset name="latin1">
  <family>Western</family>
  <description>cp1252 West European</description>
  <alias>csisolatin1</alias>
  <alias>iso-8859-1</alias>
  <alias>iso-ir-100</alias>
  <alias>iso_8859-1</alias>
  <alias>iso_8859-1:1987</alias>
  <alias>l1</alias>
  <alias>latin1</alias>
  <collation name="latin1_german1_ci"    id="5"    order="German Duden"/>
  <collation name="latin1_swedish_ci"    id="8"    order="Finnish, Swedish">
    <flag>primary</flag>
    <flag>compiled</flag>
  </collation>
  <collation name="latin1_danish_ci"    id="15"    order="Danish"/>
  <collation name="latin1_german2_ci"    id="31"    order="German Phonebook"   
flag="compiled"/>
  <collation name="latin1_spanish_ci"    id="94"    order="Spanish"/>
  <collation name="latin1_bin"        id="47"    order="Binary">
    <flag>binary</flag>
    <flag>compiled</flag>
  </collation>
  <collation name="latin1_general_ci"    id="48">
    <order>Dutch</order>
    <order>English</order>
    <order>French</order>
    <order>German Duden</order>
    <order>Italian</order>
    <order>Latin</order>
    <order>Portuguese</order>
    <order>Spanish</order>
  </collation>
  <collation name="latin1_general_cs"    id="49">
    <order>Dutch</order>
    <order>English</order>
    <order>French</order>
    <order>German Duden</order>
    <order>Italian</order>
    <order>Latin</order>
    <order>Portuguese</order>
    <order>Spanish</order>
  </collation>
</charset>

--
from https://bugs.kde.org/show_bug.cgi?id=225333#c14 :

I'm having the same error:
...
Database error: Cannot open database.
  Last driver error: "QMYSQL: Unable to connect"
  Last database error: "Can't initialize character set latin1 (path:
/usr/share/mysql/charsets/)"
...

Akonadi starts, but is unusable for korganizar and kmail (empty folder list in
resources config, when there are some akonadi calendar sources) 

It's not a bad MySQL config. I purged and reinstalled it and removed all
akonadi config files from ~. 
Comment #12 works for me, downgrading to .41 in debian solves the problem, but
with .43 is not working.


Debian and Mandriva look affected. Nothing in the mysql changelog:
http://dev.mysql.com/doc/refman/5.1/en/news-5-1-43.html
Comment 1 Christophe Marin 2010-02-15 11:22:29 UTC
Can one of you look in /usr/share/mysql/charsets and report if there's a 'latin1.xml' file there ?
Comment 2 Christophe Marin 2010-02-15 11:45:41 UTC
@ Arkadiusz your akonadi log indicates that the mysql server is being searched in these directories: /opt/mysql/libexec, /opt/local/lib/mysql5/bin

Where is mysqld installed on your machine ? 

can you paste the output of : `mysql_config`
Comment 3 Arkadiusz Miskiewicz 2010-02-15 12:20:19 UTC
latin1.xml is there:
-rw-r--r-- 1 root root 9770 02-02 10:12 /usr/share/mysql/charsets/latin1.xml

mysqld is at /usr/sbin/mysqld (and akonadi finds it now since:
arekm     3659  0.0  0.8 226480 31952 ?        Sl   Feb13   1:04 /usr/sbin/mysqld --defaults-file=/home/users/arekm/.local/share/akonadi//mysql.conf --datadir=/home/users/arekm/.local/share/akonadi/db_data/ --socket=/home/users/arekm/.local/share/akonadi/db_misc/mysql.socket
)

[arekm@carme-pld ~]$ mysql_config
Usage: /usr/bin/mysql_config [OPTIONS]
Options:
        --cflags         [-I/usr/include/mysql  -DUNIV_LINUX -DUNIV_LINUX]
        --include        [-I/usr/include/mysql]
        --libs           [-L/usr/lib64 -lmysqlclient -lz -lcrypt -lm -lssl -lcrypto]
        --libs_r         [-L/usr/lib64 -lmysqlclient_r -lz -lpthread -lcrypt -lm -lpthread -lssl -lcrypto]
        --plugindir      [/usr/lib64/plugin]
        --socket         [/var/lib/mysql/mysql.sock]
        --port           [0]
        --version        [5.1.43]
        --libmysqld-libs [-L/usr/lib64 -lmysqld -ldl -lz -lpthread -lcrypt -lm -lpthread -lwrap -lrt -lssl -lcrypto]
Comment 4 Christophe Marin 2010-02-15 15:35:25 UTC
Thanks.

I just built MySQL 5.1.43 from sources with the following command line:

./configure --prefix=/usr/local --with-plugins=max --enable-shared --enable-static --with-ssl --without-docs

and wiped the previous akonadi DB to test a first run initialization.
No issue so far:

# which mysqld
/usr/local/libexec/mysqld

# mysqld --version
mysqld  Ver 5.1.43 for pc-linux-gnu on i686 (Source distribution)

# akonadictl status
Akonadi Control: running
Akonadi Server: running
Akonadi Server Search Support: available (backend: Virtuoso)

# ps x |grep mysqld 
21519 ?        Sl     0:01 /usr/local/libexec/mysqld --defaults-file=/home/krop/.local/share/akonadi//mysql.conf --datadir /home/krop/.local/share/akonadi/db_data/ --socket=/home/krop/.local/share/akonadi/db_misc/mysql.socket

I'll now try the configure parameters used in the Debian package to check whether there's a difference or no.
Comment 5 Christophe Marin 2010-02-15 16:37:20 UTC
I didn't have more luck with the Debian (SID) options:

# ./configure --prefix=/usr/local --exec-prefix=/usr/local --libexecdir=/usr/local/sbin --datadir=/usr/local/share --localstatedir=/var/lib/mysql --includedir=/usr/local/include --infodir=/usr/local/share/info --mandir=/usr/local/share/man --build=i486-linux-gnu --host=i486-linux-gnu --with-server-suffix="-1" --with-comment="(Debian)" --with-system-type="debian-linux-gnu" --enable-shared --enable-static --enable-thread-safe-client --enable-assembler  --enable-local-infile --with-pstack --with-fast-mutexes --with-big-tables --with-unix-socket-path=/var/run/mysqld/mysqld.sock --with-mysqld-user=mysql --with-libwrap --without-readline --with-ssl --without-docs --with-extra-charsets=all --with-plugins=max --without-ndbcluster --with-embedded-server --with-embedded-privilege-control

and Akonadi still works.

# mysql_config 
Usage: /usr/local/bin/mysql_config [OPTIONS]
Options:
        --cflags         [-I/usr/local/include/mysql -DUNIV_LINUX -DUNIV_LINUX]
        --include        [-I/usr/local/include/mysql]
        --libs           [-rdynamic -L/usr/local/lib/mysql -lmysqlclient -lz -lcrypt -lnsl -lm]
        --libs_r         [-rdynamic -L/usr/local/lib/mysql -lmysqlclient_r -lz -lpthread -lcrypt -lnsl -lm -lpthread]
        --plugindir      [/usr/local/lib/mysql/plugin]
        --socket         [/var/run/mysqld/mysqld.sock]
        --port           [0]
        --version        [5.1.43]
        --libmysqld-libs [-rdynamic -L/usr/local/lib/mysql -lmysqld -ldl -lz -lpthread -lcrypt -lnsl -lm -lpthread -lrt]
Comment 6 Christophe Marin 2010-02-15 21:23:35 UTC
I applied the Debian patches, still no luck. I'm unable to reproduce the charset issue.
Comment 7 Facundo Aguilera 2010-02-15 22:23:04 UTC
This is the answer from debian packagers
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=569549

I only needed to downgrade mysql packs
Comment 8 Rex Dieter 2010-02-18 21:35:52 UTC
See also downstream report,
https://bugzilla.redhat.com/show_bug.cgi?id=566547

init'ing akonadi with 5.1.43 fails with the error(s) listed here, and that 5.1.42 was last-known good.

is there anything concrete to go on to be able forward on to mysql upstream that "foo and bar" in 5.1.43 is broken?  Or is it a matter of just being incompatible somehow?
Comment 9 Robin Johnson 2010-02-19 20:09:59 UTC
(Gentoo maintainer of MySQL here)
There IS an entry in the 5.1.43 changelog about charset handling, with code that was changed between 5.1.42 and 5.1.43:
http://bugs.mysql.com/bug.php?id=45058

I'm not a KDE user myself, so if somebody could try 5.1.43 with the following patch reverted:
http://lists.mysql.com/commits/87152
That would probably help a lot in tracing. Alternatively try 5.1.42 with the patch added and see if it breaks where it worked before.
Comment 10 Rex Dieter 2010-02-19 20:15:04 UTC
I tried, but it seems only parts of that referenced commit were included in 5.1.43, so would require a bit of surgery to revert.
Comment 11 Bartosz Fabianowski 2010-02-20 19:00:24 UTC
A patch reverting the charset handling changes in 5.1.43 is now available in Redhat's Bugzilla:

https://bugzilla.redhat.com/show_bug.cgi?id=566547#c11

This fixed Akonadi with MySQL 5.1.43 under Redhat Linux for the patch author. I am using MySQL 5.5.1 under FreeBSD. The same patch fixed Akonadi for me as well. So yes, it is this change to MySQL that makes Akonadi fail. It would seem like it is not a KDE bug at all.
Comment 12 Rex Dieter 2010-02-20 20:29:16 UTC
update:  mysql has been notified of the problem introduced in
http://bugs.mysql.com/45058

(waiting for a bit more detail and evidence before closing->upstream)
Comment 13 Christophe Marin 2010-02-21 14:56:50 UTC
Adding the upstream URL. 
I could finally reproduce this bug. Looks like my local 5.1.43 build was loading libraries from another installed version.

The issue can also be reproduced with with 5.5.1.2 from the unstable opensuse repo (I then get the same backtrace as the one added to the upstream report).

[akonadiserver] "[
[akonadiserver] 0: akonadiserver(_Z11akBacktracev+0x2e) [0x805a292]
[akonadiserver] 1: akonadiserver [0x805a676]
[akonadiserver] 2: [0xffffe400]
[akonadiserver] 3: /usr/lib/libmysqlclient_r.so.16(my_strcasecmp_8bit+0x2b) [0xb60fb56b]
[akonadiserver] 4: /usr/lib/libmysqlclient_r.so.16(get_charset_number+0x7a) [0xb60edefa]
[akonadiserver] 5: /usr/lib/libmysqlclient_r.so.16(get_charset_by_csname+0x6e) [0xb60eeafe]
[akonadiserver] 6: /usr/lib/libmysqlclient_r.so.16(mysql_init_character_set+0x82) [0xb6110182]
[akonadiserver] 7: /usr/lib/libmysqlclient_r.so.16(mysql_real_connect+0x2a8) [0xb61122c8]
[akonadiserver] 8: /usr/lib/qt4/plugins/sqldrivers/libqsqlmysql.so [0xb6252acb]
[akonadiserver] 9: /usr/lib/libQtSql.so.4(_ZN12QSqlDatabase4openEv+0x3f) [0xb6ea360f]
[akonadiserver] 10: /kde/inst/akonadi/lib/libakonadiprivate.so.1(_ZN7Akonadi9DataStore4openEv+0x233) [0xb76f76ab]
[akonadiserver] 11: /kde/inst/akonadi/lib/libakonadiprivate.so.1(_ZN7Akonadi9DataStoreC1Ev+0xaa) [0xb76f7104]
[akonadiserver] 12: /kde/inst/akonadi/lib/libakonadiprivate.so.1(_ZN7Akonadi9DataStore4selfEv+0x3f) [0xb76f7b23]
[akonadiserver] 13: /kde/inst/akonadi/lib/libakonadiprivate.so.1(_ZN7Akonadi12CacheCleaner3runEv+0x14) [0xb77265d8]
[akonadiserver] 14: /usr/lib/libQtCore.so.4 [0xb73caf4f]
[akonadiserver] 15: /lib/libpthread.so.0 [0xb73506e5]
[akonadiserver] 16: /lib/libpthread.so.0 [0xb7350600]
[akonadiserver] ]
Comment 14 Robin Johnson 2010-02-23 01:27:18 UTC
Trying to help upstream MySQL guys on this, can somebody please provide the full argument string that akonadi uses to start the private mysqld instance, as well as any private my.cnf files with mysqld settings?
Comment 15 Christophe Marin 2010-02-23 08:17:17 UTC
mysqld is started like that:

/usr/sbin/mysqld --defaults-file=/home/login/.local/share/akonadi//mysql.conf --datadir /home/login/.local/share/akonadi/db_data/ --socket=/home/login/.local/share/akonadi/db_misc/mysql.socket

and the mysql.conf used by akonadi 1.3.1 is here: 
http://websvn.kde.org/tags/akonadi/1.3.1/server/src/storage/mysql-global.conf?view=markup
Comment 16 Robin Johnson 2010-02-26 19:32:32 UTC
Please see the MySQL-upstream bug for their final patches to solve the issue.
Comment 17 Rex Dieter 2010-02-26 19:34:35 UTC
I can confirm in my initial testing that those do the trick.

stick a fork in it!