Bug 195006

Created attachment 34203 [details]
error message

Andrzej,

What do you mean by "national characters" ?

Gilles Caulier

in my case Polish characters like: ąćęłńśóżź
e.g. "photos" in Polish is "zdjęcia" and I cannot place database file in a directory named like this.

I confirm dysfunction

Gilles Caulier

Marcel, 

I cannot found where digiKam code is weird here. We pass dtabase path as QString... So the problem must be in sqlite or Qt4 Database driver...

Gilles Caulier

I have just tested to create a database in my russian-polish testing directory "Мои рисунки ąćęłńśóżź" and this worked without any problem. My system is Opensuse, system charset is UTF8. Maybe this is a problem at a lower level?

I use LANG=pl_PL that by default uses ISO 8859-2

2.0.0-rc is out, please re-open if needed

Still present in Digikam 2.2.o

Andrezej: SQLite or MySQL?

SQLite

Qt is internally 100% UTF-16, and the filename in the database driver is passed to sqlite3_open_v2 as UTF-8. So I dont see anything we can do in digikam about this. I have only used UTF-8 systems in the last years, perhaps someone with experience in non-UTF-8 can help us here.

Andrzej,

It still valid using digiKam 2.4 ?

Gilles Caulier

Gilles,

Yes it is still valid using digiKam 2.4.

I know where exactly problem is:
In digikamrc database location is stored in UTF8.

in my case:
/D/zdjęcia/

is stored like this:
Database Name=/D/zdjÄcia/
Database Name Thumbnails=/D/zdjÄcia/

This is UTF8 encodings seen in ISO 8859-2 locale.

so the solution would be convert the path stored in UTF8 in to current locale encodings just before accessing the database. In my case UTF8 -> ISO 8859-2.

Apparently now digiKam instead of looking for /D/zdjęcia/ is looking for /D/zdjÄcia/ that does not exit.

To confirm this I created symlink to /D/zdjęcia/ called /D/zdjÄcia/ and digiKam had no problem in finding the location.

If you start digikam on the console, it will output the database parameters:
digikam(9021)/digikam (core) Digikam::AlbumManager::setDatabase: DatabaseParameters: [ Type "QSQLITE", Name "/media/fotos/digikam4.db" (Thumbnails Name "/media/fotos/thumbnails-digikam.db"); ]

What does it say for you?

digikam(1786)/digikam (core): Error while opening the database. Error details [ QSqlError(-1, "Błąd podczas otwierania bazy danych", "unable to open database file") ]
digikam(1786)/digikam (core): Error while opening the database. Error details [ QSqlError(-1, "Błąd podczas otwierania bazy danych", "unable to open database file") ]
digikam(1786)/digikam (core): Error while opening the database. Error details [ QSqlError(-1, "Błąd podczas otwierania bazy danych", "unable to open database file") ]

and so on

ensure 50003 is enabled in kdebugdialog. It must be among the first few lines, before any db errors.

there is no kdebugdialog.
The:
digikam(1786)/digikam (core): Error while opening the database. Error details [
QSqlError(-1, "Błąd podczas otwierania bazy danych", "unable to open database
file") ]
error is the first message being displayed.

I've compiled digiKam with -DCMAKE_BUILD_TYPE=Release.
I will compile digiKam with more debug info.

I'll let you know.

kdebugdialog is a program from KDE :

http://www.youtube.com/watch?v=uw8FdebrIJs

You need to install it on your system...

Gilles Caulier

Created attachment 68155 [details]
konsole output

50003 was enabled in kdebugdialog.
I also compiled with -DCMAKE_BUILD_TYPE=relwithdebinfo.
But output is the same.

when I click on "cancel" there is plenty o repeated messages like below: 

QSqlDatabase: available drivers: QSQLITE QSQLITE3 QMYSQL3 QMYSQL
QSqlDatabase:  driver not loaded
QSqlDatabase: available drivers: QSQLITE QSQLITE3 QMYSQL3 QMYSQL
QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-168059104' is still in use, all queries will cease to work.
QSqlDatabasePrivate::addDatabase: duplicate connection name 'digikamDatabase-168059104', old connection removed.
QSqlQuery::prepare: database not open
QSqlDatabase:  driver not loaded
QSqlDatabase: available drivers: QSQLITE QSQLITE3 QMYSQL3 QMYSQL
QSqlQuery::prepare: database not open
QSqlDatabase:  driver not loaded
QSqlDatabase: available drivers: QSQLITE QSQLITE3 QMYSQL3 QMYSQL
QSqlQuery::prepare: database not open
QSqlDatabase:  driver not loaded
QSqlDatabase: available drivers: QSQLITE QSQLITE3 QMYSQL3 QMYSQL
QSqlDatabase:  driver not loaded
QSqlDatabase: available drivers: QSQLITE QSQLITE3 QMYSQL3 QMYSQL

Marcel,

I compiled with -DCMAKE_BUILD_TYPE=debugfull and I have the output you asked for.
the paths are displayed OK:
-> digikam
digikam(2426)/digikam (core) Digikam::KInotify::Private::open: Successfully opened connection to inotify: 10
digikam(2426)/digikam (core) Digikam::AlbumManager::setDatabase: DatabaseParameters: [ Type "QSQLITE", Name "/D/zdjęcia/digikam4.db" (Thumbnails Name "/D/zdjęcia/thumbnails-digikam.db"); ]

still digiKam is looking for database in nonexistent location because of wrong encoding.

Andrzej

"/D/zdjęcia/digikam4.db" seems to be the correct file name, and this one is read from the config file and passed as UTF-8 to the Qt SQL driver.
Well, I cannot find out from here where exactly something is wrong.

Marcel,

Database file name is read correctly from the config file but it it seems that it shouldn't be passed as UTF-8 to the Qt SQL driver but depending on locale settings because Qt SQL driver seems to be using it directly without taking locale settings in to account.

As I mentioned before:
To confirm this I created symlink to /D/zdjęcia/ called /D/zdjÄcia/ and digiKam i.e  Qt SQL driver had no problem in finding the location.

So the solution should be to pass the database path to Qt SQL driver depending on locale settings i.e. on UTF-8 systems settings it should be passed as UTF-8 but
on ISO 8859-2 systems it should be passed as ISO 8859-2.

I know that digiKam is checking what are locale settings so this information is already available.

Andrzej

If you have a look at
http://sqlite.org/c3ref/open.html
passing the filename as UTF-8 to sqlite_open_v2 is explicitly required, any other attempt would be a violation of the API documentation. See the note at the bottom of the page, even though it is only directed at Windows users: "The encoding used for the filename argument of sqlite3_open() and sqlite3_open_v2() must be UTF-8, not whatever codepage is currently defined. Filenames containing international characters must be converted to UTF-8 prior to passing them into sqlite3_open() or sqlite3_open_v2()"

This is what I mean: we have a 100% unicode passthrough of the filename through digikam and Qt, and obviously the data we have in digikam is correct.

Requirement that: "The encoding used for the filename argument of sqlite3_open() and sqlite3_open_v2() must be UTF-8" is from sqlite.

As far as understand there is an intermediary between digiKam and sqlite: Qt SQL driver.

Maybe Qt SQL driver does not have this requirement and non UTF-8 encoding can be passed.

The error message suggest that it is Qt SQL driver that is confused.

Which element should checked system locale and access database file accordingly?
Qt SQL driver or sqlite?

Either database path is provided as UTF-8 and then converted to local setting encoding by the element accessing database
or it should be possible to provide database path also as non UTF-8 if no locale settings are checked afterwards 

Andrzej

qsql_slite.cpp:508
bool QSQLiteDriver::open(const QString & db, const QString &, const QString &, const QString &, int, const QString &conOpts)
...
    if (sqlite3_open_v2(db.toUtf8().constData(), &d->access, openMode, NULL) == SQLITE_OK) {
...

QString is UTF-16 internally. So if the QString we pass in the beginning is correct Unicode, it is not changed in any way I can see. This is what I mean by 100% unicode.
Obviously something goes wrong, but I dont know where and particularly, I dont know what we could change.

We suspect a low problem to Qt sqlite plugin...

Gilles Caulier

Comment #14 says digiKam writes wrong configuration files, because the current locale isn't UTF-8.

There are too many issues with locale conversion, and Unicode has been invented to prevent those, so please change your locale to UTF-8.

Thanks Christoph for your feedback.

I move back this file into digiKam, for archiving purpose.

Gilles Caulier