Bug 379987 - UpdateSchemaFromV7ToV8: Unable to execute query (MySQL specific)
Summary: UpdateSchemaFromV7ToV8: Unable to execute query (MySQL specific)
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Mysql (show other bugs)
Version: 5.5.0
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-18 20:37 UTC by Quincy
Modified: 2018-01-04 08:26 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.8.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Quincy 2017-05-18 20:37:52 UTC
Starting digikam 5.5.0 for the first time after upgrading from (likely) 4.14 it reports the following error:

digikam.dbengine: Loading SQL code from config file "/usr/share/digikam/database/dbconfig.xml"
digikam.dbengine: Checking XML version ID => expected:  3  found:  3
digikam.coredb: Core database: running schema update
digikam.coredb: Core database: have a structure version  7
digikam.coredb: Core database: makeUpdates  7  to  8
digikam.dbengine: Failure executing query:
 "" 
Error messages: "QMYSQL: Unable to execute query" "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'IF EXISTS identifier' at line 2" 1064 2 
Bound values:  ()
digikam.dbengine: Error while executing DBAction [ "UpdateSchemaFromV7ToV8" ] Statement [ "ALTER TABLE AlbumRoots\n                                        DROP INDEX IF EXISTS identifier;" ]
digikam.coredb: Core database: schema update to V 8 failed!
digikam.coredb: Core database: cannot process schema initialization

My database is running on a local MySQL Server version 5.6.35.
Obviously this part was introduced as a fix for bug #372312.

According to the MySQL documentation this syntax is not supported by my MySQL version or the following one, but by MariaDB (as in the mentioned bug) and others.
Comment 1 Maik Qualmann 2017-08-23 19:46:17 UTC
*** Bug 383927 has been marked as a duplicate of this bug. ***
Comment 2 serge 2017-10-17 10:21:55 UTC
I have the same problem with 5.7.0. Are there any workaround here?
Comment 3 caulier.gilles 2017-12-13 22:43:01 UTC
With next 5.8.0 release Mysql support have been well improved and a lots of
bugs fixed.

Please test with pre release 5.8.0 bundles that we provide and give us a
feedback

https://files.kde.org/digikam/

Thanks in advance

Gilles Caulier
Comment 4 Maik Qualmann 2017-12-19 07:43:53 UTC
*** Bug 381386 has been marked as a duplicate of this bug. ***
Comment 5 Quincy 2017-12-30 02:35:30 UTC
Wanted to check the new update capabilities, but was unfortunately stopped by bug #388345.
Comment 6 Maik Qualmann 2017-12-30 07:01:21 UTC
Note: The problem will persist with MySQL. Our syntax works only with MariaDB. In the future we will probably only support MariaDB. The differences between the two projects grow with each new version.

One possible workaround, manually change the database:

mysql -u USERNAME -p

USE CORE_DATABASE_NAME

UPDATE Settings SET value=9 WHERE keyword='DBVersion';
UPDATE Settings SET value=9 WHERE keyword='DBVersionRequired';

Now start digiKam and copy the database with the migration tool into a new database.

Maik
Comment 7 Quincy 2017-12-30 08:58:20 UTC
As Maik said the new version does not solve/change this bug, so I am wondering why Gilles has posted to try the appimage to especially this bug.

At least I could test the upgrade from V8 to 9 with my manually "migrated" V7->8 DB using the appimage which works without new complains.
Still I am using copies of database and digikamrc, because they will be changed during the test run. I was wondering if "downgrading" afterwards will work at all.

If MariaDB is the only way to go you should state that more clearly, because e.g. the documentation tab in the database settings says literally "...to be connected to a remote Mysql database server (or MariaDB)" which implies that MySQL is usable, if not the first choice.
Comment 8 caulier.gilles 2017-12-30 10:11:52 UTC
Warning.

When i rebuild the last AppImage from 29 december, Maik has commit some supplemental fixes not included inside.

I'm not at home for 4 days, so i canot rebuild the AppImage for the moment. I will do it when i will go back next year.

PS : Happy new year.

Gilles
Comment 9 Quincy 2017-12-30 11:26:19 UTC
I'n not in a hurry given the raised points. A new image with localhost/socket support does not change things.

@Maik: I already went for a more manual (and cumbersome) step-by-step approach to migrate V7->8:
- doing the intended things ("IF EXISTS") by hand if suitable and commenting these things in dbconfig.xml
- delete some entries in the Images table because of foreign key violations (likely rubbish from older versions/crashes). I identified these via SELECT * FROM ImageMetadata WHERE imageid in (SELECT imageid FROM `ImageMetadata` LEFT JOIN Images ON(ImageMetadata.imageid=Images.id) WHERE Images.id IS NULL)
- creating faceDB by hand

But your suggestion sounds way easier for possible future updates and maybe others still being stuck on V7.

@all: Have a good start into 2018
Comment 10 caulier.gilles 2017-12-30 16:50:41 UTC
I'm not sure to understand all.

You use Mysql (or Mariadb) on your host computer (not in a remote one), but connected to digiKam as a remote one. Right ? 

If yes, this is why Maik talk about locahost address to use with AppImage. A local server (in term of digiKam) is a local database used as sqlite, not through the network.

So, my Q is : why not to use a local Mysql DB instead a Remote one locally. After all the performance will be the same and you don't need to customize the netwrok connection at all.

Gilles Caulier
Comment 11 Quincy 2017-12-30 17:36:34 UTC
I'm using MySQL on the very same computer as digikam. This is "local", but not internal to digikam as the MySQL server is running separately (for web development and other stuff, too). Because it is the same computer this obviously usually works via the socket. With the AppImage this has to go to "local network" likely because the AppImage is somehow complete in itself. This is still on the same computer, but then communicating via TCP/IP.

Indeed I would have other options for this setup, but was always (since the beginning with digikam 2.x) planning to move the MySQL server (together with the pictures) away from the digikam computer to end up in some kind of "multi-user" digikam. Network/server wise this would result in the distributed setup I already almost have (seperate MySQL instance). Main problem is/was access to remote collections (which was improved in the meantime) and some kind of "locking" mechanism to avoid multiple DigiKam instances working on the same DB and files which would cause big evil. But I didn't investigate that for quite a long time (other "projects" being more important).

So back to topic: No need for an updated AppImage from my side, as I got it running via the TCP/IP approach suggested by Maik. If it could work by finding the local socket (outside of the AppImage) it would be added value/less strange, but on that point personally I could switch to TCP/IP all the time.

Does that clarify things?
Comment 12 caulier.gilles 2017-12-30 17:57:28 UTC
yes it clear now.

I would to patc the handbook about the 127.0.01 tip from Maik when you use a local (remote) database, but if you said that it do not work as expected, i wait more details before.

Gilles Caulier
Comment 13 Maik Qualmann 2017-12-30 18:23:26 UTC
Git commit a4b162bba56e8555df6f54176ea3dcdb8e82f001 by Maik Qualmann.
Committed on 30/12/2017 at 18:21.
Pushed by mqualmann into branch 'master'.

add drop procedure to delete foreign key
FIXED-IN: 5.8.0

M  +2    -1    NEWS
M  +43   -33   data/database/dbconfig.xml.cmake.in

https://commits.kde.org/digikam/a4b162bba56e8555df6f54176ea3dcdb8e82f001
Comment 14 Maik Qualmann 2017-12-30 19:22:32 UTC
Git commit 051d57aa66d72ff3aa35e7240aed027434576254 by Maik Qualmann.
Committed on 30/12/2017 at 19:21.
Pushed by mqualmann into branch 'master'.

add drop procedure to delete index

M  +24   -18   data/database/dbconfig.xml.cmake.in

https://commits.kde.org/digikam/051d57aa66d72ff3aa35e7240aed027434576254
Comment 15 Quincy 2018-01-04 08:26:04 UTC
Current AppImage Version (including your fixes) happily upgrade DB Version 8->9.

Upgrade of a restored V7 DB to 8 (and then 9) first fails with:

digikam.dbengine: Failure executing query:
Error messages: "QMYSQL: Unable to execute query" "Cannot add or update a child row: a foreign key constraint fails (`digikam-appimage-core`.`#sql-948_247`, CONSTRAINT `ImageMetadata_Images` FOREIGN KEY (`imageid`) REFERENCES `Images` (`id`) ON DELETE CASCADE ON UPDATE CASCADE)" 1452 2 
Bound values:  ()
digikam.dbengine: Error while executing DBAction [ "UpdateSchemaFromV7ToV8" ] Statement [ "ALTER TABLE ImageMetadata ADD CONSTRAINT ImageMetadata_Images FOREIGN KEY (imageid) REFERENCES Images (id) ON DELETE CASCADE ON UPDATE CASCADE, ENGINE InnoDB;" ]
digikam.coredb: Core database: schema update to V 8 failed!

This is the issue I mentioned earlier with orphaned entries in (my) ImageMetadata which I could solve by hand (just included here for reference of the error message).


After removal of these entries, there are some complaints after the update process about thumbnails.ThumbSettings not being present:

digikam.dbengine: Loading SQL code from config file "/run/firejail/appimage/.appimage-5447/usr/share/digikam/database/dbconfig.xml"
digikam.dbengine: Checking XML version ID => expected:  3  found:  3
digikam.coredb: Core database: running schema update
digikam.coredb: Core database: have a structure version  7
digikam.coredb: Core database: makeUpdates  7  to  9
digikam.coredb: Core database: success updating to version  8
digikam.coredb: Core database: success updating to version  8
digikam.coredb: Core database: success updating to version  9
digikam.coredb: Core database: success updating to version  9
..snip..
digikam.dbengine: Prepare failed!
digikam.dbengine: Failure executing query:
 "SELECT value FROM ThumbSettings WHERE keyword=?;" 
Error messages: "QMYSQL3: Unable to prepare statement" "Table 'digikam-appimage-thumbnails.ThumbSettings' doesn't exist" 1146 2 
Bound values:  ()
digikam.dbengine: Failure executing query:
 "SELECT value FROM ThumbSettings WHERE keyword='DBThumbnailsVersion';" 
Error messages: "QMYSQL: Unable to execute query" "Table 'digikam-appimage-thumbnails.ThumbSettings' doesn't exist" 1146 2 
Bound values:  (QVariant(QString, "DBThumbnailsVersion"))
digikam.dbengine: Error while executing DBAction [ "SelectThumbnailSetting" ] Statement [ "SELECT value FROM ThumbSettings WHERE keyword=:keyword;" ]
digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret =  1
digikam.dbengine: Prepare failed!
digikam.dbengine: Failure executing query:
 "SELECT value FROM ThumbSettings WHERE keyword=?;" 
Error messages: "QMYSQL3: Unable to prepare statement" "Table 'digikam-appimage-thumbnails.ThumbSettings' doesn't exist" 1146 2 
Bound values:  ()
digikam.dbengine: Failure executing query:
 "SELECT value FROM ThumbSettings WHERE keyword='DBThumbnailsVersionRequired';" 
Error messages: "QMYSQL: Unable to execute query" "Table 'digikam-appimage-thumbnails.ThumbSettings' doesn't exist" 1146 2 
Bound values:  (QVariant(QString, "DBThumbnailsVersionRequired"))
digikam.dbengine: Error while executing DBAction [ "SelectThumbnailSetting" ] Statement [ "SELECT value FROM ThumbSettings WHERE keyword=:keyword;" ]
digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret =  1
digikam.thumbsdb: Thumbs database: have a structure version  ""
digikam.thumbsdb: ThumbDB SelectThumbnailLegacySetting val ret =  0
digikam.thumbsdb: ThumbDB SelectThumbnailLegacySetting val ret =  0
digikam.general: Thumbnails database ready for use


This was true in the original ThumbsDB before the update (V2: named "Settings" there), but it is renamed during the update process V2->V3 (visible in the table and dbconfig.xml, but not on console). Therefore these errors do not show up on a second start of digikam, but I was wondering why they show up right after the update run. So just a minor glitch, which I would not even have recognized when not watching console output...

Many thanks for your efforts resolving this!