Bug 387150

Summary:	crash when syncing metadata from images to database
Product:	[Applications] digikam	Reporter:	meku <kde.bugs>
Component:	Database-Mysql	Assignee:	Digikam Developers <digikam-bugs-null>
Status:	RESOLVED FIXED
Severity:	crash	CC:	caulier.gilles, metzpinguin
Priority:	NOR
Version First Reported In:	5.8.0
Target Milestone:	---
Platform:	Appimage
OS:	Linux
Latest Commit:	https://commits.kde.org/digikam/6a9fd9891a3ff341dc42fe36158cbf0101d513b6	Version Fixed/Implemented In:	5.8.0
Sentry Crash Report:

Description meku 2017-11-20 12:04:10 UTC

Attempting to read metadata to rebuild Tags database using the Maintenance tool, digikam crashes every time.

Running:
digikam-5.8.0-20171119094605-x86-64.appimage
external MySQL 5.7 database
Ubuntu 17.10

The console fills with repeats the database messages and then crashes:
SqlQuery::prepare: database not open
QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-140147803221072' is still in use, all queries will cease to work.
QSqlDatabasePrivate::addDatabase: duplicate connection name 'digikamDatabase-140147803221072', old connection removed.
QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-140147803221072' is still in use, all queries will cease to work.
QSqlDatabasePrivate::addDatabase: duplicate connection name 'digikamDatabase-140147803221072', old connection removed.
QSqlQuery::prepare: database not open
QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-140147937386496' is still in use, all queries will cease to work.
QSqlDatabasePrivate::addDatabase: duplicate connection name 'digikamDatabase-140147937386496', old connection removed.
QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-140147937386496' is still in use, all queries will cease to work.
QSqlDatabasePrivate::addDatabase: duplicate connection name 'digikamDatabase-140147937386496', old connection removed.

(process:26680): GLib-ERROR **: Creating pipes for GWakeup: Too many open files

/tmp/.mount_gtXgLF/AppRun: line 73: 26680 Trace/breakpoint trap (core dumped) digikam $@

Comment 1 Maik Qualmann 2017-11-20 19:33:07 UTC

Why here the connection to the DB fails, I can not say. But right now it's best to start with a completely clean DB. The tag table will be corrupted even if they delete all tags. It also has internal invisible tags. And you need a digiKam version of today, so that delete or move no longer comes to a destruction of the tags table.

Maik

Comment 2 caulier.gilles 2017-11-20 20:05:29 UTC

Maik,

Tomorrow, i will start AppImage bundle build with current implementation.

For the DB disconnection, this can be an issue with a network timeout if Mysql Db is a remote one.

I see also huge time latency with mysql when album contents is large and when thumbnails need to be stored to DB. In fact, the transactions in DB are not enough cut in small parts and this introduce timeout in low level interface. For this problem, it's a design of DB interface i think. The problem is also reproducible at face registrations or similarity fingerprints storage.

Gilles

Comment 3 caulier.gilles 2017-12-13 22:42:56 UTC

With next 5.8.0 release Mysql support have been well improved and a lots of
bugs fixed.

Please test with pre release 5.8.0 bundles that we provide and give us a
feedback

https://files.kde.org/digikam/

Thanks in advance

Gilles Caulier

Comment 4 Maik Qualmann 2017-12-26 21:25:18 UTC

Git commit 6a9fd9891a3ff341dc42fe36158cbf0101d513b6 by Maik Qualmann.
Committed on 17/12/2017 at 19:20.
Pushed by mqualmann into branch 'master'.

delete threads from memory when they are finished
Related: bug 375035, bug 321784, bug 325712, bug 328732, bug 330227, bug 331912, bug 344661, bug 345395, bug 350549, bug 381877, bug 338249, bug 329651, bug 329091, bug 387821, bug 381222

M  +2    -1    NEWS
M  +25   -0    libs/database/dbjobs/dbjobsmanager.cpp
M  +35   -0    libs/iojobs/iojobsmanager.cpp

https://commits.kde.org/digikam/6a9fd9891a3ff341dc42fe36158cbf0101d513b6