Bug 448585

Hi,

Please try with the just release 7.5.0 AppImage bundle available here :

https://download.kde.org/stable/digikam/7.5.0/

Look if problem have not been resolved yet...

Thanks in advance

Gilles Caulier

7.5.0 doesn't seem to automatically rebuild the thumbnails in the background which definitely fixed the issue. Thanks for that. I'll now manually rebuild the thumbnail db and report what happens.

FYI: On 7.1.0 I saw the following on the db log:

2022-01-16 09:46:04.522313	digikam[digikam] @  [192.168.0.197]	4234	1	Prepare	REPLACE INTO UniqueHashes (uniqueHash, fileSize, thumbId) VALUES (?,?,?)
2022-01-16 09:46:04.522426	digikam[digikam] @  [192.168.0.197]	4234	1	Reset stmt	
2022-01-16 09:46:55.311592	digikam[digikam] @  [192.168.0.197]	4234	1	Close stmt	

So the statement is prepared but never executed and the table seems to be locked in the mean time.
I'll have a look if something similar can be observed in 7.5.0.

Such a problem has not been reported before. The database discards the preparation. Either your database is broken or you have a drive error. The long hang time would also indicate a drive error.

Maik

(In reply to Maik Qualmann from comment #4)
> Such a problem has not been reported before. The database discards the
> preparation. Either your database is broken or you have a drive error. The
> long hang time would also indicate a drive error.
> 
> Maik

Hi Maik,

a database issue seems unlikely as my nextcloud instance is running on the same db and works fine. Db driver is part of the app image I'm assuming and that I've been using since Nov 2020, which would leave the question why the issue is appearing now.

Anyway, I'm still in the process of rebuilding the thumbnail db with the 7.5.0 app image and I'll report back if the issue is fixed once that is done.

One thing I noticed is, that with 7.1.0 each thumbnail was commited to the db separately (you could see the rows in the db immediately). That doesn't seem to be the case with 7.5.0 anymore. I'm through about 4000 photos now and the db tables are still empty. Looks to me like something there has changed?

Which database and which version do you use? digiKam is currently not compatible with MariaDB-10.6.x. As of MariaDB-10.6.x, certain changes are needed in Qt that are not present in the AppImage.

Maik

(In reply to Maik Qualmann from comment #6)
> Which database and which version do you use? digiKam is currently not
> compatible with MariaDB-10.6.x. As of MariaDB-10.6.x, certain changes are
> needed in Qt that are not present in the AppImage.
> 
> Maik

I'm on MariaDb 10.4.11

Okay, I'm done regenerating the thumbnails.

What I did:
Truncate humbnails, UniqueHashes and FilePath tables
Started to rebuild the thumbnail db via the corresponding maintenance option and waited till it is finished.

Result:
The three tables above are empty.

I tried again on a small subset. If I hit F5 digikam regenerates all the thumbnails but nothing is written to the db. This is repeatable and I'm pretty sure it is not what is supposed to happen.
You can also look at the debug log:
https://zerobin.net/?2495f223346befa1#n9rpdVGixUBCz2gkXmwiyrfR7ja9+uD9Ky1pawPIKMs=
Password: Underfed-Earshot2-Drudge

In your log file do not see any errors, normal behavior when creating thumbnails.

There is definitely no problem with the thumbnail database in digiKam, also the maintenance tool creates new thumbnails without any problems in the DB.

A REPLACE INTO UniqueHashes action should take almost no measurable time if the DB is ok. 

Maik

So does that mean that the fact that the Thumbnails, UniqueHashes and FilePath remain empty after the thumbnails db has been rebuild is expected behavior?

Why should the tables remain empty? As soon as thumbnails are requested, e.g. from the Item View Model and they are not available in the DB, they are created immediately and stored in the DB. So even when viewing albums, new thumbnails are created on the fly.

Maik

This is what I thought, but as I've written above, this is not what I'm seeing with 7.5.0. Instead the db tables remain permanently empty. This was not the case with 7.1.0. Hence I assume that there is an issue here. But (same as you) I can't see any error messages in the debug log.
Any ideas?

Please take a screenshot of the digiKam settings on the database settings page.

Maik

Created attachment 145582 [details]
Database settings

Sure. Added the screenshot as an attachment.

Ok, you are currently using a local SQLite DB as thumbnail database. We need to fix that. I'm amazed that QFileInfo maps the "photos" entry to your mount path "photos" as the local database path. Although it's not a full path.

Maik

ah, thanks for catching that.
That would also explain, why on my account the thumbnails are currently working fine while on my wife's account I'm still running into the same issues as with 7.1.0.
I.e., REPLACE INTO being stuck and taking forever to complete.
I can also try and upgrade the database. What is the highest version of mariaDb you are currently supporting?

As the upgrade to 7.5.0 has not resolved the original issue, I was thinking: Is there a possibility that the thread which creates the thumbnail, gets hung up somewhere between the PREPARE and the EXECUTE statement? That would also explain the behavior. Also, if I quit digikam while it is hung up on such an issue, the process doesn't actually end until the statement is closed. (Of course that would also be the case if the issue is with the db. Hence my suggestion that I'll try and update the db version.)

Btw.: The other issue (thumbnails silently being saved into a SQLite DB) seems to have happened because I had a folder in my home folder which had the same name as the db (photos).
Funnily this issues is a fix for the actual issue of this bug report. With the thumbnail db in a local SQLite file the "REPLACE INTO" issue does not appear.

Git commit b633b3dd412d26660bb0ed94cdc4f8d316b029f9 by Maik Qualmann.
Committed on 17/01/2022 at 22:04.
Pushed by mqualmann into branch 'master'.

check for absolute path for thumbnail name

M  +1    -1    core/libs/database/thumbsdb/thumbsdbaccess.cpp
M  +1    -1    core/libs/database/utils/widgets/dbsettingswidget.cpp

https://invent.kde.org/graphics/digikam/commit/b633b3dd412d26660bb0ed94cdc4f8d316b029f9

But the problem with REPLACE INTO must be in your system/database. In digiKam, a thread can hang if the database isn't ready, but a hanging thread can never block another thread that wants to use the database. We use a lock when accessing the database, only one thread can read/write from the database at a time. This is necessary for SQLite and would not necessarily be necessary for MySQL.
Please create a complete DebugView log with the problem.

Maik

I'm happy to do that.
Are you refering to anything specific when you say "DebugView" or just running digikam with
>  export QT_LOGGING_RULES="digikam*=true"
?

DebugView is a small application from Microsoft to catch debug message from Windows kernel and applications.

Download it and run before to start digiKam as explained here :

https://www.digikam.org/contribute/

Gilles Caulier

Ignore my request for DebugView, you are under Linux and can read the log directly in the terminal after setting the debug variable. I thought you were on Windows.

Maik

Okay, so to prepare, I've created a separate schema for the thumbnails (because of the naming issue that you just fixed) and pointed the thumbnail db in the settings to this. 
And now the issue is gone. I don't really understand it as it is still the same app image and still the same database but since I can't reproduce the issue I'll close this ticket and reopen one if I the issue appears again and I can reproduce it.
Thanks for your help.