Bug 430451

Summary: Building Fingerprint abruptly stops
Product: [Applications] digikam Reporter: vadeinpacem
Component: Database-SimilarityAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: crash CC: caulier.gilles, metzpinguin
Priority: NOR    
Version: 7.2.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed In: 7.2.0
Sentry Crash Report:
Attachments: log file for building fingerprints

Description vadeinpacem 2020-12-16 07:42:57 UTC
Created attachment 134110 [details]
log file for building fingerprints

I am using the December 14 beta build for digikam 7.2.0 Win10, SQLite.

When building the fingerprint dB (from the maintenance menu)the program abruptly stops and digikam shuts down.  Restarting digikam shows similar behavior.

Thanks;  --Haim
Comment 1 caulier.gilles 2020-12-16 07:53:54 UTC
Comment on attachment 134110 [details]
log file for building fingerprints

It's difficult to found the origin of crash as there is nothing visible on the log. Typically the JPEG files are decoded while fingerprint computation.

It can be a JPEG decoding or a metadata parsing with Exiv2 failure.

Gilles Caulier
Comment 2 caulier.gilles 2020-12-16 07:56:14 UTC
Under Windows, the binary installers comes with debug symbols and a DrMinGw crash course handler, which will generate a file backtrace in you home directory as below:

    C:\Users\_user_name_\AppData\Local\digikam_crash.log

Just replace “user_name” with your Windows login account.

The crash log file exists on your computer ? If yes, please share...

Gilles Caulier
Comment 3 vadeinpacem 2020-12-16 09:12:31 UTC
Thanks for your prompt response.

I do not have a log file.  I installed from the Win64 binary that did not have debug symbols.
Please note that I used to work with local MySQL (MariaDB) and there was no problem with the fingerprint process.  There were other problems, hence I moved to SQLite.
Also, the image files are write protected.
I will install a version with debug symbols and see what happens.

Thanks Much.
Comment 4 vadeinpacem 2020-12-16 10:31:51 UTC
Hello,
Installed digiKam-7.2.0-beta2-20201214T120943-Win64-debug.
I do not see any digikam_crash.log
Please note that there is still a digikam process running on my machine.  It does not consume any CPU.
Thanks Much
Comment 5 caulier.gilles 2020-12-16 10:52:25 UTC
So, it's not a crash, but a break point somewhere, as a database lock for example...

Where is stored your sqlite database file ? Locally i hope, as sqlite do not support network shared file system...

Gilles Caulier
Comment 6 vadeinpacem 2020-12-16 12:17:57 UTC
Thanks for your comment.
I removed the similarity.db, started digikam and run the fingerprint process.
digikam broke at what seems to be the same point as before (the DB size is similar as well as the process progress bar - not really a scientific way).
The DBs are on local disk.
Is there a way for a more detailed log to suggest where the problem is?
Thanks Much;  --Haim
Comment 7 caulier.gilles 2020-12-16 12:30:13 UTC
If finger print process stop at the same place, it's more scientific that you expect. This want probably mean that an image cannot be parsed and generated an exception.

Another possibility is a file size restriction on your systeù. Which is the size of similarity database ? Do you have an active virus protection ?

Gilles Caulier
Comment 8 vadeinpacem 2020-12-16 13:07:52 UTC
Hello,
The scientific comment related to the progress bar.  I share your suspicion that there may be a problem with parsing a specific image.

1 - I did not add new images recently, and things worked OK with MariaDB.
2 - the similarity.db is small ~37MB.  The thumbnails.db is ~9.5GB.  I doubt its a problem with size on my HDD.
3 - I have the standard Win10 anti malware running.
4 - All my image files seems to work OK with exiftool.  This is the tool I use to add copyright as part of image ingestion.
5 - Is there a way to look at the similarity.db table and identify the last image it processed?

Thanks Much;  --Haim
Comment 9 vadeinpacem 2020-12-16 14:21:58 UTC
Hello,
I decided to skip fingerprinting and do facial recognition through the maintenance menu.
Results are the same.  digikam stops at what seems to be the same position.
It may reaffirm the notion that failure is at the same file.  Please remember that the thumbnails creation completed successfully.
Thanks Much;  --Haim
Comment 10 Maik Qualmann 2020-12-16 17:52:30 UTC
This extreme debug output from libjpeg is strange. It can be this file:

D:/Photography/Pictures/טיולים בחול/09-2018 Laugavegur איסלנד/05-09-2018 Reykjav?k/Iceland_2018-09-05_0129.jpg

You have also activated the tasks for cleaning up the database in the maintenance tool. You should deactivate this. If it's done once a year, that's enough.

Maik
Comment 11 vadeinpacem 2020-12-17 12:00:54 UTC
Thanks for your comments.
The file indicated seems OK and exiftool has no warnings regarding it.
Also, its probably not an issue with a single file.  
I run the facial recognition on 2 unique catalog subsets and it broke in each of them.  Of course its possible each contained a suspicious file.
I will continue checking.
Thanks much;  --Haim
Comment 12 vadeinpacem 2020-12-17 12:54:46 UTC
Hello,
I am now able to easily recreate the problem.  I identified the file.  Its a "psd" file 596MB.
I removed all collections for digikam, created a new collection with this file and digikam stops abruptly.
Please let me know what information is needed.
Thanks Much;  --Haim
Comment 13 Maik Qualmann 2020-12-17 13:01:02 UTC
The crash with 16 bit PSD files has just fixed in the current version. Please install and report Beta2 (2020-12-16)from here:

https://files.kde.org/digikam/

Maik
Comment 14 vadeinpacem 2020-12-17 13:10:17 UTC
Thanks Much;  --Haim
Comment 15 Maik Qualmann 2020-12-17 16:47:41 UTC
If you search for ".psd" in the log, you can see that this file is actually the last attempt to load.

https://invent.kde.org/graphics/digikam/commit/8e1a9ea9f88559ae5a9ca762135ab77845aaa78d

Maik
Comment 16 vadeinpacem 2020-12-19 10:14:43 UTC
Thanks much.

I would appreciate your observation on YOLO based facial recognition.  I am running it for close to a day and it scanned about 10% of the images.  It does not seem to use my GPU even though the 'disable hardware acceleration OpenCL' is not chosen.  Any way to use my GPU with YOLO?

Take Care;  --Haim