Bug 388608 - Faces engine hangs and crashes
Summary: Faces engine hangs and crashes
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Engine (other bugs)
Version First Reported In: unspecified
Platform: Debian stable Linux
: NOR crash
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-01-06 10:13 UTC by Mathieu
Modified: 2018-08-24 09:58 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed/Implemented In: 6.0.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mathieu 2018-01-06 10:13:58 UTC
I handle large collection of images (about 100.000). I have used face engine to search and recognize faces on pictures (about 20.0000 faces).
Recognition is not perfect but globally works well. Now, I check and correct the affection for the faces and during this process, Digikam hangs, uses a lot of memory and crashes.

Last crash, I got these messages
...
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
digikam.facesengine: Clone Array size [ 1 ,  16384 ] of type  5
Minuterie d'alerte

Seems that's "Minuterie d'alerte" is a french translation for SIGALARM

Note that I have test with last version of digikam (5.8.0) using appimage and I got the same problem.
Comment 1 Maik Qualmann 2018-01-07 16:29:12 UTC
I think the cause is clear. We save a histogram for each detected face. So if a person is assigned the name 1000x, we have 1000 histograms of that person in the DB alone. That's just too much. Here, the number of histograms per person must be limited. We will have to test out what number still makes sense and ignore other histograms or update old ones.

Maik
Comment 2 caulier.gilles 2018-01-07 16:43:31 UTC
yes i agree that histogram must be limited in DB.

But this is true for LBPH recognition algorithm which is the current one.

What's the stability of new recognition algorithms introduced in 6.0.0 branch, as EigenFace, FisherFace, and DNN. The last one is completly different of all other as it's based on Deep Neural Network. This must be tested on a huge collection.

Gilles Caulier
Comment 3 caulier.gilles 2018-01-07 18:21:07 UTC
Maik,

I just tested with DNN recognition algorithm, after to remove recognition.db. The file is initialized, but nothing is stored in this DB with neural network.

Gilles
Comment 4 caulier.gilles 2018-08-17 21:27:28 UTC
Can you reproduce the dysfunction using digiKam 6.0.0 pre-release bundle available here :

https://files.kde.org/digikam/

Gilles Caulier
Comment 5 Mathieu 2018-08-18 09:24:04 UTC
Hi,

Okay, i will try next week.
Just to be sure: if I do a backup recognition.dd, digikam4.db and thumbnails-digikam.db, would I be able to come back to my current version ?

Mathieu
Comment 6 caulier.gilles 2018-08-18 09:27:51 UTC
yes it is. In all case, always do a backup of your database before to test with a beta version.

Gilles Caulier
Comment 7 Mathieu 2018-08-24 08:49:19 UTC
Hi,

I performed some tests with beta version 6.0.0.0 and it seems to work now.
Used memory does not increase anymore and Digikam does not crash.

Thanks !