Bug 431797

Summary: Face recognition <50% despite extensive training data of up to 300 occasions of faces.
Product: [Applications] digikam Reporter: Christian <chrispopp>
Component: Faces-RecognitionAssignee: Michael Miller <michael_miller>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, iain+kde, KDE, metzpinguin, michael_miller, neoderhacker, pschonmann, quochungtran1999, Stewart.Bodzin
Priority: NOR    
Version: 8.2.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed In: 8.5.0
Sentry Crash Report:

Description Christian 2021-01-18 21:10:38 UTC
SUMMARY
Face recognition delivers lousy results, i.e. correct proposals <1%. Furthermore, the proposals are predominantly persons, which I have only one confirmed picture for in the training set, while people with >100 manually confirmed pictures are never recognized. I don't understand, why this can happen at all, as documentation says: "Note: To be able to recognize a new face automatically, a minimum of 4 similar faces tagged in different images need to be previously trained using manual face management workflow." I find this a very senseful restriction, but unfortunately it doesn't seem to apply to my version of digiKam. I already tried "Clear and rebuild all training data" with no improvement. What can I do?

STEPS TO REPRODUCE
1. People view
2. Face detection 
3. Face recognition (Face accuracy 70, training done with manual face recognition for approx 100 different persons in approx. 1000 pictures)
4. Scan collection for faces

OBSERVED RESULT
Literally none of the proposals is correct. Proposed people are pre-dominantly "exotic" ones, which occured on only one picture in the training data

EXPECTED RESULT
At least correct proposals for people, which I had >100 occasions for in the training data

SOFTWARE/OS VERSIONS
Windows: Windows 10

ADDITIONAL INFORMATION
I apologize, if this is not a real bug. Is there a forum for digiKam users?
Comment 1 Maik Qualmann 2021-01-18 21:14:22 UTC
Use digiKam-7.2.0-RC from here:

https://files.kde.org/digikam/

Rebuild training database, the option is now in the maintenance tool.

Maik
Comment 2 Maik Qualmann 2021-01-18 21:54:18 UTC
See also comment:

https://bugs.kde.org/show_bug.cgi?id=415782#c22

Maik
Comment 3 Christian 2021-01-18 23:13:36 UTC
Hi Maik,

thanks for the advise, I'm currently running your proposed procedure, took a while. It came out with a significantly higher share of positive results, about 20%. So thank you very much, already helps a lot!

Indeed I drew my manual rectangles around faces rather generously with the intention to see people's complete heads in the people view. Only later I recognized that the people's view shows faces with some surroundings as standard, so that heads can be seen in most cases and I don't need to make sure by larger rectangles. If this creates problems for the training/recognition, I would really have an issue, as I did manual face detection in surely more than 1000 cases. Do I get it right that waiting for a later release might help or is this a substantial issue, which I only can undo with a complete rescan? (And indeed I later intend to include collections with faces detected by Picasa...)

Best, 
Christian
Comment 4 Thomas Beckler 2021-12-18 16:32:09 UTC
Same issue for me. Could you give an Update in this topic? If i have Seen correctly you have Not been able to get a Student working in this during Google Summer of Code. Have you Been working on any improvment?
I am looking forward to any change you plan.
Comment 5 Maik Qualmann 2021-12-18 19:52:02 UTC
During GSoC-2021, a student worked on the code. The code still has to be finalized and integrated into the digiKam master branch. I cannot confirm the poor face recognition of the current code. It is important that the face rectangles are detected by digiKam. Self-drawn rectangles or face rectangles from other programs (Picasa) are not recognized or recognized poorly.

Maik
Comment 6 Thomas Beckler 2021-12-21 21:06:19 UTC
Thank you for the update. This is great news that there has been some development.

I confirm that the YOLOv3 face detection/recognition is excellent. This thread is about the use case of manually drawn rectangles or rectangles drawn by other programs. 

https://bugs.kde.org/show_bug.cgi?id=415782#c22 states:
We will try to improve this as part of the GSoC 2021. With a current digikam, it is important to rebuild the training database

The rebuild of the training database doesn't help on my collection.  Can you confirm that the integration of the GSoC-2021 is going to include improvement in respect to the face rectangles drawn by others than digikam?
Comment 7 Maik Qualmann 2022-06-02 17:05:33 UTC
Git commit 5258bccf8eca4ae28ae5075438c1a85bf895d4c5 by Maik Qualmann.
Committed on 02/06/2022 at 17:03.
Pushed by mqualmann into branch 'qt5-maintenance'.

fix wrong accuracy value in face recognition from maintenance dialog
Related: bug 436544, bug 432537, bug 436727, bug 444160

M  +1    -1    core/utilities/facemanagement/database/facescansettings.cpp
M  +3    -1    core/utilities/maintenance/maintenancemngr.cpp

https://invent.kde.org/graphics/digikam/commit/5258bccf8eca4ae28ae5075438c1a85bf895d4c5
Comment 8 caulier.gilles 2023-04-30 08:14:07 UTC
@Christian

digiKam 8.0.0 is out. This entry still valid with this release ?

Best regards

Gilles Caulier
Comment 9 Maik Qualmann 2023-07-07 08:32:36 UTC
*** Bug 472031 has been marked as a duplicate of this bug. ***
Comment 10 iain 2023-07-07 19:43:50 UTC
I still face this issue with v8.0.0. I did not manually draw any of the face rectangles.

As described in 472031 my observation is that it worked well with 4-5 faces for a person but when I re-trained with hundreds the accuracy reduced massively.

Also worth noting a lot of my faces where tagged with previous digikam versions (6+).
Comment 11 Maik Qualmann 2023-07-26 07:47:49 UTC
*** Bug 472645 has been marked as a duplicate of this bug. ***
Comment 12 caulier.gilles 2023-10-15 10:34:26 UTC
@iain,


This problem still reproducible with the new digiKam 8.2.0 pre-release Windows
installer available at usual place:

https://files.kde.org/digikam/

This new bundle is based on last Qt framework 5.15.11 and KDE framework 5.110.

Thanks in advance

Gilles Caulier
Comment 13 iain 2023-10-29 21:41:11 UTC
(In reply to caulier.gilles from comment #12)
> @iain,
> 
> 
> This problem still reproducible with the new digiKam 8.2.0 pre-release
> Windows
> installer available at usual place:
> 
> https://files.kde.org/digikam/
> 
> This new bundle is based on last Qt framework 5.15.11 and KDE framework
> 5.110.
> 
> Thanks in advance
> 
> Gilles Caulier

Hi @Gilles,

I've tested 8.2.0 (revision 6ab20bb9162eba8823081cb7fc15597286c1001b) and it is much much better, I'd say it is about 50% accurate now. Still not as accurate as Picasa/Facebook manage.

Iain
Comment 14 caulier.gilles 2023-10-29 21:50:49 UTC
So the title of this file can be adjusted i think ?
Comment 15 caulier.gilles 2024-10-08 08:35:54 UTC
Hi Michael,

Probably the new SFace DNN model will be able to accurate over 50%. Right?

Best

Gilles
Comment 16 Michael Miller 2024-10-08 10:45:22 UTC
The new facial models coming in digiKam 8.5.0 are over 10x more accurate at recognizing the correct face when used with the proper settings and there aren't any accidentally confirmed bad faces assigned to a person.  This should make things much, much better.

Cheers,
Mike
Comment 17 caulier.gilles 2024-10-16 04:17:10 UTC
Michael,

Another entry (with duplicates) which can be closed for the next 8.5.0 release.
And yes, if a bugzilla entry has duplicates files referenced, usually we close as Resolved/Fixed this files to cleanup bugzilla database and update/notify users for the changes. 

Gilles
Comment 18 Maik Qualmann 2024-11-02 22:56:36 UTC
Git commit c003f95327d1239378241226140f9098b989e52e by Maik Qualmann, on behalf of Michael Miller.
Committed on 02/11/2024 at 22:56.
Pushed by mqualmann into branch 'master'.

closing tickets
Related: bug 469329, bug 415782, bug 472031, bug 464266, bug 423113, bug 444160, bug 436544
FIXED IN: 8.5.0

M  +8    -8    NEWS

https://invent.kde.org/graphics/digikam/-/commit/c003f95327d1239378241226140f9098b989e52e