Bug 444160 - Facing matching is guess work at best
Summary: Facing matching is guess work at best
Status: CONFIRMED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Recognition (show other bugs)
Version: 7.3.0
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-10-20 23:35 UTC by Jonathan
Modified: 2024-02-08 11:29 UTC (History)
9 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Wrong image recognition (80.51 KB, image/webp)
2023-10-15 11:04 UTC, Hans Lauter
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan 2021-10-20 23:35:18 UTC
SUMMARY

I added a collection of 40,000 pictures, and set digikam to the yolo v3 highest level scanning.  It detected a lot of face, after 4 days of scanning.
However, in step 2 of face matching it just didn't match many faces and most of the ones it did match were not even close.

STEPS TO REPRODUCE
1. Add collection of 40,000 photos
2. set to yolo v3 and multi-core,run face detection wait 4 days
3. Do face matching 

OBSERVED RESULT
95% of the results were just plain wrong.  I tried adjusting the accuracy slider, but it just resulted in few results.  Using pHash set to 98% usually results in better matches, but this program doesn't support it.

Google picasa is way faster, and it's matches are mostly right.

EXPECTED RESULT

Facing matching to actually match faces correctly.

SOFTWARE/OS VERSIONS
Windows: 10 build 20H2
Comment 1 Maik Qualmann 2021-10-21 05:51:16 UTC
How many faces did you learn per person?

Maik
Comment 2 Jonathan 2021-10-21 11:58:03 UTC
Obviously it varies some have 100+ faces, and some only have a few samples.  However, I still have 20,000+ unmatched faces.  Even though they may not match a known person they should be auto categorized into groups of different unknown faces.
Comment 3 dajomu 2021-11-05 12:47:26 UTC
I agree with Jonathan. Instead of listing unknown and unconfirmed faces as an endless list of photos, similar faces should be grouped like e.g. Picasa. In digikam there is one "album" which contains unknown faces while in Picasa there are one album for each unknown face.
One example can be seen here: https://spekxvision.files.wordpress.com/2016/05/052316_0930_facialrecog1.png
Comment 4 dajomu 2022-03-21 11:36:15 UTC
Are there any progress on this issue and especially what is mentioned in this comment "Even though they may not match a known person they should be auto categorized into groups of different unknown faces." Grouping unknowns by face would make the training a whole lot easier.
Comment 5 Maik Qualmann 2022-06-02 17:05:41 UTC
Git commit 5258bccf8eca4ae28ae5075438c1a85bf895d4c5 by Maik Qualmann.
Committed on 02/06/2022 at 17:03.
Pushed by mqualmann into branch 'qt5-maintenance'.

fix wrong accuracy value in face recognition from maintenance dialog
Related: bug 436544, bug 432537, bug 431797, bug 436727

M  +1    -1    core/utilities/facemanagement/database/facescansettings.cpp
M  +3    -1    core/utilities/maintenance/maintenancemngr.cpp

https://invent.kde.org/graphics/digikam/commit/5258bccf8eca4ae28ae5075438c1a85bf895d4c5
Comment 6 caulier.gilles 2023-05-02 08:20:26 UTC
@Jonathan

digiKam 8.0.0 is out. This entry still valid with this release ?

Best regards

Gilles Caulier
Comment 7 caulier.gilles 2023-10-15 10:07:58 UTC
@Jonathan,


This problem still reproducible with the new digiKam 8.2.0 pre-release Windows
installer available at usual place:

https://files.kde.org/digikam/

This new bundle is based on last Qt framework 5.15.11 and KDE framework 5.110.

Thanks in advance

Gilles Caulier
Comment 8 Hans Lauter 2023-10-15 10:57:39 UTC
I have the same problem.

The suggestions of face recognition output many wrong results. Around 20-40% of all suggestions are right. This makes the manual rework quite tedious (the benefit of using a face recognition evaporates). Digikam suggests faces, which are completely different from each other. For example a young boy gets assigned to the same person like the ones from a 80 yo lady. It makes no sense at all. On the other side images which are completely similar to already labelled ones, are not suggested, but are put in the group "unknown". So yeah, I can confirm, this recognition is guess work. Face detection works quite good (it sometimes detects objects as people, but the failure rate is here is acceptable), but face recognition malperforms.

Some additions:
- Deleting all databases and rebuilding them did not help
- Retraining the model (maintenance/...) did not help
- Changing the accuracy slider from the default 70% to 90% did not help (maybe it changed the quantity, but not quality).

I use digikam 8.1.0.

Sidenote: I noted that if I did labeled just a few pictures (~10) for a person, then the suggestions are way better but also much less. So if I label 10 pictures to a person X, the algortihm returns lets sa 5-20 pictures, which are actually that person. But for a person Y, if I label > 100 more pictures, the suggestion become really bad. Isn't it the opposite, that AI get better, the more you give?
You might say: Then I gave many bad labelled images to person Y. But no, the pictures for person Y are in a good quality, similar facial expression, so the variance is not too high.
Comment 9 Hans Lauter 2023-10-15 11:04:29 UTC
Created attachment 162319 [details]
Wrong image recognition

An example: If I go on the suggestions for myself as a person in digikam. Around 50-100 different people are suggested there, several flowers,  a dog, a package of burger king and dumbledore is assigned to me.

In case of the flowers and the package of burger king: Ok the image detection failed. But why are they assigned to me? Its ok to have them in the category "unknown", but they have nothing to do with me (considering the accuracy is set to 90%).

Moreover, the 50-100 people look totally different than me. Nevertheless, they all tag-suggested as me.
Comment 10 atoms 2024-01-14 07:39:42 UTC
Yeah there is something really wrong with the face recognition, it is simply not working as it should - not exactly a hard thing to do any more either, the tech has been out for a very long time.  So not only are we met with 4 days or more of waiting to detect faces, then we're met with bad results.  It's quicker to do them manually, even for 90,000 images and that's embarrassing.

I suspect, clicking minus (not this person) and tick (yes this person), is not actually training the model as it should be.  That's what it 'feels' like to me, but could be wrong.  This is still happening on 8.3.0.

I have access to Windows, Linux and Mac and various drives and an Nvidia GPU if anyone wants me to help in any way.

On my wish list, we would get GPU acceleration working first, which seems counter intuitive if it gives bad results, but the reasoning is we don't have to wait days and redo it all the time just to find out it didn't work.

Then after GPU we would focus on quality face matching problems.  Unless of course it's a simple as 'oh the tick and minus aren't actually training it', then yeah, probably pays to do that.

I also have a large number of scanned negatives that are 300MB tiffs at 4800dpi.  Obviously they're slow.  The faces do detect though.  But perhaps that screws up the recognition system, I don't know.

Sadly, in my relatively new, but fairly exhausting usage (exhausting as in time spent this weekend), face detection really isn't a working feature.  Hopefully we can get it fixed.
Comment 11 chimney@thespicers.net 2024-02-08 11:29:59 UTC
I can confirm this behaviour with the 04/02/2024 10:33 8.3.0 snapshot on Windows 11. Face *detection* across ~13,000 photos works well. However, after I manually identify ~14 new people from Unknown and assign faces to them (at least 5 faces for 8 of the people), the recognition step proceeds to incorrectly recognise hundreds of faces as just one person. The vast majority of these are completely different people that don't resemble the assigned person at all. I have not seen it ever assign faces to any other person.