Bug 501257 - People are not recognized
Summary: People are not recognized
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Recognition (show other bugs)
Version: 8.6.0
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-03-09 10:11 UTC by Andy
Modified: 2025-03-12 14:58 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.6.0
Sentry Crash Report:


Attachments
Picture1 (883.40 KB, image/png)
2025-03-10 10:58 UTC, Andy
Details
20250312_144600 (216.30 KB, application/pdf)
2025-03-12 14:01 UTC, Andy
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andy 2025-03-09 10:11:01 UTC
Hello,

I have a series of photos with the same 11 people. Created in a matter of seconds.
In most of the photos I have confirmed all the faces. In some of them only a few faces have already been confirmed. If I now right-click on the image and click "Recognize faces", nothing happens. 0 found.
The people really do look identically into the camera.

What could be the reason that actually known faces cannot be assigned and remain unknown? Insufficient training data? In the settings, both values ​​are set to 7.

I have also manually assigned the same people without names in many other group photos.

I'm really sorry for being so annoying.

All the best
Andy
Comment 1 caulier.gilles 2025-03-09 11:56:40 UTC
It's the same context, eg. using a NAS to store your collection | database ?
Comment 2 Andy 2025-03-09 12:10:57 UTC
MariaDB Ver 11.4 in Synology Docker.
Comment 3 Maik Qualmann 2025-03-10 07:20:23 UTC
It is possible that the confirmed images are of too poor quality (blurry, too small, etc.), these are now filtered out and not added to the training.

Maik
Comment 4 caulier.gilles 2025-03-10 07:54:26 UTC
yes, It's a possible side effect of the filter add to 8.6.0 by Michael.

Andy, can you reproduce this dysfunction with 8.5.0 where the filter must be not present if i remember? I recommend to create a test database and do not use the production one created with 8.6.0.

Gilles Caulier
Comment 5 Andy 2025-03-10 08:30:18 UTC
Hi!
I can't download 8.5.0. Nothing happens. I've tried three browsers.
https://www.digikam.org/download/
Comment 6 Andy 2025-03-10 08:41:59 UTC
Forget it. The blue button at the top right just links to the page. Phew.
Comment 7 Michael Miller 2025-03-10 10:34:53 UTC
Hi Andy,
Maik and Gilles are correct.  I introduced a Face Image Quality Assessment (FIQA) in 8.6.0.  FIQA filters out small, blurry, and noisy faces from being used for face recognition because the quality of the information extracted from the face is too poor, and would cause incorrect matches (false positives).

What are the dimensions of the images with the people?  Are all 11 people in every image (are they group photos)? The face thumbnail needs to be at least 37 pixels on each side.  

Cheers,
Mike
Comment 8 Andy 2025-03-10 10:58:54 UTC
Created attachment 179259 [details]
Picture1

Hi!
I have a screenshot here. The size should actually be sufficient.
There are several almost identical versions of this photo. I have assigned the faces to almost all of them. I am now using two of them for testing (right click -> recognize faces).

I have installed 8.5.0. The training data is currently being rebuilt.
Regards
Andy
Comment 9 Andy 2025-03-10 12:45:41 UTC
Hi!
I removed all the faces in one of the pictures in question and added the frames again. Then clicked "Recognize faces". Some of the faces were recognized correctly. 2 or 3 others were unknown.
I started "Recognize faces" several times without the remaining faces being recognized. Then I confirmed one of the recognized faces and pressed "Recognize faces" again. Now one of the unknown faces was also assigned a name. When I confirmed one again and pressed "Recognize faces", another face was correctly suggested.
Is there a limit for face recognition per photo?

Regards
Andy
Comment 10 Michael Miller 2025-03-10 12:47:12 UTC
Hi Andy,
Can you upload the original image?  I need the original, and without the face rects detected by digiKam.  The sample image faces are definitely being rejected by FIQA because the face thumbnails are too small.  I'd like to try the test on the original image.

Cheers,
Mike
Comment 11 Michael Miller 2025-03-10 12:51:22 UTC
(In reply to Andy from comment #9)
> Hi!
> I removed all the faces in one of the pictures in question and added the
> frames again. Then clicked "Recognize faces". Some of the faces were
> recognized correctly. 2 or 3 others were unknown.
> I started "Recognize faces" several times without the remaining faces being
> recognized. Then I confirmed one of the recognized faces and pressed
> "Recognize faces" again. Now one of the unknown faces was also assigned a
> name. When I confirmed one again and pressed "Recognize faces", another face
> was correctly suggested.
> Is there a limit for face recognition per photo?
> 
> Regards
> Andy

Hi Andy,
No, there isn't a limit to the number of faces recognized in each photo.  In version 8.5.0, we use the face for recognition if the AI model can extract the face features.  Sometimes the AI can't extract the features, and so the face isn't used for training or matching.  Sometimes because of thumbnail image quality factors the extracted features are so pool that they don't match anything.

Cheers,
Mike
Comment 12 Andy 2025-03-10 13:11:22 UTC
OK, then it was a coincidence.
I clicked many more times. The name does disappear and reappear. Often the face is not even recognized as a face.
I can install 8.6.0 again and test it.

Regards
Andy
Comment 13 Michael Miller 2025-03-10 13:14:15 UTC
(In reply to Andy from comment #12)
> OK, then it was a coincidence.
> I clicked many more times. The name does disappear and reappear. Often the
> face is not even recognized as a face.
> I can install 8.6.0 again and test it.
> 
> Regards
> Andy

Hi Andy,
This makes sense.  There are several face detection improvements in 8.6.0, so I can understand if 8.5.0 doesn't even detect the face.

Cheers,
Mike
Comment 14 Andy 2025-03-10 13:48:43 UTC
Done. 8.6.0 is installed again.
The database has been updated.
Opened a photo and removed all faces. Searched for faces. When the search is finished, names are suggested. There are two faces that remain unknown. Start "Recognize faces" again and these two also have the correct name.
One face is not recognized.

Regards
Andy
Comment 15 Andy 2025-03-10 13:49:30 UTC
So, a face is not recognized as a face.
Comment 16 Michael Miller 2025-03-10 13:54:50 UTC
Hi Andy,
Can you please attach the original image so I can run it through FIQA to see if this is what's causing the faces to not be recognized?

Cheers,
Mike
Comment 17 Andy 2025-03-10 14:19:58 UTC
Here: https://Zunar.synology.me:55241/d/s/woCCEnd9K44ORfBcsVVyh9Otce6pWnX6/we6r3ACohVnbxl3P-tF6CWKMD94gHFD7-Yr8gpCo7HQw

The woman in the back row in the middle is not recognized as a face.

Regards
Andy
Comment 18 Michael Miller 2025-03-10 14:31:43 UTC
(In reply to Andy from comment #17)
> Here:
> https://Zunar.synology.me:55241/d/s/woCCEnd9K44ORfBcsVVyh9Otce6pWnX6/
> we6r3ACohVnbxl3P-tF6CWKMD94gHFD7-Yr8gpCo7HQw
> 
> The woman in the back row in the middle is not recognized as a face.
> 
> Regards
> Andy

Hi Andy,
Thank you.  Can you send me one of the other shots, too, so I can run recognition between the 2 imges?

Also, the woman in the back row is just barely below the face detection threshold when the settings are 7 and large faces.  If you change the detection threshold to 5, OR if you set the "Face Size" to medium, then her face is detected.  Either setting change is enough to find her face. There is a section in the digiKam documentation that describes changing the detection settings for group photos.

Cheers,
Mike
Comment 19 Andy 2025-03-10 14:42:12 UTC
https://Zunar.synology.me:55241/d/s/12TzKM68zliQYnaGlfDPZNSOrQ7BHhG8/kSLYcTo6fv_6EvyQAiEs83jshyXE8XT8-jbGgaoM9HQw

Hello Mike,
here are a few more examples.

I have set "Extra Small" in the settings. I'll look at the documentation.
Have you been able to determine why for some people the face recognition only happens after a second run?

Regards
Andy
Comment 20 Michael Miller 2025-03-10 16:01:55 UTC
(In reply to Andy from comment #19)
> https://Zunar.synology.me:55241/d/s/12TzKM68zliQYnaGlfDPZNSOrQ7BHhG8/
> kSLYcTo6fv_6EvyQAiEs83jshyXE8XT8-jbGgaoM9HQw
> 
> Hello Mike,
> here are a few more examples.
> 
> I have set "Extra Small" in the settings. I'll look at the documentation.
> Have you been able to determine why for some people the face recognition
> only happens after a second run?
> 
> Regards
> Andy

Hi Andy,
Thank you.  I'll test all the images today.

As for why it only recognizes faces on the second run, I'm not entirey sure.  I've seen it happen a few times, but I've neer been able to figure out exactly why it's happening.  The thumbnail passed to the recognition engine is exactly te same.  I'll take another look at it for 8.7.0.

Also, you gave me a good idea for another change for 8.7.0.  Instead of "Face Size" of X-Small, Small, Medium, Large, and X-Large; I think I'm going to simplify things and only have 3 options of "Potrait, Normal, and Group Photo".  The X-Small through X-Large options were introduced in 8.5.0.  With the additional face detection optimizations in 8.6.0, I don't think we need 5 choices anymore, and the 3 new options I just listed should cover almost all the use cases.  Since you use facial recongnition regularly, what do you think of the idea?

Cheers,
Mike
Comment 21 Andy 2025-03-10 20:54:12 UTC
Hello Mike,

I'd be happy to help. :-) I'm happy to test it.

Maybe some faces aren't ready for face recognition so soon after being found. Maybe there's a delay when writing and reading in the database.

It would be interesting if "group photo" was automatically recognized based on the number of X faces in the photo. I can only set one option in the settings and run through it. XS has been a good compromise for everything for me so far.

Regards
Andy
Comment 22 Michael Miller 2025-03-11 17:28:09 UTC
Hi Andy,
Can you explain something please?  How did you get the superpower to find bugs in code that has been working well for months.  :)

You found another well-hidden bug in digiKam.

Maik/Gilles,
The issue is the thumbnail QImage being returned from the thumbnail service is has a format of QImage::Format_RGB32. If I raw convert the QImage to an OpenCV cv::Mat, I should get a RGB cv::Mat image. What I get is actually BGR. YuNet and SFace want RGB format images, and not BGR.  

I don't know if this is adysfunction in the thumbnail service, but in other places when I convert a QImage to cv::Mat the color coding is correct. 

If I convert the thumbnail QImage to BGR, the result thumbnail is actually RGB, and is processed correctly.

Andy, I also found out why you have run recognize twice.  I'm fixing that, too.  There should be a new build in a few days.

Cheers,
Mike
Comment 23 Bug Janitor Service 2025-03-12 02:33:18 UTC
A possibly relevant merge request was started @ https://invent.kde.org/graphics/digikam/-/merge_requests/351
Comment 24 Michael Miller 2025-03-12 02:54:23 UTC
Git commit 61f0fd3a45fd3fb132adf036db12253601079eca by Michael Miller.
Committed on 12/03/2025 at 02:54.
Pushed by michmill into branch 'master'.

fix 501270

M  +1    -1    NEWS
M  +57   -11   core/utilities/facemanagement/pipelines/detectrecognize/facepipelinedetectrecognize.cpp
M  +2    -2    core/utilities/facemanagement/pipelines/facepipelinebase.cpp

https://invent.kde.org/graphics/digikam/-/commit/61f0fd3a45fd3fb132adf036db12253601079eca
Comment 25 Michael Miller 2025-03-12 13:03:32 UTC
Hi Andy,
I'm pretty sure I fixed the issues.  The new build of digiKam should have the changes.  Please download and test the latest build.

One important note: After you install the latest build, you MUST rebuild all face training by going to Tools->Maintenance->Detect and recognize faces->Rebuild all training data.  After you rebuild the training data, you should be able to test.

Let me know how it goes.  Thank you for everything. You're one of the best software testers I think I've met.

Cheers,
Mike
Comment 26 Andy 2025-03-12 14:01:42 UTC
Created attachment 179343 [details]
20250312_144600

Hello Mike,

Thank you for the great praise! But you all are doing an excellent job too. :-)

I've recreated the training data and tested the recognition.
Finding faces and the subsequent face recognition now work much better. The woman in the back center isn't always recognized, but it's more often now than before. It varies depending on the photo.
The names are also being assigned to the people better. But there are still some where this doesn't happen. Only a further person recognition in the photo reveals the name.

See attachment. At the top, only "Find faces." At the bottom, when I click "Recognize faces" again. The woman in the foreground.
In other photos, it may be a different person.

Regards
Andy
Comment 27 Michael Miller 2025-03-12 14:33:30 UTC
(In reply to Andy from comment #26)
> Created attachment 179343 [details]
> 20250312_144600
> 
> Hello Mike,
> 
> Thank you for the great praise! But you all are doing an excellent job too.
> :-)
> 
> I've recreated the training data and tested the recognition.
> Finding faces and the subsequent face recognition now work much better. The
> woman in the back center isn't always recognized, but it's more often now
> than before. It varies depending on the photo.
> The names are also being assigned to the people better. But there are still
> some where this doesn't happen. Only a further person recognition in the
> photo reveals the name.
> 
> See attachment. At the top, only "Find faces." At the bottom, when I click
> "Recognize faces" again. The woman in the foreground.
> In other photos, it may be a different person.
> 
> Regards
> Andy

Hi Andy,
I'm happy to hear it's working better, but I understand it's not perfect yet.  We are only a few days away from the official 8.6.0 release, so I don't want to make any more changes.  I'll look into the problem more for version 8.7.0.

Would you agree that face engine 8.6.0 is an improvement over 6.5.0?

Cheers,
Mike
Comment 28 Andy 2025-03-12 14:58:58 UTC
(In reply to Michael Miller from comment #27)
> Would you agree that face engine 8.6.0 is an improvement over 6.5.0?


Absolutely! A great success!

Regards
Andy