Bug 437896 - Face Recognition with Time taken into consideration
Summary: Face Recognition with Time taken into consideration
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Recognition (show other bugs)
Version: 7.3.0
Platform: Other Other
: NOR wishlist
Target Milestone: ---
Assignee: Michael Miller
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-05-31 10:20 UTC by Martin
Modified: 2024-10-20 16:26 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.5.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin 2021-05-31 10:20:44 UTC
First of all, it is a great software you have programmed. Thank you!

I have a lot of Photos in my database spanning a large period of time. And as we are all no Hollywood stars, our look changes with time.
Would it be possible to cluster the Face-Recognition Data with time ?

e.g. If a large time period is specified, take 10y for a face. If it's not the first years of life, this should improve Recognition a lot. Otherwise I would have to use different "Persons" in the database e.g. Gandalf 1..2y, Gandalf 3..5y Gandalf 6..12, and so on.

Also I would really like to see "second best match". When I do not confirm a face from a suggestion, I would like that to be calculated to match another face WITHOUT the one I disconfirmed. Would that be possible ?

Thank you.
Comment 1 Thilo 2021-10-30 08:01:24 UTC
Hi Martin, (In reply to Martin from comment #0)
> First of all, it is a great software you have programmed. Thank you!
> 
> I have a lot of Photos in my database spanning a large period of time. And
> as we are all no Hollywood stars, our look changes with time.
> Would it be possible to cluster the Face-Recognition Data with time ?
> 
> e.g. If a large time period is specified, take 10y for a face. If it's not
> the first years of life, this should improve Recognition a lot. Otherwise I
> would have to use different "Persons" in the database e.g. Gandalf 1..2y,
> Gandalf 3..5y Gandalf 6..12, and so on.
> 
> Also I would really like to see "second best match". When I do not confirm a
> face from a suggestion, I would like that to be calculated to match another
> face WITHOUT the one I disconfirmed. Would that be possible ?
> 
> Thank you.

Hi Martin!

I also thought of this problem of growing children, there is some research on this (A Review of Face Recognition against Longitudinal Child
Faces; Sodomsky) and I like your proposition. Since most parents really take a lot of pictures, data should be sufficient to train "yearly" models of childrens persons. 
I do not know, if digikam is able to internally maintain a 1:n relationship between person labels and face recognition models, but I think this would be a senseful improvement. 
An alternative would be to have a hiearchy where you have a personlabel and "under" it some person-age labels. there is a recognition model for person-age which is individually trained and when it is recognized the face is automatically labeled person and person-age.

However, the recognition experts here can maybe raise their opinion if change like the proposed would improve the recognition performance for children
Comment 2 Maik Qualmann 2022-01-29 10:44:00 UTC
*** Bug 449310 has been marked as a duplicate of this bug. ***
Comment 3 caulier.gilles 2024-10-08 08:33:33 UTC
Hi Michael,

A possible Recognition settings to considerate for the future, with the new SFace DNN model.

Best

Gilles
Comment 4 Michael Miller 2024-10-08 13:24:50 UTC
The new SFace model handles this use case much better than OpenFace.  The matching algorithm is 1-to-1.  In other words, each unknow face evaluated against known faces.  There isn't a "blended" score for each person based on all the faces.  Each face has its own unique score.  Assuming you have images from different points in time, the algorithm will work like this:

Tag an image of newborn child.  The system will match other pictures of the same newborn child.  In 6 months, you add new pictures.  Because the pictures at 6 months aren't significantly different than newborn, the face engine will probably match the images with the newborn.

Assuming you have images from when the child at 1 years old, this will match more closely with the 6 months pictures, but less with the newborn pictures.  This is ok because the newborn and 6 month pictures are grouped together in the database (this is called clustering).

Imagine the process repeating as the child ages. Year 2 pictures will match with year 1.  Year 3 will match with year 2.  Year 4 will match with year 3.

The process also works exactly the same in reverse.  If you have a current picture, it will find slightly older pictures.  The slightly older pictures will in turn match with even older pictures.

This is how it works now, and it works well.  I did this 2 days ago as I'm developing the new SFace model recognition engine.  It did take several scans to find almost all the pictures of 13 year-old son, but it did find them. I only had to tag 8 or 9 images by hand and digiKam found over 1,700 other matches. I did have to run "recognize faces" several times and accept the suggestions, but it was much easier than looking for the faces manually.

Please try this approach when 8.5.0 is released.  I will keep this ticket open for a while after 8.5.0 is released and wait for your feedback.

Cheers,
Mike
Comment 5 Michael Miller 2024-10-20 16:04:43 UTC
My testing shows this can probably be closed.

Cheers,
Mike
Comment 6 caulier.gilles 2024-10-20 16:06:26 UTC
ok i close this one and the duplicate 449310
Comment 7 caulier.gilles 2024-10-20 16:09:52 UTC
Michael,

I reopen 449310, as the topic sounds like different.

Gilles
Comment 8 Michael Miller 2024-10-20 16:26:05 UTC
(In reply to caulier.gilles from comment #7)
> Michael,
> 
> I reopen 449310, as the topic sounds like different.
> 
> Gilles

Yes, I agree.  They are different suggestions.

Cheers,
Mike