*** Bug 432412 has been marked as a duplicate of this bug. ***

*** Bug 468629 has been marked as a duplicate of this bug. ***

I have some suggestions for improvements that encompass this problem.

On a typical set of images the ability to accurately recognise faces declines as more images are added. The main reason for this is that poor quality images pollute the reference data that the software uses for the recognition of faces. Recognised faces that are blurred, out of focus, wearing sunglasses or low resolution contain information that is too similar to other faces in the pool of faces to be recognised.

Selective Training Data
One of the best methods to improve the rate of face recognition is to improve the training data. Instead of having just two categories for faces: Confirmed and Unconfirmed, an additional category could be added, Reference. In this case only pictures of a face that are in the Reference category would be used in the recognition training data. Pictures could be categorised into the Reference category either manually or automatically. By the manual method good quality faces across a range of ages and situations (ie wearing glasses) could be tagged to be used in the Reference training data.

Obviously a large number of reference faces would still be required for the training data, but in a large album of pictures that won’t be a problem. For example I have person’s face appearing in over 14,000 images, however many of these are blurred, out of focus or low resolution. A subset of these images would still be suitable even if it only contained 10% of the images. In addition using a smaller training data set would improve the speed of clearing and rebuilding the training data.

The automatic method could use existing algorithms to exclude “Confirmed” face pictures that are blurred, low resolution or out of focus from the training data for that particular face. While an image in totality may not be blurred, low resolution or out of focus, but the section of the image that includes the face could be, and thus should be excluded from the training data.

Hi Michael,

Do you think that YuNet model accept a settings about the tolerance of blurred faces ?

Best

Gilles

YuNet is much more accurate at detecting a blurred face.  SFace is also more accurate at matching pixelated or blurred faces.  I believe the issue is significantly improved by moving to the new models.  

I addition to using the new models, I'm also exploring using a quality metric like eDifFIQA to limit the training data.  I think it's a good idea, too.  Further testing an analysis is needed on YuNet and SFace to determine if limiting the training data is required.

Cheers,
Mike

Adding facial detection and recognition is a large effort, but is a possibility.  First we could generate a list of jpegs from the key frames of a video, and run the existing face pipeline on the jpegs.  FFMpeg does this quite easily.   Then doing a first-pass detection and recognition of only the faces in extracted jpegs we could eliminate duplicate faces.  The resulting set of faces could then be tied to the video.  The challenge is displaying the face tag in the appropriate time-dimension of the video.  A human sometimes needs context to remember or recognize a face, so showing the tagged face in context in the video is a good idea.

As I said, it possible, but will require a large amount of work, particularly in the digiKam UI.

Cheers,
Mike

Hi Paul,
We've added a face image quality assessment (FIQA) in 8.6.0.  FIQA automatically filters out blurry, pixelated, and noisy images from being used for face training.

Cheers,
Mike