| Summary: | In face detection, an option to suppress low quality faces as detected might improve recognition training and results | ||
|---|---|---|---|
| Product: | [Applications] digikam | Reporter: | gessel <gessel> |
| Component: | Faces-Detection | Assignee: | Digikam Developers <digikam-bugs-null> |
| Status: | RESOLVED FIXED | ||
| Severity: | wishlist | CC: | caulier.gilles, chrisc.gigamail, haselnuss87, metzpinguin, michael_miller |
| Priority: | NOR | ||
| Version First Reported In: | 8.1.0 | ||
| Target Milestone: | --- | ||
| Platform: | Ubuntu | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed/Implemented In: | 8.6.0 | |
| Sentry Crash Report: | |||
|
Description
gessel
2023-10-11 23:59:15 UTC
An alternative would be to scan the collection first without YoloV3 using the "standard" engine. Maik I'm giving that a try, cleared all unconformed and re-scanning for faces. I see that in some face tags where the metadata is polluted with low quality face data where I've confirmed (generally correctly) a face tag with low pixel counts/low contrast that using that data to identify more faces, even with 10's or 100's of high quality face tags yields a true positive rate scanning for additional tags of ~0% accuracy (1:1000 or so suggested). While this is clearly a user error, it could be of assistance to users when rebuilding the training data set to offer to clear low quality faces, either interactively or automatically based on similar criterion. Some faces are just a few dozen pixels square, noisy, blurry, or very low contrast. It would seem fairly plausible to: * Scan the entire collection for confirmed faces * Compute each confirmed face rectangle's total pixel count and compare to some threshold, offer to delete/auto delete/exclude from training any face rectangles below selected minimum threshold * pass the survivors to the image quality sorter algo and compute blur, noise, under/over exposure levels, offer to delete/auto delete/exclude from training any face rectangles below selected minimum threshold * flush recognition training data * rebuild recognition training data with good faces. * rescan collection with clean, high quality recognition data. This is independent of a sometimes necessary human guided task of ensuring face tags are not mixed up, which would also obviously confuse the algo. It would seem a useful non-destructive automating option would be to simply tag confirmed but low quality faces as not suitable for training, but that gets back to the original ask of not considering them in the first place, however in this mode I'm suggesting a new "maintenance" option for resetting/refreshing the face recognition engine. Thumbs up. Good write-up and suggestion. Michael, I think this settings exists now with your new implementation. Right ? Gilles Thank Gilles. Yes. Adding a quality measurement such as eDifFIQA(T) to determine if the face thumbnail is usable for recognition is an option I'm exploring to improve accuracy. My testing with SFace, however, is showing this may not be needed as the accuracy of SFace is orders of magnitude higher than the existing OpenFace model. I will continue to research how best to improve face detection and face recognition accuracy. Cheers, Mike References: https://ieeexplore.ieee.org/abstract/document/10468647 https://github.com/opencv/opencv_zoo/tree/main/models/face_image_quality_assessment_ediffiqa I'd suggest verifying that face detection and face recognition are well matched. The issue is that face detection recognizes "faces" or face-like features fairly aggressively, especially in low-light/low resolution/low contrast modes while face recognition, understandably and completely consistent with human recognition, needs some measure of clarity to reliably assign the detected face to some specific recognized, unique tag. Humans are the same: this isn't a problem with automated recognition but with our human expectations of it. Humans are great at detecting a face-like feature looking at them, even at oblique angles, low light, high noise situations but actually accurately recognizing the face is a whole 'nother story. With this algo, we're not so much concerned with "is there a face-like structure in the image" than "this (tag) is in the picture." That means a very conservative face detected but not conclusively recognized modality. We may be well served by a "face detected but not recognized or even recognizable" tool, but the current UI assumes that a detected face can be reliably recognized and so this should be true. -DAvid Hi gessel, We introduced a face image quality assessment check in the face training pipelines in 8.6.0. Blurry, noisy, and pixelated images will no longer be used for face training. Cheers, Mike That's awesome! Thank you!! I'l retrain my face library and very much look forward to more consistent results. |