Created attachment 145005 [details] Face detection screenshot SUMMARY When using Sony ARW files from a Sony Alpha 6000 mixed with JPG files of various sources and using face detection the JPG files show the expected outline after detection but ARW files show an elongated rectangle including the neck and sometimes chest but missing part of the face. This also seems to poison face recognition training resluts (unconfirmed) but shows a seemingly valid face area in the recognition confirmation tool window. STEPS TO REPRODUCE 1. Import JPG and ARW files from multiple Folders 2. Select YOLO v3 and 90% recognition 3. Open any ARW file with faces 4. Right click -> select detect faces OBSERVED RESULT Rectangles do not fit faces. https://imgur.com/a/KK1QfRB EXPECTED RESULT Rectangles should fit faces like for JPEG SOFTWARE/OS VERSIONS Linux/KDE Plasma: not using plasma (XFCE 4.16)
Hi, with the same image in JPEG format, the face areas match the right regions ? Best regards and happy new year Gilles Caulier
The problem will certainly be related to the fact that the size of the JPG preview image in the ARW differs greatly from the original RAW. Sony previews are usually much smaller. But we actually have a detection in digiKam and then use half the RAW preview loading. However, this will probably lead to the error here horizontally in the case of very large images. Can you send me the image on my private mail? Maik
(In reply to caulier.gilles from comment #1) > Hi, with the same image in JPEG format, the face areas match the right regions ? > Best regards and happy new year > Gilles Caulier Yes, i have tested this. Here are screenshots of an ARW and a JPG created from the same ARW with Darktable: https://imgur.com/a/LbQSN8e
(In reply to Maik Qualmann from comment #2) > Can you send me the image on my private mail? Yes, certainly. I have sent you ARW + JPG + XMP files to your address.
Looks like image on the top is extended a little bit on height compared to the bottom one. Right ? This can explain the dysfunction. digiKam store the face area coordinates in relative to the image sizes. Like this, if you resize image, the areas still at the right places. So i suspect the geometry properties in image metadata not well handled or not homogeneous between JPEG and ARW. Can you show the exiftool output from the both files please ? Gilles Caulier
(In reply to caulier.gilles from comment #5) > Looks like image on the top is extended a little bit on height compared to > the bottom one. Right ? No, I made freehand screenshots and did not duplicate the area exactly. > So i suspect the geometry properties in image metadata not well handled or > not homogeneous between JPEG and ARW. > Can you show the exiftool output from the both files please ? arw: https://pastebin.com/X5x3sLg8 jpg: https://pastebin.com/uY3tqvwf br
Thanks for the sample image. Several problems come together in this image. Due to the 16mm focal length, we would actually have to carry out a lens correction first, as the RAW already differs considerably from the existing JPG. You look at the image in digiKam with the quick preview, if you would switch to the half RAW size preview, it would be better. The main problem is that the RAW is a 3:2 image, the embedded preview is 16:9. I haven't checked to see if there are multiple previews images in the RAW, but the first one digiKam extracts is 16:9. This results in the elongated face rectangles. Is there a setting for the size/format of the preview images for RAW in the camera? That will definitely be interesting to solve the problem. Maik
Git commit c3a56aff7d9cf1ad71992d1c3eb95faf8fa12224 by Maik Qualmann. Committed on 01/01/2022 at 21:06. Pushed by mqualmann into branch 'master'. get face regions from half RAW image limit the size for openCV to 2000px M +12 -2 core/utilities/facemanagement/threads/facepreviewloader.cpp https://invent.kde.org/graphics/digikam/commit/c3a56aff7d9cf1ad71992d1c3eb95faf8fa12224
Git commit cd062b0bebda159d7d583c46ffaa9fa497757e7f by Maik Qualmann. Committed on 02/01/2022 at 08:44. Pushed by mqualmann into branch 'master'. adjust preview to aspect ratio from RAW image FIXED-IN: 7.5.0 M +1 -1 NEWS M +5 -3 core/utilities/facemanagement/items/facegroup.cpp M +21 -1 core/utilities/facemanagement/items/facegroup_p.cpp M +2 -0 core/utilities/facemanagement/items/facegroup_p.h https://invent.kde.org/graphics/digikam/commit/cd062b0bebda159d7d583c46ffaa9fa497757e7f
(In reply to Maik Qualmann from comment #7) > The main problem is that the RAW is a 3:2 image, the embedded preview is > 16:9. I haven't checked to see if there are multiple previews images in the > RAW, but the first one digiKam extracts is 16:9. This results in the > elongated face rectangles. Is there a setting for the size/format of the > preview images for RAW in the camera? The camera was set to produce RAW (ARW, native at 3:2 i think) and JPEG at 16:9 format. It seems that it applies the JPEG format to the RAW embedded preview (including in-camera processing). I think i can only set the image format (applying to the JPEG) bot not the preview format separately. I have changed the JPEG format to 3:2 now because I assume it is a straight crop from the RAW. Maybe there is an option to re-generate the previews including lens correction.
Git commit e8e3eeb5497d0df35c7cbed1fc57f125d5621f10 by Maik Qualmann. Committed on 03/01/2022 at 18:15. Pushed by mqualmann into branch 'master'. disable face region to aspect ratio at moment M +4 -0 core/utilities/facemanagement/items/facegroup_p.cpp https://invent.kde.org/graphics/digikam/commit/e8e3eeb5497d0df35c7cbed1fc57f125d5621f10
Thank you all for your work! What is the Plan for this in the future?
I plan to fix it. You should set your preview to "completely" and half RAW preview in the digiKam settings as a workaround with digiKam-7.5.0. At Sony cameras, the embedded preview image is very small anyway and makes a pretty blurred preview. Maik
Is there any update on this?
@Igor, I think not yet, at least reported to this file. Anyway, digiKam 8.0.0 is out. This entry still valid with this release ? Best regards Gilles Caulier
(In reply to caulier.gilles from comment #15) > Anyway, digiKam 8.0.0 is out. This entry still valid with this release ? @caulier.gilles I'd have to check, is there a reason that it might be fixed in the 8.x release?
@Igor, What's about this file using current 8.2.0 AppImage Linux bundle ? It's reproducible ? https://files.kde.org/digikam/ Thanks in advance Gilles Caulier
(In reply to caulier.gilles from comment #17) > @Igor, What's about this file using current 8.2.0 AppImage Linux bundle ? It's > reproducible ? @caulier I don't have an active Setup where I can check this, but I can take the original file and try it out. Currently very busy but I will try to make time to give it a go.
@caulier Just wanted to update. Unfortunately still did not test this, I am sorry. I have not forgotten (keep it in my to do) but real life responsibilities keep me busy.
Igor, Any feedback in this room please ? Best Gilles Caulier
I shoot Sony (a7R III). The new facial detection and recognition AI models in 8.5.0 are working perfectly for me. Cheers, Mike
@gilles.caulier I did some testing right now: * installed Digikam 8.4.0 from flatpak today * left all install options on default * downloaded the same test files i sent you * selected yolo v3 * did face discovery and recognition Unfortunately it is still the same as before. The JPEG works fine but the ARW has misplaced detection rectangles. Comparison: https://imgur.com/GFYJZGK
@Michael, the problem here is a little different, the camera takes a preview image in 16:9 (cropped), the RAW is 3:2. Therefore the face rectangles don't match. @Igor, if you set the digiKam view to full with half RAW view, it would work for you. Maik
> @Igor, if you set the digiKam view to full with half RAW view, it would work for you. @Maik I think we already established that it would work like that as a workaround. What it it that you would like me to test? Are we expecting a regression on that?
The problem is that we would always have to look at the aspect ratio of the preview image and the RAW image and then convert the coordinates according to the current view mode. I had already tried this, but the result is actually not satisfactory either, the faces still do not fit pixel-perfectly (visible deviation). Because the camera probably applies the lens correction to the RAW, perhaps with cropping, and then generates the cropped 16:9 preview image. We would also have to recreate this internal camera processing. In my opinion, if possible, you should not set the preview image in the camera to 16:9. Maik
@Maik This precise issue is mostly resolved for me with the workaround. Maybe Digikam should give a one-time warning to raw format users if the preview image ratio does not match the raw image ratio?
Gilles, I shoot Sony (a7R III). Face regions are detected correctly in my RAW .ARW files. I have verified over 300 files. Igor, what model of Sony do you have? Cheers, Mike
Hi Michael, I have the test sample, it's a Sony ILCE-6000. The camera is set to take 16:9 images. Therefore, the embedded preview is 16:9, but the RAW image is 3:2. This creates the problem that depending on the view option, we see the 16:9 image or the RAW image in digiKam. Therefore, the face regions cannot always be correct, since the 16:9 preview image is a cropped version of the RAW image. We always used to do face detection on the preview image, but now we do it on the RAW image. At least there are no problems if RAW preview is set in digiKam. With the embedded preview view, we then of course have the problem again. Maik
Created attachment 179698 [details] Thumbnail image preview of face
Created attachment 179699 [details] Preview image bounding box of face
I'm having a similar issue and have attached two images to this issue. Digikam 8.6.0 Windows 10 Sony ILCE-6700 ARW files When I look at the preview the bounding box for the face detection is mostly accurate, however in the thumbnail window it is shifted significantly. I'm assuming the model training/fit data is based off the bounding image seen in the preview image and not the thumbnail image. I want to make sure I don't poison the training data.
The problem is that in the new face engine code, we load the thumbnail from the preview image. @Michael, even if it takes time, we have to use the full image for the thumbnails, or for RAW, use the half RAW loading function. Maik
(In reply to Curtis Barnard from comment #31) > I'm having a similar issue and have attached two images to this issue. > > Digikam 8.6.0 > Windows 10 > Sony ILCE-6700 > ARW files > > When I look at the preview the bounding box for the face detection is mostly > accurate, however in the thumbnail window it is shifted significantly. I'm > assuming the model training/fit data is based off the bounding image seen in > the preview image and not the thumbnail image. I want to make sure I don't > poison the training data. Hi Curtis, Would you mind attaching at full image to this ticket so I can test with it? I also shoot Sony (a7RIII - ILCE-7RM3), and I don't have this issue. It would help with fixing this issue. Cheers, Mike
Hi Michael, I can send you the sample image this evening in my time zone. Or set the camera to 16:9. You'll then have a 16:9 preview image, but the RAW will still be 3:2. Maik
(In reply to Maik Qualmann from comment #34) > Hi Michael, I can send you the sample image this evening in my time zone. Or > set the camera to 16:9. You'll then have a 16:9 preview image, but the RAW > will still be 3:2. > > Maik Hi Maik, Danke! I'll take some shots today to test. Cheers, Mike
Git commit 15ad7b947d73f13c1422a1843b9a67179ef72765 by Michael Miller. Committed on 26/03/2025 at 00:35. Pushed by michmill into branch 'master'. Use Raw or HalfRaw for Sony Raw Images. M +14 -2 core/utilities/facemanagement/pipelines/detectrecognize/facepipelinedetectrecognize.cpp https://invent.kde.org/graphics/digikam/-/commit/15ad7b947d73f13c1422a1843b9a67179ef72765
I'm happy to email one of my photos to someone, but would prefer not to attached it here. I checked and I'm in 3:2 shooting mode, not 16:9. Thanks for working on this!
(In reply to Curtis Barnard from comment #37) > I'm happy to email one of my photos to someone, but would prefer not to > attached it here. I checked and I'm in 3:2 shooting mode, not 16:9. Thanks > for working on this! Thank you. Let's see if the change I made yesterday fixes the issue. It should be in the build tomorrow. If not, then yes, please send me a sample image. My email can be found in the digiKam credits and documentation. Cheers, Mike