Bug 499208 - Add a new AI based tool to auto-crop image.
Summary: Add a new AI based tool to auto-crop image.
Status: REPORTED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-Bqm-WishForNewTools (show other bugs)
Version: 8.7.0
Platform: Microsoft Windows Microsoft Windows
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-01-27 18:02 UTC by Orontion
Modified: 2025-04-14 14:22 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Incorrectly processed image (305.20 KB, image/jpeg)
2025-01-27 18:02 UTC, Orontion
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Orontion 2025-01-27 18:02:23 UTC
Created attachment 177722 [details]
Incorrectly processed image

SUMMARY
With certain settings, "Lens Auto-Correction" feature does not crop image sides to properly fit processed image.

STEPS TO REPRODUCE
1. Open .NEF images 1 and 2 from https://drive.google.com/drive/folders/1V8xu7YMF4bZuh06omMzVC0qPW7y2Z0lT?usp=drive_link in Image Editor
2. Apply "Lens Auto-Correction" to each one

OBSERVED RESULT
- For image 1 function crops sides of corrected image properly
- For image 2 function does not crop sides of corrected image and fills it with artifacts (see attached file "screenshot 2")

EXPECTED RESULT
- Function automatically crops distorted borders for both images

SOFTWARE/OS VERSIONS
Windows: 10
Comment 1 Maik Qualmann 2025-01-27 20:23:02 UTC
I'll make the bug to a wish, actually we don't have an automatic crop function in the lens auto correction tool. I'm not sure now if Lensfun actually provides this as a feature. I'll check.

Maik
Comment 2 Maik Qualmann 2025-01-27 20:43:09 UTC
Also check the Advanced Metadata settings. There are now settings for the pick label. "xmp.digiKam.PickLabel" should be at the top of the list.

Maik
Comment 3 Maik Qualmann 2025-01-27 20:44:17 UTC
Sorry, Comment 2 is wrong here.

Maik
Comment 4 Orontion 2025-01-27 20:47:57 UTC
Just as opinion - lens auto-correction feature joesn't make much sense if it leaves artifacts on sides of image as a result. Also, these artifacts make auto-correction unusable during batch processing.
Comment 5 Maik Qualmann 2025-01-27 21:30:50 UTC
If black edges appear, it could be solved using our automatic cropping tool in BQM. Not with this image where the correction goes in the other direction. Darktable uses a scaling of 1.023 by default to make the edges disappear, this would be an option.

Maik
Comment 6 Orontion 2025-01-27 21:37:05 UTC
Yes, this would be acceptable solution to pair auto-correction with auto-crop, but unfortunately artifacts on sides are just lines with color of outermost pixels, so auto-crop has no chance to detect borders and remove them. I have never seen black borders in my scenarios.

Also, black borders will confuse auto-crop if processed images have significant dark/black areas, for example night time photos.
Comment 7 caulier.gilles 2025-04-11 23:23:43 UTC
Hi Michael, 

This is an entry where the current basic auto-crop tool (Edtor + BQM) can be replaced by an AI based tool...

Best

Gilles
Comment 8 Orontion 2025-04-12 08:10:18 UTC
(In reply to caulier.gilles from comment #7)
> Hi Michael, 
> 
> This is an entry where the current basic auto-crop tool (Edtor + BQM) can be
> replaced by an AI based tool...
> 
> Best
> 
> Gilles

Sorry, but why this super-basic functionality requires AI?.. This is literally just simplest geometry operation: crop image by innermost unchanged rectangle, what kind of AI is needed here?

And why original bug name has changed?
Comment 9 caulier.gilles 2025-04-12 13:45:20 UTC
Because this is the plan : add an AI based processor to fix images, not only crop, but perspective, rotation (already done), etc.

The original code to crop image is weird, and need to be rewritten. This is why the idea to use a DNN with right models (based on modules) sounds like the right way to go.

Michael Miller in this room can explain better than me the concept.

Best

Gilles Caulier
Comment 10 Michael Miller 2025-04-12 15:32:44 UTC
(In reply to caulier.gilles from comment #9)
> Because this is the plan : add an AI based processor to fix images, not only
> crop, but perspective, rotation (already done), etc.
> 
> The original code to crop image is weird, and need to be rewritten. This is
> why the idea to use a DNN with right models (based on modules) sounds like
> the right way to go.
> 
> Michael Miller in this room can explain better than me the concept.
> 
> Best
> 
> Gilles Caulier

Yes, Gilles is correct. The plan is to add more auto-tools to digiKam to correct and adjust multiple image parameters, including cropping.  In your scenario it is easy to auto-crop based on the innermost unchanged rectange, but that assumes the cropping was done in digiKam.  There are other use cases where the user may want to auto-crop an imported image that already has black or white edges.  In those cases an AI border detection model is needed.  It's better to use the same method for auto-cropping for all use cases instead of implementing multiple different methods.

Cheers,
Mike
Comment 11 Orontion 2025-04-12 15:38:04 UTC
Okay, sounds fair. Thank you for explanation!

I hope, this AI capabilities do not require internet connection and do not send user's data to third-party organizations for processing?
Comment 12 Michael Miller 2025-04-12 15:40:38 UTC
(In reply to Orontion from comment #11)
> Okay, sounds fair. Thank you for explanation!
> 
> I hope, this AI capabilities do not require internet connection and do not
> send user's data to third-party organizations for processing?

Hi Orontion,
The digiKam team takes your (and our) privacy seriously.  None of the AI models implemented in digiKam send your data to the cloud. All models are downloaded to your computer and run locally.  Your data is exactly that, yours.

Cheers,
Mike
Comment 13 Orontion 2025-04-14 14:22:08 UTC
Thank you for information! Glad to hear that!