Bug 508292 - Add AI based feature to remove object and Fill.
Summary: Add AI based feature to remove object and Fill.
Status: REPORTED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-Editor-WishForNewTools (other bugs)
Version First Reported In: 8.8.0
Platform: Microsoft Windows Microsoft Windows
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-08-15 14:02 UTC by jbrauher
Modified: 2025-09-14 19:04 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed/Implemented In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jbrauher 2025-08-15 14:02:14 UTC
In recent versions of Digikam, you have noted that you started using OpenCV for some tasks.  This feature request is for a more advanced Generative Remove Object and Fill capability within the Image editor.  There are some tools in the OpenCV toolkit for object removal and infill painting that may need to be combined with some other models you are currently using in the background for other AI tasks in Digikam.  

You now see this image editing capability onboard Android and Apple phone image editors as well as recent releases of Adobe Lightroom.
Comment 1 caulier.gilles 2025-08-15 14:23:00 UTC
digiKam uses OpencV since more than 15 years at least...
Comment 2 jbrauher 2025-08-26 15:53:22 UTC
Thanks Gilles.  I wondered if there may be something for this remove object and fill functionality similar to what you added for the proposed Generative background blur in the Enhance menu?
Comment 3 caulier.gilles 2025-08-27 05:21:23 UTC
Hi JBrauher,

Yes, we can try to propose to Mistral AI to study the implementation of this plugin. We will see which solution it propose to solve the problem.

Best regards

Gilles Caulier
Comment 4 caulier.gilles 2025-08-27 05:40:17 UTC
This is the response of Mistral AI about to atch the current HealingClone image editor tool to use the OpenCV::inpainting() and drop automatically an area from an image:

// --------------------------------------------

Here’s a concise and clear translation of my last response for your Bugzilla entry, focusing on the technical integration of OpenCV-based inpainting as a plugin enhancement for digiKam:

Proposal: Add OpenCV Inpainting as an Option to the HealingClone Plugin
Context
The existing HealingClone plugin in digiKam allows users to remove objects by cloning pixels from a source area. While functional, this method requires significant user interaction. This proposal suggests adding an OpenCV-based inpainting option to the plugin, enabling users to remove objects with a single mask selection, similar to the iPhone’s "Remove Object" feature.

Implementation Plan
1. Add a UI Option

Add a checkbox or radio button in the plugin’s UI (e.g., healingcloneui.ui) to toggle between:

Cloning Mode (current behavior).
Inpainting Mode (new OpenCV-based option).

Example UI addition:
<widget class="QCheckBox" name="useInpaintingCheckBox">
    <property name="text">
        <string>Use Inpainting (OpenCV)</string>
    </property>
</widget>

2. Modify the Plugin Logic

In healingclone.h, add a boolean flag:
private:
    bool m_useInpainting; // Default: false (cloning mode)

In the plugin’s slotApply (or equivalent), add a conditional branch:
void HealingCloneTool::slotApply()
{
    if (m_useInpainting) {
        applyInpainting();
    } else {
        // Existing cloning logic
    }
}

3. Implement applyInpainting

Add this method to healingclone.cpp:
#include <opencv2/photo.hpp>

void HealingCloneTool::applyInpainting()
{
    // Get the current image and mask (already managed by the plugin)
    QImage image = m_originalImage;
    QImage mask  = m_maskImage; // Binary mask (0=keep, 255=remove)

    // Convert QImage to cv::Mat
    cv::Mat cvImage(image.height(), image.width(), CV_8UC3, (void*)image.bits(), image.bytesPerLine());
    cv::Mat cvMask(mask.height(), mask.width(), CV_8UC1, (void*)mask.bits(), mask.bytesPerLine());

    // Apply inpainting
    cv::Mat result;
    cv::inpaint(cvImage, cvMask, result, 3, cv::INPAINT_TELEA); // Radius=3, TELEA algorithm

    // Convert cv::Mat back to QImage
    QImage resultImage(result.data, result.cols, result.rows, result.step, QImage::Format_RGB888);
    m_previewImage = resultImage.rgbSwapped(); // Update preview

    // Emit the result
    emit signalImageReady(m_previewImage);
}

4. Ensure Mask Compatibility

Verify that m_maskImage is binary (0 or 255). If not, binarize it:
for (int y = 0; y < mask.height(); ++y) {
    uchar* line = mask.scanLine(y);
    for (int x = 0; x < mask.width(); ++x) {
        line[x] = (line[x] > 0) ? 255 : 0;
    }
}

5. Update CMakeLists.txt

Ensure OpenCV is linked:
find_package(OpenCV REQUIRED)
target_link_libraries(dplugin_healingclone PRIVATE OpenCV::OpenCV)

6. Connect the UI Option

In the plugin’s constructor, connect the checkbox to the flag:
connect(ui->useInpaintingCheckBox, &QCheckBox::toggled, this, &HealingCloneTool::setUseInpainting);

7. Add a Slot for the Toggle

In healingclone.cpp:
void HealingCloneTool::setUseInpainting(bool enable)
{
    m_useInpainting = enable;
}

User Workflow

Open the HealingClone plugin.
Check "Use Inpainting (OpenCV)".
Draw a mask over the object to remove (as usual).
Click "Apply": The object is removed using OpenCV inpainting.

Advantages

Reuses existing code: No need to modify the editor or mask management.
Simple UI integration: Users can switch between cloning and inpainting.
Performance: OpenCV’s inpainting is fast and requires no manual pixel selection.

Considerations

Quality: OpenCV inpainting is less advanced than AI-based tools but far simpler than manual cloning.
Undo/Redo: Already handled by the plugin.
Large images: For very high-resolution images, consider downsampling during preview.

Next Steps

Locate the UI file (healingcloneui.ui) and add the checkbox.
Implement applyInpainting and test with sample images.
Adjust the inpainting radius (e.g., 3–10) for optimal results.

// ----------------------------------------------------

Gilles Caulier
Comment 5 jbrauher 2025-08-28 17:09:26 UTC
Let me know if you run a demo of it and post to a video as you did with Generative blur background.  I would be curious to see how the inpainting works and how well it does.
Comment 6 jbrauher 2025-09-14 19:04:57 UTC
Just checking back on this to see if this may make it into 8.8 with the above code block suggestion from Mistral AI, etc.