In recent versions of Digikam, you have noted that you started using OpenCV for some tasks. This feature request is for a more advanced Generative Remove Object and Fill capability within the Image editor. There are some tools in the OpenCV toolkit for object removal and infill painting that may need to be combined with some other models you are currently using in the background for other AI tasks in Digikam. You now see this image editing capability onboard Android and Apple phone image editors as well as recent releases of Adobe Lightroom.
digiKam uses OpencV since more than 15 years at least...
Thanks Gilles. I wondered if there may be something for this remove object and fill functionality similar to what you added for the proposed Generative background blur in the Enhance menu?
Hi JBrauher, Yes, we can try to propose to Mistral AI to study the implementation of this plugin. We will see which solution it propose to solve the problem. Best regards Gilles Caulier
This is the response of Mistral AI about to atch the current HealingClone image editor tool to use the OpenCV::inpainting() and drop automatically an area from an image: // -------------------------------------------- Here’s a concise and clear translation of my last response for your Bugzilla entry, focusing on the technical integration of OpenCV-based inpainting as a plugin enhancement for digiKam: Proposal: Add OpenCV Inpainting as an Option to the HealingClone Plugin Context The existing HealingClone plugin in digiKam allows users to remove objects by cloning pixels from a source area. While functional, this method requires significant user interaction. This proposal suggests adding an OpenCV-based inpainting option to the plugin, enabling users to remove objects with a single mask selection, similar to the iPhone’s "Remove Object" feature. Implementation Plan 1. Add a UI Option Add a checkbox or radio button in the plugin’s UI (e.g., healingcloneui.ui) to toggle between: Cloning Mode (current behavior). Inpainting Mode (new OpenCV-based option). Example UI addition: <widget class="QCheckBox" name="useInpaintingCheckBox"> <property name="text"> <string>Use Inpainting (OpenCV)</string> </property> </widget> 2. Modify the Plugin Logic In healingclone.h, add a boolean flag: private: bool m_useInpainting; // Default: false (cloning mode) In the plugin’s slotApply (or equivalent), add a conditional branch: void HealingCloneTool::slotApply() { if (m_useInpainting) { applyInpainting(); } else { // Existing cloning logic } } 3. Implement applyInpainting Add this method to healingclone.cpp: #include <opencv2/photo.hpp> void HealingCloneTool::applyInpainting() { // Get the current image and mask (already managed by the plugin) QImage image = m_originalImage; QImage mask = m_maskImage; // Binary mask (0=keep, 255=remove) // Convert QImage to cv::Mat cv::Mat cvImage(image.height(), image.width(), CV_8UC3, (void*)image.bits(), image.bytesPerLine()); cv::Mat cvMask(mask.height(), mask.width(), CV_8UC1, (void*)mask.bits(), mask.bytesPerLine()); // Apply inpainting cv::Mat result; cv::inpaint(cvImage, cvMask, result, 3, cv::INPAINT_TELEA); // Radius=3, TELEA algorithm // Convert cv::Mat back to QImage QImage resultImage(result.data, result.cols, result.rows, result.step, QImage::Format_RGB888); m_previewImage = resultImage.rgbSwapped(); // Update preview // Emit the result emit signalImageReady(m_previewImage); } 4. Ensure Mask Compatibility Verify that m_maskImage is binary (0 or 255). If not, binarize it: for (int y = 0; y < mask.height(); ++y) { uchar* line = mask.scanLine(y); for (int x = 0; x < mask.width(); ++x) { line[x] = (line[x] > 0) ? 255 : 0; } } 5. Update CMakeLists.txt Ensure OpenCV is linked: find_package(OpenCV REQUIRED) target_link_libraries(dplugin_healingclone PRIVATE OpenCV::OpenCV) 6. Connect the UI Option In the plugin’s constructor, connect the checkbox to the flag: connect(ui->useInpaintingCheckBox, &QCheckBox::toggled, this, &HealingCloneTool::setUseInpainting); 7. Add a Slot for the Toggle In healingclone.cpp: void HealingCloneTool::setUseInpainting(bool enable) { m_useInpainting = enable; } User Workflow Open the HealingClone plugin. Check "Use Inpainting (OpenCV)". Draw a mask over the object to remove (as usual). Click "Apply": The object is removed using OpenCV inpainting. Advantages Reuses existing code: No need to modify the editor or mask management. Simple UI integration: Users can switch between cloning and inpainting. Performance: OpenCV’s inpainting is fast and requires no manual pixel selection. Considerations Quality: OpenCV inpainting is less advanced than AI-based tools but far simpler than manual cloning. Undo/Redo: Already handled by the plugin. Large images: For very high-resolution images, consider downsampling during preview. Next Steps Locate the UI file (healingcloneui.ui) and add the checkbox. Implement applyInpainting and test with sample images. Adjust the inpainting radius (e.g., 3–10) for optimal results. // ---------------------------------------------------- Gilles Caulier
Let me know if you run a demo of it and post to a video as you did with Generative blur background. I would be curious to see how the inpainting works and how well it does.
Just checking back on this to see if this may make it into 8.8 with the above code block suggestion from Mistral AI, etc.