Bug 497565 - Using OpenVINO backend in OpenCV
Summary: Using OpenVINO backend in OpenCV
Status: REPORTED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Engine (show other bugs)
Version: 8.5.0
Platform: Microsoft Windows Microsoft Windows
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-12-16 15:56 UTC by js333031
Modified: 2024-12-23 22:58 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description js333031 2024-12-16 15:56:17 UTC
Hello, would it be possible to add OpenVINO as backend for OpenCV's DNN module?

https://github.com/opencv/opencv/wiki/Intel-OpenVINO-backend

https://opencv.org/blog/how-to-use-opencv-with-openvino/
Comment 1 Michael Miller 2024-12-21 03:17:22 UTC
(In reply to js333031 from comment #0)
> Hello, would it be possible to add OpenVINO as backend for OpenCV's DNN
> module?
> 
> https://github.com/opencv/opencv/wiki/Intel-OpenVINO-backend
> 
> https://opencv.org/blog/how-to-use-opencv-with-openvino/

Hello,
I will look into it.  I just added full support for OpenCL to the face DNN modules today, and will be working on adding OpenCL to the object detection (auto-tagging) features in the next few weeks.  While I'm in the code I'll see if I can detect Open-VINO and select that OpenCV backend as well.

Cheers,
Mike
Comment 2 js333031 2024-12-21 12:00:14 UTC
Thank you Mike. Please see https://github.com/opencv/opencv/wiki/Intel-OpenVINO-backend#usage

The link, if I understood correctly states that OpenVINO backend will be used if it's installed.  The optional part of that section mentions device selection. That would be useful to select between integrated/discreet Intel GPUs or VPU.
Comment 3 Michael Miller 2024-12-21 13:05:41 UTC
(In reply to js333031 from comment #2)
> Thank you Mike. Please see
> https://github.com/opencv/opencv/wiki/Intel-OpenVINO-backend#usage
> 
> The link, if I understood correctly states that OpenVINO backend will be
> used if it's installed.  The optional part of that section mentions device
> selection. That would be useful to select between integrated/discreet Intel
> GPUs or VPU.

Here's a list of the targets and backends I'm currently working to support:

>    const std::map<std::string, int> str2backend
>    {
>       { "default", cv::dnn::DNN_BACKEND_DEFAULT          },
>        { "halide",  cv::dnn::DNN_BACKEND_HALIDE           },
>        { "ie",      cv::dnn::DNN_BACKEND_INFERENCE_ENGINE },
>        { "opencv",  cv::dnn::DNN_BACKEND_OPENCV           },
>        { "cuda",    cv::dnn::DNN_BACKEND_CUDA             }
>    };
>
>    const std::map<std::string, int> str2target
>    {
>        { "cpu",         cv::dnn::DNN_TARGET_CPU           },
>        { "opencl",      cv::dnn::DNN_TARGET_OPENCL        },
>        { "myriad",      cv::dnn::DNN_TARGET_MYRIAD        },
>        { "vulkan",      cv::dnn::DNN_TARGET_VULKAN        },
>        { "opencl_fp16", cv::dnn::DNN_TARGET_OPENCL_FP16   },
>        { "cuda",        cv::dnn::DNN_TARGET_CUDA          },
>        { "cuda_fp16",   cv::dnn::DNN_TARGET_CUDA_FP16     }


You can see cv::dnn::DNN_BACKEND_INFERENCE_ENGINE is already on the list.  It looks like the correct combination for OpenVINO is DNN_TARGET_OPENCL target with DNN_BACKEND_INFERENCE_ENGINE backend.  The issue is that most of the models used by digiKam are in .onnx format.  The documentation link you sent says DNN_BACKEND_INFERENCE_ENGINE can only be used with .bin or .xml DNN models.  I will see if that has changed since that documentation is a year and half old, which is ancient by OpenCV standards.

When selecting a target and backend, I need to check if the hardware installed in the computer supports the model.  If so, then I enable GPU processing, otherwise it defaults to CPU.  

Right now I only have a check to see if OpenCL is enabled.  I will add checks for other hardware soon.

Cheers,
Mike
Comment 4 js333031 2024-12-21 14:15:11 UTC
What version of OpenCV are you using? More recent versions of OpenVINO can use ONNX directly.

I also think that OpenCV needs to be compiled with OpenVINO support. By default, it's not included. 

I am wondering if OpenCV (w/OpenVINO support) can be compiled independent of digikam and then the dlls placed in digikam's install dir in order to take advantage of OpenVINO. What does your development flow look like? Complete compilation of digikam w/OpenCV changes or can OpenCV be compiled independently and resultant bins placed in digikam's install folder? I am using digikam version 8.5.0.
Comment 5 Michael Miller 2024-12-21 15:34:09 UTC
(In reply to js333031 from comment #4)
> What version of OpenCV are you using? More recent versions of OpenVINO can
> use ONNX directly.
> 
> I also think that OpenCV needs to be compiled with OpenVINO support. By
> default, it's not included. 
> 
> I am wondering if OpenCV (w/OpenVINO support) can be compiled independent of
> digikam and then the dlls placed in digikam's install dir in order to take
> advantage of OpenVINO. What does your development flow look like? Complete
> compilation of digikam w/OpenCV changes or can OpenCV be compiled
> independently and resultant bins placed in digikam's install folder? I am
> using digikam version 8.5.0.

You can compile OpenCV yourself and use your DLLs.  For compatibility, we have checks for a minimum OpenCV version of 4.6, but can use newer version if the libraries are found in the path.

I'll have to check the Windows build to see if we compile and bundle OpenCV with digiKam.  We include OpenCV for some platforms, but not all.  It's been a while since I worked on the Windows build. It's odd, I rarely use Windows since I stopped working for Microsoft :).

Cheers,
Mike
Comment 6 Maik Qualmann 2024-12-22 06:03:47 UTC
Hi Michael, we currently only use OpenCV with the default features in digiKam under Windows.

https://invent.kde.org/graphics/digikam/-/blob/master/project/bundles/vcpkg/01-build-vcpkg.sh?ref_type=heads#L148

Otherwise, features must be activated: https://vcpkg.io/en/package/opencv4.html

Maik
Comment 7 js333031 2024-12-23 20:55:37 UTC
I tried to cross-compile Windows digiKam using a Ubuntu host and ran into a vcpkg issue. I posted a question on vcpkg repo. https://github.com/microsoft/vcpkg/discussions/42900

Can you please let me know if this should be possible?
Comment 8 caulier.gilles 2024-12-23 21:14:46 UTC
Cross compilation for Windows under Linux using VCPKG do not work. I already tried this way in the pass...

Best

Gilles Caulier
Comment 9 js333031 2024-12-23 22:58:42 UTC
(In reply to caulier.gilles from comment #8)
> Cross compilation for Windows under Linux using VCPKG do not work. I already
> tried this way in the pass...
> 
> Best
> 
> Gilles Caulier

I was hoping that info at this site was still valid: https://da-mkay.github.io/blog/linux/ubuntu/2021/04/18/cross-compile-digikam-7.2.0-for-windows-on-ubuntu-20.04.html