Using OpenVINO with the OpenCV DNN module

OpenCV 4.8.0 has been released recently. Also, OpenVINO just released 2023.0.1 last week so it’s a good time to see how they can be used together to perform inference on a IR optimised model. If you haven’t installed OpenVINO yet, you can learn how to do it here. If you haven’t installed OpenCV, you can follow this guide.

For this, I’m going to use a monocular depth estimation model, MiDaS. This model takes as input a color image, and it outputs an inverse depth estimation for every pixel. The closer the object is to the camera, the lighter the pixel, and vice-versa. It looks like this:

Let’s grab the original ONNX model and convert it to the Intermediate Representation(IR) to be used with OpenVINO:

omz_downloader --name midasnet
omz_converter --name midasnet

We can now use OpenVINO from inside OpenCV’s DNN module:

 cv::dnn::Net net = cv::dnn::readNetFromModelOptimizer("../public/midasnet/FP32/midasnet.xml", "../public/midasnet/FP32/midasnet.bin");
net.setPreferableBackend(cv::dnn::Backend::DNN_BACKEND_INFERENCE_ENGINE);

Then we can proceed exactly as how we would normally do with the OpenCV DNN module:

cv::Mat blob = cv::dnn::blobFromImage(originalImage, 1., cv::Size(384, 384));
net.setInput(blob);
cv::Mat output = net.forward();

And that’s pretty much all you need to use OpenVINO from inside OpenCV’s DNN module. It’s basically almost the same, you only need to change how to read the model, and set the back-end to use the Inference Engine instead of the default OpenCV DNN one.

Posted in Computer Vision, Open Source, OpenCV, OpenVINO.

Tagged with depth, open source, openvino, performance.

rev="post-646" No comments

By samontab – July 12, 2023

How to build OpenVINO 2023.0 in Ubuntu 22.04.2 LTS and run an example

Intel has recently released the new OpenVINO 2023.0 with major new features such as macOS ARM64 support, support for Python 3.11, the ability to directly run models without converting them, and many more performance gains and new features. I’ll show you here how to compile it for Ubuntu 22.04.2 LTS.

First we need to make sure we have the latest updates installed in the OS:

sudo apt-get update
sudo apt-get upgrade

Now we need to install some build tools in case you don’t have them installed:

sudo apt-get install build-essential git cmake

Now we get the OpenVINO 2023.0 source code:

cd ~
git clone --recursive --branch releases/2023/0 https://github.com/openvinotoolkit/openvino.git openvino-2023.0
cd openvino-2023.0
git submodule update --init --recursive

There are some OpenVINO dependencies that need to be installed first. These require you to have access to sudo in order to install them using the provided script, like this:

sudo ./install_build_dependencies.sh

After that’s finished we are ready to install some python requirements, configure OpenVINO, and build it with OpenCV support:

pip install -r src/bindings/python/src/compatibility/openvino/requirements-dev.txt
mkdir build
cd build
cmake -DENABLE_PYTHON=ON -DENABLE_OPENCV=ON -DCMAKE_INSTALL_PREFIX=install ..
make -j`nproc`
make install

OpenVINO 2023.0 is now installed. You can test that it’s working properly by running this:

cd ~/openvino-2023.0/bin/intel64
./hello_query_device

You should see at least a CPU device in there.

[ INFO ] Build ................................. 2023.0.0-11004-caae459f547-releases/2023/0
[ INFO ]  
[ INFO ] Available devices:  
[ INFO ] CPU
[ INFO ]        SUPPORTED_PROPERTIES:
etc...

Now we will install the OpenVINO development tools, such as the model optimizer, with this:

pip install openvino-dev==2023.0.0

These scripts are now installed at ~/.local/bin which is not in the path, so we need to add it there:

export PATH=$PATH:~/.local/bin

This will only work for the current terminal. If you want to make this change permanent, just add that previous line to the end of the ~/.bashrc file.

Now we can use the tools. You can for example list all the available models for download like this:

omz_downloader --print_all

OK, now let’s download a model, convert it to the OpenVINO Intermediate Representation format (IR), and run an inference on it. We are going to classify this image (you can use any image you want):

cd ~/openvino-2023.0/bin/intel64
omz_downloader --name alexnet
omz_converter --name alexnet
./classification_sample_async -m public/alexnet/FP32/alexnet.xml -i image.jpg

The result we get is the top 10 classes that it thinks the image represents:

Top 10 results:

Image image.jpg

classid probability
------- -----------
207     0.8424635  
208     0.1065520  
222     0.0185891  
219     0.0128466  
257     0.0080099  
216     0.0044186  
175     0.0013159  
212     0.0008337  
244     0.0008299  
194     0.0007491

This means that the object with class ID 207 has a probability of 84.2% to be there in the photo. Let’s see what object that is. These IDs are based on the Imagenet 1000 class list, that you can see here for example. We’re going to grab a txt version of this list and grep it to see the contents:

wget https://gist.githubusercontent.com/yrevar/942d3a0ac09ec9e5eb3a/raw/238f720ff059c1f82f368259d1ca4ffa5dd8f9f5/imagenet1000_clsidx_to_labels.txt
cat imagenet1000_clsidx_to_labels.txt | grep 207

And the answer is: