OpenVINO performance for state of the art real-time monocular depth estimation

Recently, an interesting paper was accepted to CVPR 2024, Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. The authors made publicly available their pre-trained models in three sizes: Depth-Anything-Small, Depth-Anything-Base, and Depth-Anything-Large.

I wanted to have an idea of how fast these models can run on a CPU these days. Since I am interested in real-time operation, I started with the smallest model to get an idea of how well it performed and how fast it can run the inference in my laptop:

You can see that the smallest model still performs relatively well. In terms of inference time, it took on average 1.02 seconds per image on my CPU, and the size of this PyTorch model, depth_anything_vits14.pth, is 95MB. It’s fast and small, but not really ideal for real-time applications. Let’s see if we can do better.

OpenVINO uses their own Intermediate Representation format, IR, which is designed to be optimised for inference. Furthermore, you can then generate a quantised model from the already OpenVINO optimised model, getting even better inference times and a smaller model.

Here is a table with the different results on my machine:

Model Name	Inference speed (FPS)	Model Size (MB)
depth_anything_vits14.pth (Original PyTorch model)	~1	95
depth_anything_vits14.(bin+xml) (OpenVINO IR)	7.65	47.11
depth_anything_vits14_int8.(bin+xml) (IR Quantised)	9.74	24.27

You can clearly see that there is a massive increase in performance when you use the quantised OpenVINO IR model compared to the original PyTorch model. This allows real time operation on a laptop. And for reference, here is the output of the quantised model:

The output is still reasonably good, with the nice bonus that it can be used in real time, plus the file size is about one quarter of the original. All thanks to OpenVINO’s great ability to optimise the inference pipeline!

Posted in Computer Vision, Open Source, OpenVINO.

Tagged with depth, monocular, openvino.

rev="post-689" No comments

By samontab – March 14, 2024

Using Qt Creator as your IDE for Rust

Qt Creator is a great IDE for C++, and it recently included support for the Language Server Protocol. This means that we can add the ability to write in other languages to the IDE, like Rust for example. In this post I will show you how to do exactly that.

First, make sure you have Rust installed and that everything works as you would expect. For example, you should be able to create a new Rust project with this command:

cargo new ide_rust

We can open these Rust files into Qt Creator in the following way. Go to File->New File or Project, then select Import Project, and Import Existing Project, as shown in the following image:

Now simply select the folder that was created by cargo and set any Project name you want:

Under file selection, simply click on the folder to add all the files:

Finally choose if you want to use git or other version control software and click on Finish:

You should now be able to navigate and open the files of your project in the IDE:

We are now going to configure Qt Creator to work with Rust. Go to Tools->Options->Kits and go to the Debuggers tab. Click on Add, specify a name like Rust-GDB for example, and under Path specify where rust-gdb is located in your machine:

Now go to the Kits tab, click on Add. Select a name, like Rust for example, and on Debugger select Rust-GDB.

Now we are going to update the syntax highlighter. Click on Text Editor and go to Generic Highlighter. Then, click on Download Definitions, and then Reload Definitions. This should update them to the latest version.

Now click on OK to close the Options window and go to Projects (on the left side), and under both, the build and clean steps, simply remove them by clicking on the small x mark on their right side:

It should look like this:

Now at the build and clean steps, click on Add Build Step, select Custom Process Step and write cargo build and cargo clean respectively at the appropriate fields:

Now click on the Run tab on the left and select your executable:

You can now go back to the Edit tab and edit the file. You can press CTRL-B to build or CTRL-R to run your application, as well as auto-complete with CTRL-Space: