{"id":689,"date":"2024-03-14T21:56:15","date_gmt":"2024-03-14T10:56:15","guid":{"rendered":"https:\/\/www.samontab.com\/web\/?p=689"},"modified":"2024-03-14T21:56:16","modified_gmt":"2024-03-14T10:56:16","slug":"openvino-performance-for-state-of-the-art-real-time-monocular-depth-estimation","status":"publish","type":"post","link":"https:\/\/www.samontab.com\/web\/2024\/03\/openvino-performance-for-state-of-the-art-real-time-monocular-depth-estimation\/","title":{"rendered":"OpenVINO performance for state of the art real-time monocular depth estimation"},"content":{"rendered":"\n

Recently, an interesting paper was accepted to CVPR 2024, Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data<\/a>. The authors made publicly available their pre-trained models in three sizes: Depth-Anything-Small, Depth-Anything-Base, and Depth-Anything-Large. <\/p>\n\n\n\n

I wanted to have an idea of how fast these models can run on a CPU these days. Since I am interested in real-time operation, I started with the smallest model to get an idea of how well it performed and how fast it can run the inference in my laptop:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\n
\"\"<\/figure>\n<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

You can see that the smallest model still performs relatively well. In terms of inference time, it took on average 1.02 seconds per image on my CPU, and the size of this PyTorch model, depth_anything_vits14.pth, is 95MB. It’s fast and small, but not really ideal for real-time applications. Let’s see if we can do better.<\/p>\n\n\n\n

OpenVINO uses their own Intermediate Representation format, IR, which is designed to be optimised for inference. Furthermore, you can then generate a quantised model from the already OpenVINO optimised model, getting even better inference times and a smaller model.<\/p>\n\n\n\n

Here is a table with the different results on my machine:<\/p>\n\n\n\n

Model Name<\/strong><\/td>Inference speed (FPS)<\/strong><\/td>Model Size (MB)<\/strong><\/td><\/tr>
depth_anything_vits14.pth (Original PyTorch model)<\/td>~1<\/td>95<\/td><\/tr>
depth_anything_vits14.(bin+xml) (OpenVINO IR)<\/td>7.65<\/td>47.11<\/td><\/tr>
depth_anything_vits14_int8.(bin+xml) (IR Quantised)<\/td>9.74<\/strong><\/td>24.27<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

You can clearly see that there is a massive increase in performance when you use the quantised OpenVINO IR model compared to the original PyTorch model. This allows real time operation on a laptop. And for reference, here is the output of the quantised model:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The output is still reasonably good, with the nice bonus that it can be used in real time, plus the file size is about one quarter of the original. All thanks to OpenVINO’s great ability to optimise the inference pipeline!<\/p>\n","protected":false},"excerpt":{"rendered":"

Recently, an interesting paper was accepted to CVPR 2024, Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. The authors made publicly available their pre-trained models in three sizes: Depth-Anything-Small, Depth-Anything-Base, and Depth-Anything-Large. I wanted to have an idea of how fast these models can run on a CPU these days. Since I am interested […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29,21,24],"tags":[46,51,25],"class_list":["post-689","post","type-post","status-publish","format-standard","hentry","category-computer-vision","category-open-source","category-openvino","tag-depth","tag-monocular","tag-openvino"],"_links":{"self":[{"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/posts\/689","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/comments?post=689"}],"version-history":[{"count":0,"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/posts\/689\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/media?parent=689"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/categories?post=689"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.samontab.com\/web\/wp-json\/wp\/v2\/tags?post=689"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}