Unofficial cpp and python implementation of depth-anything model using tensorrt api.
Depth estimation is the task of measuring the distance of each pixel relative to the camera. This repo provides a TensorRT implementation of the Depth-Anything depth estimation model in both C++ and Python, enabling efficient real-time inference.
The inference time includes the pre-preprocessing and post-processing stages:
Device | Model | Model Input (WxH) | Image Resolution (WxH) | Inference Time(ms) |
---|---|---|---|---|
RTX4090 | Depth-Anything-S | 518x518 | 1280x720 | 3 |
RTX4090 | Depth-Anything-B | 518x518 | 1280x720 | 6 |
RTX4090 | Depth-Anything-L | 518x518 | 1280x720 | 12 |
[!NOTE] Inference was conducted using
FP16
precision, with a warm-up period of 10 frames. The reported time corresponds to the last inference.
depth-anything-tensorrt.exe <onnx model> <input image or video>
depth-anything-tensorrt.exe <engine> <input image or video>
Example:
# infer image
depth-anything-tensorrt.exe depth_anything_vitb14.engine test.jpg
# infer folder(images)
depth-anything-tensorrt.exe depth_anything_vitb14.engine data
# infer video
depth-anything-tensorrt.exe depth_anything_vitb14.engine test.mp4 # the video path
cd depth-anything-tensorrt/python
# infer image
python trt_infer.py --engine <path to trt engine> --img <single-img> --outdir <outdir> [--grayscale]
Refer to our docs/INSTALL.md for C++ environment installation.
cd <tensorrt installation path>/python
pip install cuda-python
pip install tensorrt-8.6.0-cp310-none-win_amd64.whl
pip install opencv-python
Perform the following steps to create an onnx model:
Download the pretrained model and install Depth-Anything:
git clone https://github.com/LiheYoung/Depth-Anything
cd Depth-Anything
pip install -r requirements.txt
Copy and paste dpt.py in this repo to <depth_anything_installpath>/depth_anything
folder. Then, copy export.py in this repo to <depth_anything_installpath>
. Note that I've only removed a squeeze operation at the end of model's forward function in dpt.py
to avoid conflicts with TensorRT.
Export the model to onnx format using export.py. You will get an onnx file named depth_anything_vit{}14.onnx
, such as depth_anything_vitb14.onnx
. Note that I used torch cpu version for exporting the onnx model as it is not necessary to deploy the model on GPU when exporting.
conda create -n depth-anything python=3.8
conda activate depth-anything
pip install torch torchvision
pip install opencv-python
pip install onnx
python export.py --encoder vitb --load_from depth_anything_vitb14.pth --image_shape 3 518 518
[!TIP] The width and height of the model input should be divisible by 14, the patch height.
opencv_world490.dll
and opencv_videoio_ffmpeg490_64.dll
into the <depth_anything_installpath>/build/Release
folder.This project is based on the following projects: