A C++ inference SDK based on TensorRT
CheetahInfer is a pure C++ inference SDK based on TensorRT, which supports fast inference of CNNs based computer vision model.
Efficient
With the help of TensorRT's optimization to CNNs and the pure C++ implmentation of preprocessing and postprocessing, CheetahInfer is really efficent. If you are interested in flexibleness, you can refer to FlexInfer.
CheetahInfer has several dependencies:
After the installation of above dependencies, we need modify the TENSORRT_INSTALL_DIR
and OPENCV_INSTALL_DIR
in file Makefile.config
and the environment variable LD_LIBRARY_PATH
and PATH
in .bashrc
file accordingly like the following.
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/home/yichaoxiong/opt/lib/tensorrt:/home/yichaoxiong/opt/lib/opencv"
export PATH="${PATH}:/usr/local/cuda-10.2/bin"
main.cpp
in classifier
folder also need be corrected accordingly.cd classifier
make -j4
./build/main --imgfp /path/to/image
If you want speficy which GPU to use, you can make it by setting the environment variable CUDA_VISIBLE_DEVICES
.
This repository is currently maintained by Hongxiang Cai (@hxcai), Yichao Xiong (@mileistone).
We got some code from TensorRT and retinanet-examples.