YOLOv4 with TensorRT engine

This package contains the yolov4_trt_node that performs the inference using NVIDIA's TensorRT engine

This package works for both YOLOv3 and YOLOv4. Do change the commands accordingly, corresponding to the YOLO model used.

Setting up the environment

Install dependencies

Current Environment:

• Jetson Xavier AGX
• ROS Melodic
• Ubuntu 18.04
• Jetpack 4.4
• TensorRT 7+

Dependencies:

• OpenCV 3.x
• numpy 1.15.1
• Protobuf 3.8.0
• Pycuda 2019.1.2
• onnx 1.4.1 (depends on Protobuf)

Install all dependencies with below commands

Install pycuda (takes awhile)
$ cd ${HOME}/catkin_ws/src/yolov4_trt_ros/dependencies
$ ./install_pycuda.sh

Install Protobuf (takes awhile)
$ cd ${HOME}/catkin_ws/src/yolov4_trt_ros/dependencies
$ ./install_protobuf-3.8.0.sh

Install onnx (depends on Protobuf above)
$ sudo pip3 install onnx==1.4.1

• Please also install jetson-inference
• Note: This package uses similar nodes to ros_deep_learning package. Please place a CATKIN_IGNORE in that package to avoid similar node name catkin_make error
• If these scripts do not work for you, do refer to this amazing repository by jefflgaol on installing the above packages and more on Jetson ARM devices.

Setting up the package

1. Clone project into catkin_ws and build it

$ cd ~/catkin_ws && catkin_make
$ source devel/setup.bash

2. Make libyolo_layer.so

$ cd ${HOME}/catkin_ws/src/yolov4_trt_ros/plugins
$ make

This will generate a libyolo_layer.so file

3. Place your yolo.weights and yolo.cfg file in the yolo folder

$ cd ${HOME}/catkin_ws/src/yolov4_trt_ros/yolo

** Please name the yolov4.weights and yolov4.cfg file as follows:

• yolov4.weights
• yolov4.cfg

Run the conversion script to convert to TensorRT engine file

$ ./convert_yolo_trt

• Input the appropriate arguments
• This conversion might take awhile
• The optimised TensorRT engine would now be saved as yolov3-416.trt / yolov4-416.trt

4. Change the class labels

$ cd ${HOME}/catkin_ws/src/yolov4_trt_ros/utils
$ vim yolo_classes.py

• Change the class labels to suit your model

5. Change the video_input and topic_name

$ cd ${HOME}/catkin_ws/src/yolov4_trt_ros/launch

• yolov3_trt.launch : change the topic_name

• yolov4_trt.launch : change the topic_name

• video_source.launch : change the input format (refer to this Link

  • video_source.launch requires jetson-inference to be installed
  • Default input is CSI camera

Using the package

Running the package

Note: Run the launch files separately in different terminals

1. Run the video_source

# For csi input
$ roslaunch yolov4_trt_ros video_source.launch input:=csi://0

# For video input
$ roslaunch yolov4_trt_ros video_source.launch input:=/path_to_video/video.mp4

# For USB camera
$ roslaunch yolov4_trt_ros video_source.launch input:=v4l2://0

2. Run the yolo detector

# For YOLOv3 (single input)
$ roslaunch yolov4_trt_ros yolov3_trt.launch

# For YOLOv4 (single input)
$ roslaunch yolov4_trt_ros yolov4_trt.launch

# For YOLOv4 (multiple input)
$ roslaunch yolov4_trt_ros yolov4_trt_batch.launch

3. For maximum performance

$ cd /usr/bin/
$ sudo ./nvpmodel -m 0	# Enable 2 Denver CPU
$ sudo ./jetson_clock	# Maximise CPU/GPU performance

• These commands are found/referred in this forum post
• Please ensure the jetson device is cooled appropriately to prevent overheating

Parameters

• str model = "yolov3" or "yolov4"

• str model_path = "/abs_path_to_model/"

• int input_shape = 288/416/608

• int category_num = 80

• double conf_th = 0.5

• bool show_img = True

• Default Input FPS from CSI camera = 30.0

• To change this, go to jetson-inference/utils/camera/gstCamera.cpp

# In line 359, change this line
mOptions.frameRate = 15

# To desired frame_rate
mOptions.frameRate = desired_frame_rate

Results obtained

Inference Results

Single Camera Input

• Will be adding inference tests for YOLOv3/4 and YOLOv3-tiny/YOLOv4-tiny for different Jetson devices and multiple camera inputs inference tests in the future

1. Screenshots

Video_in: .mp4 video (640x360)

Tests Done on Xavier AGX

Avg FPS : ~38 FPS

Licenses and References

1. TensorRT samples from jkjung-avt

Many thanks for his project with tensorrt samples. I have referenced his source code and adapted it to ROS for robotics applications.

I also used the pycuda and protobuf installation script from his project

Those code are under MIT License

2. Jetson-inference from dusty-nv

Many thanks for his work on the Jetson Inference with ROS. I have used his video_source input from his project for capturing video inputs.

Those code are under NVIDIA License