[CVPR 2021] Official PyTorch Code of GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection
Abhinav Kumar, Garrick Brazil, Xiaoming Liu
project, supp, 5min_talk, slides, demo, poster, arxiv
This code is based on Kinematic-3D, such that the setup/organization is very similar. A few of the implementations, such as classical NMS, are based on Caffe.
Please cite the following paper if you find this repository useful:
@inproceedings{kumar2021groomed,
title={{GrooMeD-NMS}: Grouped Mathematically Differentiable NMS for Monocular {$3$D} Object Detection},
author={Kumar, Abhinav and Brazil, Garrick and Liu, Xiaoming},
booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2021}
}
Requirements
This is tested with NVIDIA 1080 Ti GPU. Other platforms have not been tested. Unless otherwise stated, the below scripts and instructions assume the working directory is the project root.
Clone the repo first:
git clone https://github.com/abhi1kumar/groomed_nms.git
Cuda & Python
Install some basic packages:
sudo apt-get install libopenblas-dev libboost-dev libboost-all-dev git
sudo apt install gfortran
# We need to compile with older version of gcc and g++
sudo apt install gcc-5 g++-5
sudo ln -f /usr/bin/gcc-5 /usr/local/cuda-8.0/bin/gcc
sudo ln -s /usr/bin/g++-5 /usr/local/cuda-8.0/bin/g++
Next, install conda and then install the required packages:
wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh
bash Anaconda3-2020.02-Linux-x86_64.sh
source ~/.bashrc
conda list
conda create --name py36 --file dependencies/conda.txt
conda activate py36
KITTI Data
Download the following images of the full KITTI 3D Object detection dataset:
Then place a soft-link (or the actual data) in data/kitti
:
n -s /path/to/kitti data/kitti
The directory structure should look like this:
./groomed_nms
├── cuda_env
├── data
│ ├── kitti
│ ├── training
│ │ ├── calib
│ │ ├── image_2
│ │ └── label_2
│ │
│ └── testing
│ ├── calib
│ └── image_2
│
├── dependencies
├── lib
├── models
└── scripts
Then, use the following scripts to extract the data splits, which use soft-links to the above directory for efficient storage:
python data/kitti_split1/setup_split.py
python data/kitti_split2/setup_split.py
Next, build the KITTI devkit eval:
h data/kitti_split1/devkit/cpp/build.sh
Classical NMS
Lastly, build the classical NMS modules:
d lib/nms
ake
d ../..
Training is carried out in two stages - a warmup and a full. Review the configurations in scripts/config
for details.
chmod +x scripts_training.sh
./scripts_training.sh
If your training is accidentally stopped, you can resume at a checkpoint based on the snapshot with the restore
flag. For example, to resume training starting at iteration 10k, use the following command:
source dependencies/cuda_8.0_env
CUDA_VISIBLE_DEVICES=0 python -u scripts/train_rpn_3d.py --config=groumd_nms --restore=10000
We provide logs/models/predictions for the main experiments on KITTI Val 1/Val 2/Test data splits available to download here.
Make an output
folder in the project directory:
mkdir output
Place different models in the output
folder as follows:
./groomed_nms
├── output
│ ├── groumd_nms
│ ├── groumd_nms_split2
│ └── groumd_nms_full_train_2
│
│ ...
To test, run the file as below:
chmod +x scripts_evaluation.sh
./scripts_evaluation.sh
For questions, feel free to post here or drop an email to this address- [email protected]