Segregator: Global Point Cloud Registration with Semantic and Geometric Cues

Pengyu Yin, Shenghai Yuan, Haozhi Cao, Xingyu Ji, Shuyang Zhang, and Lihua Xie

Paper links: Arxiv | IEEE

Segregator is a global point cloud registration pipeline using both semantic and geometric information. Instead of focusing solely on point level features, we build degenerancy-robust correspondences between two LiDAR scans on a mixed-level (geometric features as well as semantic clusters). Additionally, G-TRIM based outlier pruning is also proposed to find out the inlier correspondence set more efficiently. Please refer to our paper for more details.

Test Environment

• Linux 18.04/20.04 LTS
• ROS Melodic/Noetic

Installation

Run the following lines for denpandencies:

sudo apt install cmake libeigen3-dev libboost-all-dev

Use catkin_tools to build the project:

mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/src
git clone [email protected]:Pamphlett/Segreagator.git
cd Segreagator && mkdir build && cd build
cmake ..
mv pmc-src/ ../../../build/
cd ~/catkin_ws
catkin build segregator 

Test on different datasets

• A toy example on KITTI

We include two distant scans (frame 0 and 4413), as well as their corresponding semantic masks, from KITTI dataset sequence 00. Please run the following lines in the catkin workspace to reproduce the figure above:

source devel/setup.bash
roslaunch segregator run_segregator.launch

• On other/self-collected dataset

Generally, apart from the pointcloud file itself, per-point semantic label is also needed to make Segregator work. We recommend using SPVNAS (the most accurate), Rangenet or SalsaNext (far more computationally efficient, range image-based methods with noticable segmentation performance drop) to generate these labels.

Illustration of registration results

Different rows corresponds to initial values, results from sota Quatro and Segregator.

Citation

If you find Segregator useful in your academic project, please cite our paper:

@INPROCEEDINGS{10160798,
  author={Yin, Pengyu and Yuan, Shenghai and Cao, Haozhi and Ji, Xingyu and Zhang, Shuyang and Xie, Lihua},
  booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)}, 
  title={Segregator: Global Point Cloud Registration with Semantic and Geometric Cues}, 
  year={2023},
  volume={},
  number={},
  pages={2848-2854},
  doi={10.1109/ICRA48891.2023.10160798}}

Contact

Please kindly reach out to me if you have any question. Any discussion is also welcome: Pengyu Yin ([email protected])

Acknowledgements

This research is supported by the National Research Foundation, Singapore under its Medium Sized Center for Advanced Robotics Technology Innovation (CARTIN).

Also, we would like to show our greatest thankfulness to authors of the following repos for making their works public:

• Teaser (baseline)
• Quatro (baseline)
• SPVNAS (semantic segmentation)
• T-LOAM (clustering)