PointMamba Save
PointMamba: A Simple State Space Model for Point Cloud Analysis
PointMamba
A Simple State Space Model for Point Cloud Analysis
Dingkang Liang1 *, Xin Zhou1 *, Xinyu Wang1 *, Xingkui Zhu1 , Wei Xu1, Zhikang Zou2, Xiaoqing Ye2, and Xiang Bai1
1 Huazhong University of Science & Technology, 2 Baidu Inc.
(*) equal contribution
📣 News
- [01/Apr/2024] ScanObjectNN with further data augmentation is now available, check it out!
- [16/Mar/2024] The configurations and checkpoints for ModelNet40 are now accessible, check it out!
- [05/Mar/2024] Our paper DAPT (github) has been accepted by CVPR 2024! 🥳🥳🥳 Check it out and give it a star 🌟!
- [16/Feb/2024] Release the paper.
Abstract
Transformers have become one of the foundational architectures in point cloud analysis tasks due to their excellent global modeling ability. However, the attention mechanism has quadratic complexity and is difficult to extend to long sequence modeling due to limited computational resources and so on. Recently, state space models (SSM), a new family of deep sequence models, have presented great potential for sequence modeling in NLP tasks. In this paper, taking inspiration from the success of SSM in NLP, we propose PointMamba, a framework with global modeling and linear complexity. Specifically, by taking embedded point patches as input, we proposed a reordering strategy to enhance SSM's global modeling ability by providing a more logical geometric scanning order. The reordered point tokens are then sent to a series of Mamba blocks to causally capture the point cloud structure. Experimental results show our proposed PointMamba outperforms the transformer-based counterparts on different point cloud analysis datasets, while significantly saving about 44.3% parameters and 25% FLOPs, demonstrating the potential option for constructing foundational 3D vision models. We hope our PointMamba can provide a new perspective for point cloud analysis.
Overview
Main Results
| Task | Dataset | Config | Acc.(Scratch) | Download (Scratch) | Acc.(pretrain) | Download (Finetune) |
|---|---|---|---|---|---|---|
| Pre-training | ShapeNet | pretrain.yaml | N.A. | here | ||
| Classification | ModelNet40 | finetune_modelnet.yaml | 92.4% | here | 93.6% | here |
| Classification | ScanObjectNN | finetune_scan_objbg.yaml | 88.30% | here | 90.71% | here |
| Classification* | ScanObjectNN | finetune_scan_objbg.yaml | \ | \ | 93.29% | here |
| Classification | ScanObjectNN | finetune_scan_objonly.yaml | 87.78% | here | 88.47% | here |
| Classification* | ScanObjectNN | finetune_scan_objonly.yaml | \ | \ | 91.91% | here |
| Classification | ScanObjectNN | finetune_scan_hardest.yaml | 82.48% | here | 84.87% | here |
| Classification* | ScanObjectNN | finetune_scan_hardest.yaml | \ | \ | 88.17% | here |
| Part Segmentation | ShapeNetPart | part segmentation | 85.8% mIoU | here | 86.0% mIoU | here |
* indicates further using simple rotational augmentation for training.
Getting Started
Datasets
See DATASET.md for details.
Usage
See USAGE.md for details.
To Do
- Release code.
- Release checkpoints.
- ModelNet40.
- Semantic segmentation.
Acknowledgement
This project is based on Point-BERT (paper, code), Point-MAE (paper, code), Mamba (paper, code), Causal-Conv1d (code). Thanks for their wonderful works.
Citation
If you find this repository useful in your research, please consider giving a star ⭐ and a citation
@article{liang2024pointmamba,
title={PointMamba: A Simple State Space Model for Point Cloud Analysis},
author={Dingkang Liang and Xin Zhou and Xinyu Wang and Xingkui Zhu and Wei Xu and Zhikang Zou and Xiaoqing Ye and Xiang Bai},
journal={arXiv preprint arXiv:2402.10739},
year={2024}
}
Open Source Agenda Badge
