SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model

Di Wang, Jing Zhang, Bo Du, Minqiang Xu, Lin Liu, Dacheng Tao, Liangpei Zhang

News | Abstract | Usage | Results | Statement

News

2023.03.25

• The rotated bounding box version of SOTA (SOTA-RBB) can be obtained from [Dataset] and [Baidu]

2023.12.07

• The SAMRS dataset can be acquired from [Baidu]

2023.09.30

• The instance and detection labels are released! See in [Dataset]

2023.09.26

• The NeurIPS version is post on arxiv!

2023.09.23

• The codes of generating SAMRS dataset are released!

2023.09.22

• The paper is accepted by NeurIPS 2023 Datasets and Benchmarks Track!

2023.08.30

• The SAMRS images are released! See in [Dataset]

2023.06.14

• The semantic labels are released! See in [Dataset]

2023.05.04

• The tech report is post on arxiv! Work in progress.

Other applications of ViTAE inlcude: VSA | ViTPose | Matting | Scene Text Spotting | Video Object Segmentation

Introduction

This is the official repository of the paper Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model

In this study, we leverage SAM and existing RS object detection datasets to develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS. SAMRS surpasses existing high-resolution RS segmentation datasets in size by several orders of magnitude, and provides object category, location, and instance information that can be used for semantic segmentation, instance segmentation, and object detection, either individually or in combination. We also provide a comprehensive analysis of SAMRS from various aspects. We hope it could facilitate research in RS segmentation, particularly in large model pre-training.

Usage

• Please see in [Generate Dataset] for the codes of producing SAMRS dataset.

• Please see in [Pretraining and Finetuning] for the codes of pretraining with SAMRS and fintuning on other datasets.

Results

The basic information of generated datasets

We present the comparison of our SAMRS dataset with existing high-resolution RS segmentation datasets in table. Based on the available high-resolution RSI object detection datasets, we can efficiently annotate 10,5090 images, which is more than ten times the capacity of existing datasets. Additionally, SAMRS inherits the categories of the original detection datasets, which makes them more diverse than other high-resolution RS segmentation collections. It is worth noting that RS object datasets usually have more diverse categories than RS segmentation datasets due to the difficulty of tagging pixels in RSIs, and thus our SAMRS reduces this gap.

Visualization of Generated Masks

In figure, we visualize some segmentation annotations from the three subsets in our SAMRS dataset. As can be seen, SOTA exhibits a greater number of instances for tiny cars, whereas FAST provides a more fine-grained annotation of existing categories in SOTA such as car, ship, and plane. SIOR on the other hand, offers annotations for more diverse ground objects, such as dam. Hence, our SAMRS dataset encompasses a wide range of categories with varying sizes and distributions, thereby presenting a new challenge for RS semantic segmentation.

Dataset Statistics and Analysis

The class distribution.

The mask size distribution.

Statement

This project is for research purpose only. For any other questions please contact [email protected].

Citation

If you find SAMRS helpful, please consider giving this repo a ⭐ and citing:

@inproceedings{SAMRS,
  title={{SAMRS}: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model},
  author={Di Wang and Jing Zhang and Bo Du and Minqiang Xu and Lin Liu and Dacheng Tao and Liangpei Zhang},
  booktitle={Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
  year={2023},
  url={https://openreview.net/forum?id=jHrgq55ftl}
}

Relevant Projects

[1] An Empirical Study of Remote Sensing Pretraining, IEEE TGRS, 2022 | Paper | Github
     Di Wang^∗, Jing Zhang^∗, Bo Du, Gui-Song Xia and Dacheng Tao

[2] Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model, IEEE TGRS, 2022 | Paper | Github
     Di Wang^∗, Qiming Zhang^∗, Yufei Xu^∗, Jing Zhang, Bo Du, Dacheng Tao and Liangpei Zhang