Color-NeuS: Reconstructing Neural Implicit Surfaces with Color

Licheng Zhong · Lixin Yang · Kailin Li · Haoyu Zhen · Mei Han · Cewu Lu

3DV 2024

Project Page | arXiv | Data

https://github.com/Colmar-zlicheng/Color-NeuS/assets/111580763/5b9a3ed9-7d4f-48d9-84b5-a60a0a6adef9

:rocket: Dependencies

git clone https://github.com/Colmar-zlicheng/Color-NeuS.git
cd Color-NeuS
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirements.txt
pip install "git+https://github.com/facebookresearch/pytorch3d.git"

:steam_locomotive: Train

Dataset

• IHO Video
• DTU (NeuS Preprocess Link | Raw Link)
• BlendedMVS (NeuS Preprocess Link | Raw Link)
• OmniObject3D

General command

• set ${DATASET} as one in [iho, dtu, bmvs, omniobject3d]
• set ${OBJECT_NAME} as the name of the object in the dataset

python train.py -g 0 --cfg config/Color_NeuS_${DATASET}.yml -obj ${OBJECT_NAME} --exp_id ${EXP_ID}

Command line arguments

• -g, --gpu_id, visible GPUs for training, e.g. -g 0. Only supports single GPU.
• --exp_id specify the name of experiment, e.g. --exp_id ${EXP_ID}. When --exp_id is provided, the code requires that no uncommitted change is remained in the git repo. Otherwise, it defaults to 'default' for training and 'eval_{cfg}_{OBJECT_NAME}' for evaluation. All results will be saved in exp/${EXP_ID}*{timestamp}.

For example

# IHO Video: ghost_bear
python train.py -g 0 --cfg config/Color_NeuS_iho.yml -obj ghost_bear --exp_id Color_NeuS_iho_ghost_bear
# DTU: dtu_scan83
python train.py -g 0 --cfg config/Color_NeuS_dtu.yml -obj 83 --exp_id Color_NeuS_dtu_83
# BlendedMVS: bmvs_bear
python train.py -g 0 --cfg config/Color_NeuS_bmvs.yml -obj bear --exp_id Color_NeuS_bmvs_bear
# OmniObject3D: doll_002
python train.py -g 0 --cfg config/Color_NeuS_omniobject3d.yml -obj doll_002 --exp_id Color_NeuS_omniobject3d_doll_002

Checkpoint

All the training checkpoints are saved at exp/${EXP_ID}_{timestamp}/checkpoints/

Other Method

We also provide our implementation of NeuS in this repo. To train NeuS, you can replace Color_NeuS_${DATASET}.yml with NeuS_${DATASET}.yml in the above command line, such as:

# IHO Video: ghost_bear
python train.py -g 0 --cfg config/NeuS_iho.yml -obj ghost_bear --exp_id NeuS_iho_ghost_bear

:monocle_face: Inference

• set corresponding ${DATASET} and ${OBJECT_NAME} as above
• set ${PATH_TO_CHECKPOINT} as the path to the checkpoint (NeuS_Trainer.pth.tar) to be loaded

python evaluation.py -g 0 --cfg config/Color_NeuS_${DATASET}.yml -obj ${OBJECT_NAME} -rr 512 --reload ${PATH_TO_CHECKPOINT}

• -rr, --recon_res is the resolution of the reconstructed mesh. The default value is 512.

:key: License

The code provided herein are available for usage as specified in the LICENSE file. By downloading and using the code you agree to the terms in the LICENSE.

:earth_asia: Citation

@inproceedings{zhong2024colorneus,
    title     = {Color-NeuS: Reconstructing Neural Implicit Surfaces with Color},
    author    = {Zhong, Licheng and Yang, Lixin and Li, Kailin and Zhen, Haoyu and Han, Mei and Lu, Cewu},
    booktitle = {International Conference on 3D Vision (3DV)},
    year      = {2024}
}

For more questions, please contact Licheng Zhong: [email protected]