RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes
PyTorch implementation of paper "RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes" by Bingchen Gong*, Yuehao Wang*, Xiaoguang Han, and Qi Dou.
A novel user-friendly color editing approach for neural radiance fields.
[Paper] [Project Website] [Selected Results]
https://user-images.githubusercontent.com/6317569/216793671-18ca0551-c668-4bb6-b929-2ba7b5059252.mp4
Tested on Ubuntu 18.04 with PyTorch 1.12.1 and CUDA 11.1.
Type the commands below to set up the running environment.
conda create -n recolornerf python=3.8
conda activate recolornerf
# PyTorch (may need to adapt to your environment)
pip install torch torchvision
# PyTorch3D
conda install pytorch3d -c pytorch3d
# Essentials
pip install tqdm scikit-image opencv-python configargparse lpips imageio-ffmpeg kornia Pillow lpips tensorboard trimesh
conda install -c conda-forge einops
# Palette extraction
conda install -c conda-forge scipy
conda install -c conda-forge cython
CVXOPT_BUILD_GLPK=1 pip install cvxopt
Download our customized initial palettes to data_palette/
for reproducing our layer decomposition results.
For scenes other than the provided ones, you can use the jupyter notebook tools/get_palette.ipynb
to generate and customize new palettes.
Type the command below to train a RecolorNeRF model:
python run_recolornerf.py --config configs/chair.txt
We provide our configurations for 15 scenes in the configs/
directory. Remember to change the datadir
option to your dataset path. For more options, please refer to the utils/opt.py
file.
We create a jupyter notebook tools/color_edit.ipynb
for recoloring an optimized RecolorNeRF in a quasi-interactive way. In this jupyter notebook, some simple GUI widgets (like color pickers) and visualization & rendering scripts are provided.
If you find our code or paper is helpful, please consider citing:
@article{gong2023recolornerf,
title={RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes},
author={Gong, Bingchen and Wang, Yuehao and Han, Xiaoguang and Dou, Qi},
journal={arXiv preprint arXiv:2301.07958},
year={2023}
}