News

• [2024/4/5] Our paper has been accepted as Highlight (Top 11.9% of accepted papers)!
• [2024/2/28] We release the code of geometry reconstruction, including test and inference.
• [2024/2/27] SIFU has been accepted by CVPR 2024! See you in Seattle!
• [2023/12/13] We release the paper on arXiv.
• [2023/12/10] We build the Project Page.

Installation

• Ubuntu 20 / 18
• CUDA=11.6 or 11.7 or 11.8, GPU Memory > 16GB
• Python = 3.8
• PyTorch = 1.13.0 (official Get Started)

We thank @levnikolaevich and @GuangtaoLyu for provide valuable advice on the installation steps.

If you don't have conda or miniconda, please install that first:

sudo apt-get update && \
sudo apt-get upgrade -y && \
sudo apt-get install unzip libeigen3-dev ffmpeg build-essential nvidia-cuda-toolkit

mkdir -p ~/miniconda3 && \
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh && \
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3 && \
rm -rf ~/miniconda3/miniconda.sh && \
~/miniconda3/bin/conda init bash && \
~/miniconda3/bin/conda init zsh

# close and reopen the shell
git clone https://github.com/River-Zhang/SIFU.git
sudo apt-get install libeigen3-dev ffmpeg
cd SIFU
conda env create -f environment.yaml
conda activate sifu
pip install -r requirements.txt

Please download the checkpoint (google drive) and place them in ./data/ckpt

Please follow ICON to download the extra data, such as HPS and SMPL (using fetch_hps.sh and fetch_data.sh). There may be missing files about SMPL, and you can download from here and put them in /data/smpl_related/smpl_data/.

Inference

python -m apps.infer -cfg ./configs/sifu.yaml -gpu 0 -in_dir ./examples -out_dir ./results -loop_smpl 100 -loop_cloth 200 -hps_type pixie

Testing

# 1. Register at http://icon.is.tue.mpg.de/ or https://cape.is.tue.mpg.de/
# 2. Download CAPE testset
bash fetch_cape.sh 

# evaluation
python -m apps.train -cfg ./configs/train/sifu.yaml -test

# TIP: the default "mcube_res" is 256 in apps/train.

Applications of SIFU

Scene Building

3D Printing

Texture Editing

Animation

In-the-wild Reconstruction