Official Code and Dataset for "High-fidelity 3D Human Digitization from Single 2K Resolution Images" (CVPR 2023 Highlight)
This repository contains the code of the 2K2K method for 3D human reconstruction.
Sang-Hun Han,
Min-Gyu Park,
Ju Hong Yoon,
Ju-Mi Kang,
Young-Jae Park, and
Hae-Gon Jeon
Accepted to
CVPR 2023
Paper | Project Page | Dataset
apt-get install -y freeglut3-dev libglib2.0-0 libsm6 libxrender1 libxext6 openexr libopenexr-dev libjpeg-dev zlib1g-dev
apt install -y libgl1-mesa-dri libegl1-mesa libgbm1 libgl1-mesa-glx libglib2.0-0
pip install -r requirements.txt
docker build -t 2k2k:1.0 .
docker run -e NVIDIA_VISIBLE_DEVICES=all -i -t -d --runtime=nvidia --shm-size=512gb --name 2k2k --mount type=bind,source={path/to/2k2k_code},target=/workspace/code 2k2k:1.0 /bin/bash
data
├ IndoorCVPR09
│ ├ airport_inside
│ └ ⋮
├ Joint3D
│ ├ RP
│ └ THuman2
├ list
├ obj
│ ├ RP
│ │ ├ rp_aaron_posed_013_OBJ
│ │ └ ⋮
│ └ THuman2
│ ├ data
│ └ smplx
├ PERS
│ ├ COLOR
│ ├ DEPTH
│ └ keypoint
└ (ORTH)
data/IndoorCVPR09/
.data/obj
first. See the folder structure above for download locations.PERS
and ORTH
(Optional). It takes about 2-3 days to render a 2048×2048 resolution images.python render/render.py --data_path ./data --data_name RP
For THuman2.0 dataset, you should use the SMPL-X model to render the front of human scans. Please download SMPL-X models anywhere. The smplx
folder should exist under the {smpl_model_path}
.
After download SMPL-X models, you can render images.
python render/render.py --data_path ./data --data_name THuman2 --smpl_model_path {smpl_model_path}
Joint3D.zip
under data/Joint3D
.unzip data/Joint3D.zip -d data/Joint3D/
python render/render_keypoint.py --data_path ./data --data_name RP
python render/render_keypoint.py --data_path ./data --data_name THuman2
python train.py --data_path ./data --phase 1 --batch_size 1
python train.py --data_path ./data --phase 2 --batch_size 1 --load_ckpt {checkpoint_file_name}
argparse
in train.py
manually.CUDA_VISIBLE_DEVICES=0,1 \
python -m torch.distributed.run \
--nnodes=1 \
--nproc_per_node=4 \
--rdzv_backend=c10d \
train.py --use_ddp=True
json
file.json
keypoint file. Please refer ./test
folder.bin\OpenPoseDemo.exe --image_dir {test_folder} --write_json {test_folder} --hand --write_images {test_folder}\test --write_images_format jpg
cd checkpoints && wget https://github.com/SangHunHan92/2K2K/releases/download/Checkpoint/ckpt_bg_mask.pth.tar && cd ..
ply
.python test_02_model.py --load_ckpt {checkpoint_file_name} --save_path {result_save_folder}
python test_03_poisson.py --save_path {result_save_folder}
@inproceedings{han2023high,
title={High-fidelity 3D Human Digitization from Single 2K Resolution Images},
author={Han, Sang-Hun and Park, Min-Gyu and Yoon, Ju Hong and Kang, Ju-Mi and Park, Young-Jae and Jeon, Hae-Gon},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2023}
}