(CVPR'20 Oral) Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild
Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi, Visual Geometry Group, University of Oxford. In CVPR 2020 (Best Paper Award).
We propose a method to learn weakly symmetric deformable 3D object categories from raw single-view images, without ground-truth 3D, multiple views, 2D/3D keypoints, prior shape models or any other supervision.
conda env create -f environment.yml
OR manually:
conda install -c conda-forge scikit-image matplotlib opencv moviepy pyyaml tensorboardX
conda install pytorch==1.2.0 torchvision==0.4.0 cudatoolkit=9.2 -c pytorch
Note: The code is tested with PyTorch 1.2.0 and CUDA 9.2 on CentOS 7. A GPU version is required for training and testing, since the neural_renderer package only has GPU implementation. You are still able to run the demo without GPU.
This package is required for training and testing, and optional for the demo. It requires a GPU device and GPU-enabled PyTorch.
pip install neural_renderer_pytorch
Note: It may fail if you have a GCC version below 5. If you do not want to upgrade your GCC, one alternative solution is to use conda's GCC and compile the package from source. For example:
conda install gxx_linux-64=7.3
git clone https://github.com/daniilidis-group/neural_renderer.git
cd neural_renderer
python setup.py install
This package is optional for the demo. It allows automatic human face detection.
pip install facenet-pytorch
img_celeba.7z
) from their website and run celeba_crop.py
in data/
to crop the images.download_synface.sh
provided in data/
.download_cat.sh
provided in data/
.download_syncar.sh
provided in data/
.Please remember to cite the corresponding papers if you use these datasets.
Download pretrained models using the scripts provided in pretrained/
, eg:
cd pretrained && sh download_pretrained_celeba.sh
python -m demo.demo --input demo/images/human_face --result demo/results/human_face --checkpoint pretrained/pretrained_celeba/checkpoint030.pth
Options:
--gpu
: enable GPU--detect_human_face
: enable automatic human face detection and cropping using MTCNN provided in facenet-pytorch. This only works on human face images. You will need to manually crop the images for other objects.--render_video
: render 3D animations using neural_renderer (GPU is required)Check the configuration files in experiments/
and run experiments, eg:
python run.py --config experiments/train_celeba.yml --gpu 0 --num_workers 4
@InProceedings{Wu_2020_CVPR,
author = {Shangzhe Wu and Christian Rupprecht and Andrea Vedaldi},
title = {Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild},
booktitle = {CVPR},
year = {2020}
}