(ICCV 2021) Code for "Unsupervised Layered Image Decomposition into Object Prototypes" paper
Pytorch implementation of "Unsupervised Layered Image Decomposition into Object Prototypes" paper (accepted to ICCV 2021)
Check out our paper and webpage for details!
If you find this code useful, don't forget to star the repo ⭐ and cite the paper:
@inproceedings{monnier2021dtisprites,
title={{Unsupervised Layered Image Decomposition into Object Prototypes}},
author={Monnier, Tom and Vincent, Elliot and Ponce, Jean and Aubry, Mathieu},
booktitle={ICCV},
year={2021},
}
conda env create -f environment.yml
conda activate dti-sprites
Optional: some monitoring routines are implemented, you can use them by specifying the
visdom port in the config file. You will need to install visdom
from source beforehand
git clone https://github.com/facebookresearch/visdom
cd visdom && pip install -e .
./download_data.sh
This command will download following datasets:
Tetrominoes
, Multi-dSprites
and CLEVR6
(link to the original repo
multi-object datasets with raw tfrecords)GTSRB
(link to the original dataset
page)Weizmann Horse database
(link
to the original dataset page)Instagram collections
associated to
#santaphoto and
#weddingkiss
(link to the original repo with
datasets links and descriptions)NB: it may happen that gdown
hangs, if so you can download them by hand with following
gdrive links, unzip and move them to the datasets
folder:
cuda=gpu_id config=filename.yml tag=run_tag ./pipeline.sh
where:
gpu_id
is a target cuda device id,filename.yml
is a YAML config located in configs
folder,run_tag
is a tag for the experiment.Results are saved at runs/${DATASET}/${DATE}_${run_tag}
where DATASET
is the dataset name
specified in filename.yml
and DATE
is the current date in mmdd
format. Some training
visual results like sprites evolution and reconstruction examples
will be saved. Here is an example from Tetrominoes dataset:
More visual results are available at https://imagine.enpc.fr/~monniert/DTI-Sprites/extra_results/.
To launch 5 runs on Tetrominoes benchmark and reproduce our results:
cuda=gpu_id config=tetro.yml tag=default ./multi_pipeline.sh
Available configs are:
tetro.yml
, dsprites.yml
, clevr6.yml
gtsrb8.yml
, svhn.yml
horse.yml
process_insta_santa.sh
script. It can take a while to scrape the 10k posts from
Instagram.cuda=gpu_id config=instagram.yml tag=santaphoto ./pipeline.sh
That's it! Overall you should end up with the following qualitative results:
If you like this project, please check out related works on deep transformations from our group: