Variational-Sparse-Coding

We aim to replicate the experiments described in the paper "Variational Sparse Coding" from the ICLR 2019 submissions, as part of our participation in the ICLR Reproducibility Challenge 2019.

Table of content

• Description
• Authors
• Results
• Usage
• References
• Acknowledgements
• License

Description

We replicate the results of the recent paper Variational Sparse Coding and extend the results to new experiments.

Authors

• Alfredo de la Fuente
• Robert Aduviri

Results

Latent Space Traversal Generation (200 dimensions - = 0.01)

MNIST	Fashion-MNIST

CelebA	dSprites

Playing with low-dimensional latent space variables

Reconstruction results by modifying encoding in 200-dimensional latent space

Varying latent space dimensionality

MNIST	Fashion-MNIST

Conditional Sampling

Usage

Setup

Requires Python 3.6 or higher.

The following lines will clone the repository and install all the required dependencies.

$ https://github.com/Alfo5123/Variational-Sparse-Coding.git
$ cd Variational-Sparse-Coding
$ pip install -r requirements.txt

Datasets

In order to download datasets used in the paper experiments we use

$ python setup.py [datasets...]

with options mnist, fashion and celeba. For example, if case you want to replicate all the experiments in the paper, we must run the following line:

$ python setup.py mnist fashion celeba

It will download and store the datasets locally in the data folder.

Pretrained Models

Aiming to simplify the reproducibility research process, we store the checkpoints of the trained models in the following link. In order to run the scripts & notebooks using pretrained models, you must download the checkpoints and put them in models within the src folder.

Train Models

$ cd src
$ python [model] [args] 

For example

$ python train-vae.py --dataset mnist --epochs 500 --report-interval 50 --lr 0.01 

$ python train-vsc.py --dataset celeba --epochs 20 --report-interval 4 --lr 0.001 --alpha 0.2 --hidden-size 2000,2000 --latent-size 800

To visualize training results in TensorBoard, we can use the following command from a new terminal within src folder.

$ tensorboard --logdir='./logs' --port=6006

References

Papers:

• Variational Sparse Coding
• Auto-Encoding Variational Bayes
• Large-Scale Feature Learning With Spike-and-Slab Sparse Coding
• Stick-Breaking Variational Autoencoders
• beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
• Disentangling by Factorising
• Neural Discrete Representation Learning
• Tutorial on Variational Autoencoders

Acknowledgements

Special thanks to Emilien Dupont for clarifying distinct aspects on variational autoencoders' implementation, and Lukas Mosser for his suggestions on training generative models.

License

MIT License