VAE and Super-Resolution VAE in PyTorch

Code release for Super-Resolution Variational Auto-Encoders

Abstract

The framework of Variational Auto-Encoders (VAEs) provides a principled manner of reasoning in latent-variable models using variational inference. However, the main drawback of this approach is blurriness of generated images. Some studies link this effect to the objective function, namely, the (negative) log-likelihood function. Here, we propose to enhance VAEs by adding a random variable that is a downscaled version of the original image and still use the log-likelihood function as the learning objective. Further, we provide the downscaled image as an input to the decoder and use it in a manner similar to the super-resolution. We present empirically that the proposed approach performs comparably to VAEs in terms of the negative log-likelihood function, but it obtains a better FID score.

Features

• Models

  • VAE
  • Super-resolution VAE (srVAE)

• Priors

  • Standard (unimodal) Gaussian
  • Mixture of Gaussians
  • RealNVP

• Reconstruction Loss

  • Discretized Mixture of Logistics Loss

• Neural Networks

  • DenseNet

• Datasets

  • CIFAR-10

Quantitative results

Qualitative results

VAE

Results from VAE with RealNVP Prior trained on CIFAR10.

Interpolations

Reconstructions.

Unconditional generations.

Super-Resolution VAE

Results from Super-Resolution VAE trained on CIFAR10.

Interpolations

Super-Resolution results of the srVAE on CIFAR-10

Unconditional generations. Left: The generations of the first step, the compressed representations that capture the _global_ structure. Right: The final result after enhasing the images with local content.

Requirements

The code is compatible with:

• python 3.6
• pytorch 1.3

Usage

• To run VAE with RealNVP prior on CIFAR-10, please execude:

python main.py --model VAE --network densenet32 --prior RealNVP

• Otherwise, to run srVAE:

python main.py --model srVAE --network densenet16x32 --prior RealNVP

Cite

Please cite our paper if you use this code in your own work:

@misc{gatopoulos2020superresolution,
    title={Super-resolution Variational Auto-Encoders},
    author={Ioannis Gatopoulos and Maarten Stol and Jakub M. Tomczak},
    year={2020},
    eprint={2006.05218},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

Acknowledgements

This work was supported and funded from the University of Amsterdam, and BrainCreators B.V..

Repo Author

Ioannis Gatopoulos, 2020