Neural-Style-Transfer-Papers

Selected papers, corresponding codes and pre-trained models in our review paper "Neural Style Transfer: A Review" [arXiv Version] [IEEE Version]

The corresponding OSF repository can be found at: https://osf.io/f8tu4/.

If I missed your paper in this review, please email me or just pull a request here. I am more than happy to add it. Thanks!

Citation

If you find this repository useful for your research, please consider citing

@article{jing2019neural,
  title={Neural Style Transfer: A Review},
  author={Jing, Yongcheng and Yang, Yezhou and Feng, Zunlei and Ye, Jingwen and Yu, Yizhou and Song, Mingli},
  journal={IEEE Transactions on Visualization and Computer Graphics},
  year={2019}
}

Please also consider citing our ECCV paper and AAAI (Oral) paper:

@inproceedings{jing2018stroke,
  title={Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields},
  author={Jing, Yongcheng and Liu, Yang and Yang, Yezhou and Feng, Zunlei and Yu, Yizhou and Tao, Dacheng and Song, Mingli},
  booktitle={ECCV},
  year={2018}
}

@inproceedings{jing2020dynamic,
  title={Dynamic Instance Normalization for Arbitrary Style Transfer},
  author={Jing, Yongcheng and Liu, Xiao and Ding, Yukang and Wang, Xinchao and Ding, Errui and Song, Mingli and Wen, Shilei},
  booktitle={AAAI},
  year={2020}
}

Thanks!

Framework

There is a recent nice NST framework called pystiche, developed by Philip Meier. If you are interested, please refer to https://github.com/pmeier/pystiche. A package that comprises reference implementations of NST papers with pystiche can be found at pystiche_papers (work in progress).

---

News!

• [June, 2019] Update the Images (TVCG) (.png) and Supplementary Material (TVCG) in the Materials. Warmly welcome to use Images (TVCG) for comparison results in your paper!

• [May, 2019] Our paper Neural Style Transfer: A Review has been accepted by TVCG as a regular paper. This repository will be updated soon.

• [July, 2018] Our paper Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields has been accepted by ECCV 2018. Our review will be updated correspondingly.

• [June, 2018] Upload a new version of our paper on arXiv which adds several missing papers (e.g., the work of Wang et al. ZM-Net: Real-time Zero-shot Image Manipulation Network).

• [Apr, 2018] We have released a new version of the paper with significant changes at: https://arxiv.org/pdf/1705.04058.pdf Appreciate the feedback!

• [Feb, 2018] Update the Images (Images_neuralStyleTransferReview_v2) in the Materials. Add the results of Li et al.'s NIPS 2017 paper.

• [Jan, 2018] Pre-trained models and all the content images, the style images, and the stylized results in the paper have been released.

Materials corresponding to Our Paper

:white_check_mark: Supplementary Material (TVCG)

:white_check_mark: Pre-trained Models

:white_check_mark: Images (TVCG)(.png)

A Taxonomy of Current Methods

1. Image-Optimisation-Based Online Neural Methods

1.1. Parametric Neural Methods with Summary Statistics

:white_check_mark: [A Neural Algorithm of Artistic Style] [Paper] (First Neural Style Transfer Paper)

:sparkle: Code:

• Torch-based
• TensorFlow-based
• TensorFlow-based with L-BFGS optimizer support
• Caffe-based
• Keras-based
• MXNet-based
• MatConvNet-based

:white_check_mark: [Image Style Transfer Using Convolutional Neural Networks] [Paper] (CVPR 2016)

:white_check_mark: [Incorporating Long-range Consistency in CNN-based Texture Generation] [Paper] (ICLR 2017)

:sparkle: Code:

•   Theano-based

:white_check_mark: [Laplacian-Steered Neural Style Transfer] [Paper] (ACM MM 2017)

:sparkle: Code:

• Torch-based & TensorFlow-based

:white_check_mark: [Demystifying Neural Style Transfer] [Paper] (Theoretical Explanation) (IJCAI 2017)

:sparkle: Code:

•   MXNet-based

:white_check_mark: [Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses] [Paper]

1.2. Non-parametric Neural Methods with MRFs

:white_check_mark: [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis] [Paper] (CVPR 2016)

:sparkle: Code:

• Torch-based

:white_check_mark: [Arbitrary Style Transfer with Deep Feature Reshuffle] [Paper] (CVPR 2018)

2. Model-Optimisation-Based Offline Neural Methods

2.1. Per-Style-Per-Model Neural Methods

:white_check_mark: [Perceptual Losses for Real-Time Style Transfer and Super-Resolution] [Paper] (ECCV 2016)

:sparkle: Code:

• Torch-based
• TensorFlow-based
• Chainer-based

:sparkle: Pre-trained Models:

• Torch-models
• Chainer-models

:white_check_mark: [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images] [Paper] (ICML 2016)

:sparkle: Code:

• Torch-based
• TensorFlow-based

:white_check_mark: [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks] [Paper] (ECCV 2016)

:sparkle: Code:

• Torch-based

2.2. Multiple-Style-Per-Model Neural Methods

:white_check_mark: [A Learned Representation for Artistic Style] [Paper] (ICLR 2017)

:sparkle: Code:

• TensorFlow-based

:white_check_mark: [Multi-style Generative Network for Real-time Transfer] [Paper]  （arXiv, 03/2017）

:sparkle: Code:

• PyTorch-based
• Torch-based

:white_check_mark: [Diversified Texture Synthesis With Feed-Forward Networks] [Paper] (CVPR 2017)

:sparkle: Code:

•   Torch-based

:white_check_mark: [StyleBank: An Explicit Representation for Neural Image Style Transfer] [Paper] (CVPR 2017)

2.3. Arbitrary-Style-Per-Model Neural Methods

:white_check_mark: [Fast Patch-based Style Transfer of Arbitrary Style] [Paper]

:sparkle: Code:

• Torch-based

:white_check_mark: [Exploring the Structure of a Real-time, Arbitrary Neural Artistic Stylization Network] [Paper] (BMVC 2017)

:sparkle: Code:

• TensorFlow-based

:white_check_mark: [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization] [Paper] (ICCV 2017)

:sparkle: Code:

• Torch-based
• TensorFlow-based with Keras
• TensorFlow-based without Keras

:white_check_mark: [Dynamic Instance Normalization for Arbitrary Style Transfer] [Paper] (AAAI 2020)

:white_check_mark: [Universal Style Transfer via Feature Transforms] [Paper] (NIPS 2017)

:sparkle: Code:

• Torch-based
• TensorFlow-based
• PyTorch-based #1
• PyTorch-based #2

:white_check_mark: [Meta Networks for Neural Style Transfer] [Paper] (CVPR 2018)

:sparkle: Code:

• Caffe-based

:white_check_mark: [ZM-Net: Real-time Zero-shot Image Manipulation Network] [Paper]

:white_check_mark: [Avatar-Net: Multi-Scale Zero-Shot Style Transfer by Feature Decoration] [Paper] (CVPR 2018)

:sparkle: Code:

• TensorFlow-based

:white_check_mark: [Learning Linear Transformations for Fast Arbitrary Style Transfer] [Paper]

:sparkle: Code:

• PyTorch-based

Improvements and Extensions

:white_check_mark: [Preserving Color in Neural Artistic Style Transfer] [Paper]

:white_check_mark: [Controlling Perceptual Factors in Neural Style Transfer] [Paper] (CVPR 2017)

:sparkle: Code:

• Torch-based

:white_check_mark: [Content-Aware Neural Style Transfer] [Paper]

:white_check_mark: [Towards Deep Style Transfer: A Content-Aware Perspective] [Paper] (BMVC 2016)

:white_check_mark: [Neural Doodle_Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artwork] [Paper]

:white_check_mark: [Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artwork] [Paper]

:sparkle: Code:

• Torch-based

:white_check_mark: [The Contextual Loss for Image Transformation with Non-Aligned Data] [Paper] (ECCV 2018)

:sparkle: Code:

• TensorFlow-based

:white_check_mark: [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis] [Paper] (CVPR 2017)

:sparkle: Code:

• Torch-based

:white_check_mark: [Instance Normalization：The Missing Ingredient for Fast Stylization] [Paper]

:sparkle: Code:

• Torch-based

:white_check_mark: [A Style-Aware Content Loss for Real-time HD Style Transfer] [Paper] (ECCV 2018)

:white_check_mark: [Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer] [Paper] (CVPR 2017)

:sparkle: Code:

• TensorFlow-based

:white_check_mark: [Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields] [Paper] (ECCV 2018)

:sparkle: Code:

• TensorFlow-based

:white_check_mark: [Depth-Preserving Style Transfer] [Paper]

:sparkle: Code:

• Torch-based

:white_check_mark: [Depth-Aware Neural Style Transfer] [Paper] (NPAR 2017)

:white_check_mark: [Neural Style Transfer: A Paradigm Shift for Image-based Artistic Rendering?] [Paper] (NPAR 2017)

:white_check_mark: [Pictory: Combining Neural Style Transfer and Image Filtering] [Paper] (ACM SIGGRAPH 2017 Appy Hour)

:white_check_mark: [Painting Style Transfer for Head Portraits Using Convolutional Neural Networks] [Paper] (SIGGRAPH 2016)

:white_check_mark: [Son of Zorn's Lemma Targeted Style Transfer Using Instance-aware Semantic Segmentation] [Paper] (ICASSP 2017)

:white_check_mark: [Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN] [Paper] (ACPR 2017)

:white_check_mark: [Artistic Style Transfer for Videos] [Paper] (GCPR 2016)

:sparkle: Code:

• Torch-based

:white_check_mark: [DeepMovie: Using Optical Flow and Deep Neural Networks to Stylize Movies] [Paper]

:white_check_mark: [Characterizing and Improving Stability in Neural Style Transfer] [Paper]) (ICCV 2017)

:white_check_mark: [Coherent Online Video Style Transfer] [Paper] (ICCV 2017)

:white_check_mark: [Real-Time Neural Style Transfer for Videos] [Paper] (CVPR 2017)

:white_check_mark: [A Common Framework for Interactive Texture Transfer] [Paper] (CVPR 2018)

:white_check_mark: [Deep Photo Style Transfer] [Paper] (CVPR 2017)

:sparkle: Code:

• Torch-based
• TensorFlow-based

:white_check_mark: [A Closed-form Solution to Photorealistic Image Stylization] [Paper] (ECCV 2018)

:sparkle: Code:

• PyTorch-based

:white_check_mark: [Photorealistic Style Transfer via Wavelet Transforms] [Paper]

:sparkle: Code:

• PyTorch-based

:white_check_mark: [Decoder Network Over Lightweight Reconstructed Feature for Fast Semantic Style Transfer] [Paper] (ICCV 2017)

:white_check_mark: [Stereoscopic Neural Style Transfer] [Paper] (CVPR 2018)

:white_check_mark: [Awesome Typography: Statistics-based Text Effects Transfer] [Paper] (CVPR 2017)

:sparkle: Code:

• Matlab-based

:white_check_mark: [Neural Font Style Transfer] [Paper] (ICDAR 2017)

:white_check_mark: [Rewrite: Neural Style Transfer For Chinese Fonts] [Project]

:white_check_mark: [Separating Style and Content for Generalized Style Transfer] [Paper] (CVPR 2018)

:white_check_mark: [Visual Attribute Transfer through Deep Image Analogy] [Paper] (SIGGRAPH 2017)

:sparkle: Code:

• Caffe-based

:white_check_mark: [Fashion Style Generator] [Paper] (IJCAI 2017)

:white_check_mark: [Deep Painterly Harmonization] [Paper]

:sparkle: Code:

• Torch-based

:white_check_mark: [Fast Face-Swap Using Convolutional Neural Networks] [Paper] (ICCV 2017)

:white_check_mark: [Learning Selfie-Friendly Abstraction from Artistic Style Images] [Paper] (ACML 2018)

:white_check_mark: [Style Transfer with Adaptation to the Central Objects of the Scene] [Paper] (NEUROINFORMATICS 2019)

Application

:white_check_mark: Prisma

:white_check_mark: Ostagram

:white_check_mark: AlterDraw

:white_check_mark: Vinci

:white_check_mark: Artisto

:sparkle: Code:

• Website code

:white_check_mark: Deep Forger

:white_check_mark: NeuralStyler

:white_check_mark: Style2Paints

:sparkle: Code:

• Website code

Application Papers

:white_check_mark: [Bringing Impressionism to Life with Neural Style Transfer in Come Swim] [Paper]

:white_check_mark: [Imaging Novecento. A Mobile App for Automatic Recognition of Artworks and Transfer of Artistic Styles] [Paper]

:white_check_mark: [ProsumerFX: Mobile Design of Image Stylization Components] [Paper]

:white_check_mark: [Pictory - Neural Style Transfer and Editing with coreML] [Paper]

:white_check_mark: [Tiny Transform Net for Mobile Image Stylization] [Paper] (ICMR 2017)

Blogs

:white_check_mark: [Caffe2Go][https://code.facebook.com/posts/196146247499076/delivering-real-time-ai-in-the-palm-of-your-hand/]

:white_check_mark: [Supercharging Style Transfer][https://research.googleblog.com/2016/10/supercharging-style-transfer.html]

:white_check_mark: [Issue of Layer Chosen Strategy][http://yongchengjing.com/pdf/Issue_layerChosenStrategy_neuralStyleTransfer.pdf]

:white_check_mark: [Picking an optimizer for Style Transfer][https://blog.slavv.com/picking-an-optimizer-for-style-transfer-86e7b8cba84b]

:white_check_mark: [Enhanced Color Style Transfer (Photo-surrealism Style Transfer)] [Project]

Others

:white_check_mark: [Conditional Fast Style Transfer Network] [Paper]

:white_check_mark: [Unseen Style Transfer Based on a Conditional Fast Style Transfer Network] [Paper]

:white_check_mark: [DeepStyleCam: A Real-time Style Transfer App on iOS] [Paper]

:white_check_mark: [Deep Feature Rotation for Multimodal Image Style Transfer] [Paper]  (NICS 2021)

:sparkle: Code:

• TensorFlow-based