Adventures in deep learning

Deep Learning publications, frameworks & resources

Overview

Deep convolutional neural networks have led to a series of breakthroughs in large-scale image and video recognition. This repository aims at presenting an elaborate list of the latest works on the field of Deep Learning since 2013.

This is going to be an evolving repository and I will keep updating it (at least twice monthly).

State-of-the-art papers (Descending order based on Google Scholar Citations)

1. Very deep convolutional networks for large-scale image recognition (VGG-net) (2014) [pdf] [video]
2. Going deeper with convolutions (GoogLeNet) by Google (2015) [pdf] [video]
3. Deep learning (2015) [pdf]
4. Visualizing and Understanding Convolutional Neural Networks (ZF Net) (2014) [pdf] [video]
5. Fully convolutional networks for semantic segmentation (2015) [pdf]
6. Deep residual learning for image recognition (ResNet) by Microsoft (2015) [pdf] [video]
7. Deepface: closing the gap to human-level performance in face verification (2014) [pdf] [video]
8. Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015) [pdf]
9. Deep Learning in Neural Networks: An Overview (2015) [pdf]
10. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification (PReLU) (2014) [pdf]
11. Faster R-CNN: Towards real-time object detection with region proposal networks (2015) [pdf]
12. Fast R-CNN (2015) [pdf]
13. Spatial pyramid pooling in deep convolutional networks for visual recognition (SPP Net) (2014) [pdf] [video]
14. Generative Adversarial Nets (2014) [pdf]
15. Spatial Transformer Networks (2015) [pdf] [video]
16. Understanding deep image representations by inverting them (2015) [pdf]
17. Deep Learning of Representations: Looking Forward (2013) [pdf]

Classic publications

• ImageNet Classification with Deep Convolutional Neural Networks (AlexNet) (2012) [pdf]
• Rectified linear units improve restricted boltzmann machines (ReLU) (2010) [pdf]

Theory

1. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images (2015) [pdf]
2. Distilling the Knowledge in a Neural Network (2015) [pdf]
3. Deep learning in neural networks: An overview (2015) [pdf]

Books

• Deep Learning Textbook - An MIT Press book (2016) [html]
• Learning Deep Architectures for AI [pdf]
• Neural Nets and Deep Learning [html] [github]

Notes from Coursera DeepLearning courses by Andrew Ng, created by @TessFerrandez

• Download from here

Courses / Tutorials (Webpages unless other is stated)

• Caffe Tutorial (CVPR 2015)
• Tutorial on Deep Learning for Vision (CVPR 2014)
• Introduction to Deep Learning with Python - Theano Tutorials [github]
• Deep Learning Tutorials with Theano/Python [github]
• Deep Learning: Take machine learning to the next level (by udacity)
• DeepLearnToolbox – A Matlab toolbox for Deep Learning [github]
• Stanford Matlab-based Deep Learning [github]
• Stanford 231n Class: Convolutional Neural Networks for Visual Recognition [github]
• Deep Learning Course (by Yann LeCun-2016)
• Generative Models (by OpenAI)
• An introduction to Generative Adversarial Networks (with code in TensorFlow)

Resources / Models (GitHub repositories unless other is stated)

• VGG-net
• GoogLeNet
• ResNet - MatConvNet implementation
• AlexNet
• Fully Convolutional Networks for Semantic Segmentation
• OverFeat
• SPP_net
• Fast R-CNN
• Faster R-CNN
• Generative Adversarial Networks (GANs)
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks)
• ResNeXt: Aggregated Residual Transformations for Deep Neural Networks)
• MultiPath Network training code

Frameworks & Libraries (Descending order based on GitHub stars)

• Tensorflow by Google [C++ and CUDA]: [homepage] [github]
• Caffe by Berkeley Vision and Learning Center (BVLC) [C++]: [homepage] [github] [Installation Instructions]
• Keras by François Chollet [Python]: [homepage] [github]
• Microsoft Cognitive Toolkit - CNTK [C++]: [homepage] [github]
• MXNet adapted by Amazon [C++]: [homepage] [github]
• Torch by Collobert, Kavukcuoglu & Clement Farabet, widely used by Facebook [Lua]: [homepage] [github]
• Convnetjs by Andrej Karpathy [JavaScript]: [homepage] [github]
• Theano by Université de Montréal [Python]: [homepage] [github]
• Deeplearning4j by startup Skymind [Java]: [homepage] [github]
• Paddle by Baidu [C++]: [homepage] [github]
• Deep Scalable Sparse Tensor Network Engine (DSSTNE) by Amazon [C++]: [github]
• Neon by Nervana Systems [Python & Sass]: [homepage] [github]
• Chainer [Python]: [homepage] [github]
• h2o [Java]: [homepage] [github]
• Brainstorm by Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA) [Python]: [github]
• Matconvnet by Andrea Vedaldi [Matlab]: [homepage] [github]

License

MIT