Valerio Maggio: PostDoc Data Scientist @ FBK/MPBA

Contacts:

@leriomaggio

[email protected]

Installed Versions

import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)

keras:  2.0.4
Theano:  0.9.0
Tensorflow:  1.2.1

Outline

• Part I: Introduction

  • Intro to Artificial Neural Networks

    • Perceptron and MLP
    • naive pure-Python implementation
    • fast forward, sgd, backprop

  • Introduction to Deep Learning Frameworks

    • Intro to Theano
    • Intro to Tensorflow
    • Intro to Keras
      • Overview and main features
      • Overview of the core layers
      • Multi-Layer Perceptron and Fully Connected
        • Examples with keras.models.Sequential and Dense
      • Keras Backend

• Part II: Supervised Learning

  • Fully Connected Networks and Embeddings

    • Intro to MNIST Dataset
    • Hidden Leayer Representation and Embeddings

  • Convolutional Neural Networks

    • meaning of convolutional filters

      • examples from ImageNet

    • Visualising ConvNets

    • Advanced CNN

      • Dropout
      • MaxPooling
      • Batch Normalisation

    • HandsOn: MNIST Dataset

      • FC and MNIST
      • CNN and MNIST

    • Deep Convolutiona Neural Networks with Keras (ref: keras.applications)

      • VGG16
      • VGG19
      • ResNet50

  • Transfer Learning and FineTuning

  • Hyperparameters Optimisation

• Part III: Unsupervised Learning

  • AutoEncoders and Embeddings
  • AutoEncoders and MNIST
    • word2vec and doc2vec (gensim) with keras.datasets
    • word2vec and CNN

• Part IV: Recurrent Neural Networks

  • Recurrent Neural Network in Keras
    • SimpleRNN, LSTM, GRU
  • LSTM for Sentence Generation

• PartV: Additional Materials:

  • Custom Layers in Keras
  • Multi modal Network Topologies with Keras

Valerio Maggio: PostDoc Data Scientist @ FBK/MPBA

Contacts:

@leriomaggio

[email protected]

Library Versions

import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)

keras:  2.0.2
Theano:  0.9.0
Tensorflow:  1.0.1

Goal

• Introduce main features of Keras APIs to build Neural Networks.
• Learn how to implement simple and complex Deep Neural Networks Architectures using Keras.
• Discover Keras Implementation and Internals.
  • Note: examples and hands-on exercises will be provided along the way.

Outline in Ten (~ish) Notebooks

1. Multi-layer Fully Connected Networks (and the backends)
2. Hidden Layers features and Embeddings
3. Convolutional Networks
4. Hyperparameter Tuning
5. Cutsom Layers
6. Deep CNN and Residual Networks
7. Transfer Learning and Fine Tuning
8. Recursive Neural Networks
9. AutoEncoders
10. Multi-Modal Networks

Valerio Maggio: PostDoc Data Scientist @ FBK/MPBA

Contacts:

@leriomaggio

[email protected]

Library Versions

import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)

keras:  2.0.2
Theano:  0.9.0
Tensorflow:  1.0.1

Outline at a glance

• Part I: Introduction to ANN using Tensorflow and Keras

  • naive pure-Python implementation
  • fast forward, sgd, backprop
  • Model + SGD using Tensorflow
  • Introduction to Keras main features
    • keras.layers.core.Dense
    • keras.backend
    • Multi-Layer Perceptron and Fully Connected Networks

• Part II: Supervised Learning and Convolutional Neural Nets

  • Intro: Focus on Image Classification
  • Intro to ConvNets
  • Advanced CNN
    • Dropout
    • MaxPooling
    • Batch Normalisation
  • Famous Models in Keras (ref: keras.applications)
  • Transfer Learning

• Part III: Unsupervised Learning

  • AutoEncoders
  • word2vec & doc2vec (gensim) & keras.datasets
    • Embedding

• Part IV: Additional Materials

  • Recurrent Neural Networks: RNN, LSTM, GRU
  • HandsOn: IMDB
  • Multi-Input/Multi-Output Network Topologies

Authors: Yam Peleg, Valerio Maggio

• Introduce main features of Keras
• Learn how simple and Pythonic is doing Deep Learning with Keras
• Understand how easy is to do basic and advanced DL models in Keras;
  • Examples and Hand-on Excerises along the way.

Installed Versions

import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)

keras:  1.0.7
Theano:  0.8.2
Tensorflow:  0.10.0rc0

Valerio Maggio: PostDoc Data Scientist @ FBK/MPBA

Contacts:

@leriomaggio

[email protected]

Goal of this Tutorial

• Introduce main features of Keras

  • Plus some introductory overview of Tensorflow

• Learn how simple and Pythonic is doing Deep Learning with Keras

• Understand how easy is to do basic and advanced Deep Learning models in Keras;

  • Examples and Hand-on Excerises along the way.

Installed Versions

import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)

keras:  1.0.7
Theano:  0.8.2
Tensorflow:  0.10.0

Valerio Maggio: PostDoc Data Scientist @ FBK/MPBA

Contacts:

@leriomaggio

[email protected]

Library Versions

import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)

keras:  1.0.7
Theano:  0.8.2
Tensorflow:  0.10.0

Goal of this Tutorial

• Introduce main features of Keras

  • Plus some introductory overview of Tensorflow

• Learn how simple and Pythonic is doing Deep Learning with Keras

• Understand how easy is to do basic and advanced Deep Learning models in Keras;

  • Examples and Hand-on Excerises along the way.