Computer Vision has become ubiquitous in our society, with applications in search, image understanding, apps, mapping, medicine, drones, and self-driving cars. Core to many of these applications are visual recognition tasks such as image classification, localization and detection. Recent developments in neural network (aka “deep learning”) approaches have greatly advanced the performance of these state-of-the-art visual recognition systems. This course is a deep dive into details of the deep learning architectures with a focus on learning end-to-end models for these tasks, particularly image classification. During the 10-week course, students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in computer vision. The final assignment will involve training a multi-million parameter convolutional neural network and applying it on the largest image classification dataset (ImageNet). We will focus on teaching how to set up the problem of image recognition, the learning algorithms (e.g. backpropagation), practical engineering tricks for training and fine-tuning the networks and guide the students through hands-on assignments and a final course project. Much of the background and materials of this course will be drawn from the ImageNet Challenge.

Table of Content

Lectures

• Lecture 1: Course Introduction (Video, Slides)
• Lecture 2: Image Classification (Video, Slides + Tutorials)
• Lecture 3: Loss Function and Optimizations (Video, Slides + Tutorials)
• Lecture 4: Introduction to Neural Networks (Video, Slides + Readings)
• Lecture 5: Convolutional Neural Networks (Video, Slides + Tutorials)
• Lecture 6: Training Neural Networks Part I (Video, Slides + Readings + Tutorials)
• Lecture 7: Training Neural Networks Part II (Video, Slides + Tutorials)
• Lecture 8: Deep Learning Software (Video, Slides)
• Lecture 9: CNN Architectures (Video, Slides + Readings)
• Lecture 10: Recurrent Neural Networks (Video, Slides)
• Lecture 11: Detection and Segmentation (Video, Slides)
• Lecture 12: Visualizing and Understanding (Video, Slides)
• Lecture 13: Generative Models (Video, Slides)
• Lecture 14: Deep Reinforcement Learning (Video, Slides)
• Lecture 15: Efficient Methods and Hardware for Deep Learning (Video, Slides)
• Lecture 16: Adversarial Examples and Adversarial Training (Video, Slides)

Assignments

• Assignment 1 (Problem, Solution)
• Assignment 2 (Problem, Solution)
• Assignment 3 (Problem, Solution)