CS:228 - Probabilistic Graphical Models

PGM ! PGM ! PGM ! One of the most interesting class yet challenging at Stanford is CS228. Graphical Models ahoi!, There's also an online preview of the course, here or here , only the overview lecture though. The course heavily follows Daphne Koller's book Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman., and There's also an online version of "Probabilistic Graphical Models" on Coursera. I completed the online version as a freshman, and here I take CS228 again. I got enrolled on Gradescope, though I am not taking the course in person. Maintain the honor code, keep learning. This is exciting!

Probabilistic graphical models are a powerful framework for representing complex domains using probability distributions, with numerous applications in machine learning, computer vision, natural language processing and computational biology. Graphical models bring together graph theory and probability theory, and provide a flexible framework for modeling large collections of random variables with complex interactions. This course will provide a comprehensive survey of the topic, introducing the key formalisms and main techniques used to construct them, make predictions, and support decision-making under uncertainty.

The aim of this course is to develop the knowledge and skills necessary to design, implement and apply these models to solve real problems. The course will cover:

• (1) Bayesian networks, undirected graphical models and their temporal extensions
• (2) exact and approximate inference methods
• (3) estimation of the parameters and the structure of graphical models.

BOOK : Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman.

𓄆 Important Books :
𓊖 Modeling and Reasoning with Bayesian
𓊖 Information Theory, Inference, and Learning Algorithms
𓊖 Machine Learning A Probabilistic Perspective
𓊖 Bayesian Reasoning and Machine Learning by David Barber
𓊖 Graphical models, exponential families, and variational inference

Homework (70%) + Final Exam (30%) | Homework - Theoretical + Programming | Topics in book

Homeworks

• Homework 1 : Programming
• Homework 2 : Programming
• Homework 3 : Programming
• Homework 4 : Programming
• Homework 5 : Programming

COURSE

♞ PRELIMINARIES

• Introduction: What is probabilistic graphical modeling? Overview of the course.
  
• Review of probability theory: Probability distributions. Conditional probability. Random variables (under construction).
  
• Examples of real-world applications: Image denoising. RNA structure prediction. Syntactic analysis of sentences. Optical character recognition (under construction).
  

♞ REPRESENTATION

• Bayesian networks: Definitions. Representations via directed graphs. Independencies in directed models.
  
• Markov random fields: Undirected vs directed models. Independencies in undirected models. Conditional random fields.
  

♞ INFERENCE

• Variable elimination: The inference problem. Variable elimination. Complexity of inference.
  
• Belief propagation: The junction tree algorithm. Exact inference in arbitrary graphs. Loopy Belief Propagation.
  
• MAP inference: Max-sum message passing. Graphcuts. Linear programming relaxations. Dual decomposition.
  
• Sampling-based inference: Monte-Carlo sampling. Importance sampling. Markov Chain Monte-Carlo. Applications in inference.
  
• Variational inference: Variational lower bounds. Mean Field. Marginal polytope and its relaxations.
  

♞ LEARNING

• Learning in directed models: Maximum likelihood estimation. Learning theory basics. Maximum likelihood estimators for Bayesian networks.
  
• Learning in undirected models: Exponential families. Maximum likelihood estimation with gradient descent. Learning in CRFs
  
• Learning in latent variable models: Latent variable models. Gaussian mixture models. Expectation maximization.
  
• Bayesian learning: Bayesian paradigm. Conjugate priors. Examples (under construction).
  
• Structure learning: Chow-Liu algorithm. Akaike information criterion. Bayesian information criterion. Bayesian structure learning (under construction).
  

♞ BRINGING IT ALL TOGETHER

• The variational autoencoder: Deep generative models. The reparametrization trick. Learning latent visual representations.
  
• List of further readings: Structured support vector machines. Bayesian non-parametrics.
  

유 PGM - Max Planck Institute for Intelligent Systems - Christopher Bishop | CMU - PGM | Probabilistic Graphical Models Tutorial — Part 1 | Understanding Probabilistic Graphical Models Intuitively | CMU - PGM - website | PGM | libDAI | OpenGM

FINAL EXAM

2016 Final , 2009 Final , 2008 Final , 2007 Final, 2006 Final | Collected from public resources | My Solution - HOMEWORKS , EXAMS | Collected from public resources