Course Calendar
Download lecture slides by clicking on the corresponding lecture title. Lecture topics (right) link to suggested readings below.
Beginning at 2:30pm in CIT 368.
Suggested Readings
Acronyms for Primary Resources
- BRML: Bayesian Reasoning and Machine Learning, David Barber, Cambridge University Press 2012. Free online.
- MLaPP: Machine Learning: A Probabilistic Perspective, Kevin Murphy, MIT Press 2012. Excerpt online.
- PRML: Pattern Recognition and Machine Learning, Christopher Bishop, Springer 2007. Excerpt online.
- GEV: Graphical Models, Exponential Families, and Variational Inference, Martin Wainwright & Michael Jordan, Foundations & Trends in Machine Learning, 2008.
- EBS: Graphical Models for Visual Object Recognition and Tracking, Erik B. Sudderth, PhD Thesis (Chapter 2), MIT 2006.
Graphical Model Tutorials
Directed & Undirected Graphs: Factorization & Markov Properties
- BRML: Chapters 2-4, excluding Sec. 3.4.
- MLaPP: Sec. 10.1-10.2, 10.5, 19.1-19.4.
- PRML: Sec. 8.1-8.3.
- GEV: Sec. 2.1-2.4.
- EBS: Sec. 2.2.1-2.2.3.
Inference via Variable Elimination
Inference via Belief Propagation: Sum-Product & Max-Product
Inference via Junction Tree Propagation
Exponential Family Distributions: Learning & Inference
- BRML: Chapter 8 excluding Sec. 8.4, 8.8.
- MLaPP: Sec. 9.2.
- PRML: Sec. 2.4.
- GEV: Chapter 3.
- EBS: Sec. 2.1.
Learning (Directed & Undirected) Graphical Model Parameters
- BRML: Chapter 9 excluding Sec. 9.5.
- MLaPP: Sec. 10.4, 19.5.
- GEV: Sec. 6.1.
Learning via the Expectation Maximization (EM) Algorithm
Learning (Directed & Undirected) Graphical Model Structure
- BRML: Sec. 9.5-9.6, 12.1-12.3, 12.5.
- MLaPP: Chapter 26 excluding Sec. 26.5-26.7.
Inference & Learning for Gaussian Graphical Models
- BRML: Sec. 8.4, Chapter 24.
- MLaPP: Sec. 10.2.5, 18.1-18.4, 19.4.4, 20.2.3, 26.7.
- PRML: Sec. 13.3.
- Embedded Trees: Estimation of Gaussian Processes on Graphs with Cycles (Chapter 2), E. Sudderth, MIT SM Thesis 2002.
- A Unifying Review of Linear Gaussian Models, S. Roweis & Z. Ghahramani, Neural Computation 1999.
- Bayesian Modeling of Uncertainty in Low-Level Vision, R. Szeliski, IJCV 1990.
Monte Carlo Methods: Rejection & Importance Sampling
Particles & Sequential Monte Carlo
- BRML: Sec. 27.6.
- MLaPP: Sec. 23.5.
- An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo, Cappe, Godsill, & Moulines, IEEE 2007.
- Nonparametric Belief Propagation, Sudderth, Ihler, Isard, Freeman, & Willsky, CACM 2010.
Markov Chain Monte Carlo (MCMC): Gibbs & Metropolis-Hastings
Variational Methods: Naive & Structured Mean Field
Variational Methods: Bethe Approximations, Loopy & Reweighted BP
Discriminative Learning: Conditional Random Fields & Structural SVMs
Neural Networks & Deep Learning
Bayesian Nonparametrics: Dirichlet Processes