### Course Overview &

Directed Models

September 8, 2016

Download lecture slides by clicking on the corresponding lecture title. Lecture topics (right) link to suggested readings below.

Directed Models

September 8, 2016

Directed Models

September 13, 2016

September 15, 2016

September 20, 2016

September 22, 2016

September 27, 2016

EM Algorithm

September 29, 2016

October 4, 2016

Directed Graphs

October 6, 2016

October 11, 2016

*Begin to form your project team & define your project area.*

Monte Carlo

October 13, 2016

October 18, 2016

October 20, 2016

October 25, 2016

Nonparametric Belief Propagation, CACM 2010.

Diverse Particle Max-Product, ICML 2014.

*Guest lecturer:* Jason Pacheco

Gibbs Samplers

October 27, 2016

November 1, 2016

November 3, 2016

November 8, 2016

November 10, 2016

November 15, 2016

November 17, 2016

November 22, 2016

(No Lecture)

November 24, 2016

November 29, 2016

December 1, 2016

*Special location: CIT 477*

Hierarchial Dirichlet Processes

Variational & Monte Carlo Inference

December 13, 2016

*Beginning at 2:30pm in CIT 368.*

We provide suggested readings from several sources. You do *not* need to read all of them. Instead, we suggest you compare various options, and choose the resource whose style you like best. Barber's Bayesian Reasoning and Machine Learning is freely available online, and is a good place to start.

- BRML:
*Bayesian Reasoning and Machine Learning*, David Barber, Cambridge University Press 2012. Free online. - MLaPP:
*Machine Learning: A Probabilistic Perspective*, Kevin Murphy, MIT Press 2012. Excerpt online. - PRML:
*Pattern Recognition and Machine Learning*, Christopher Bishop, Springer 2007. Excerpt online. - GEV:
*Graphical Models, Exponential Families, and Variational Inference*, Martin Wainwright & Michael Jordan, Foundations & Trends in Machine Learning, 2008. - EBS:
*Graphical Models for Visual Object Recognition and Tracking*, Erik B. Sudderth, PhD Thesis (Chapter 2), MIT 2006.

- A Brief Introduction to Graphical Models & Bayesian Networks, K. Murphy, 1998.
- Graphical Models, M. Jordan, Statistical Science 2004.

- BRML: Chapters 2-4, excluding Sec. 3.4.
- MLaPP: Sec. 10.1-10.2, 10.5, 19.1-19.4.
- PRML: Sec. 8.1-8.3.
- GEV: Sec. 2.1-2.4.
- EBS: Sec. 2.2.1-2.2.3.

- BRML: Sec. 5.1-5.2.
- MLaPP: Sec. 20.2.
- PRML: Sec. 8.4.
- GEV: Sec. 2.5.
- EBS: Sec. 2.2.5, 2.3.2.
- Factor Graphs and the Sum-Product Algorithm, F. Kschischang, B. Frey, & H. A. Loeliger, IEEE Trans. Info Theory 2001.

- BRML: Chapter 6.
- MLaPP: Sec. 20.4-20.5.
- GEV: Sec. 2.5.
- A Short Course on Graphical Models, M. Paskin, 2003.

- BRML: Chapter 8 excluding Sec. 8.4, 8.8.
- MLaPP: Sec. 9.2.
- PRML: Sec. 2.4.
- GEV: Chapter 3.
- EBS: Sec. 2.1.

- BRML: Sec. 11.1-11.3.
- MLaPP: Sec. 11.4.
- PRML: Chapter 9.
- GEV: Sec. 6.2.
- EBS: Sec. 2.3.3.
- A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants, R. Neal & G. Hinton, 1998.

- BRML: Sec. 8.4, Chapter 24.
- MLaPP: Sec. 10.2.5, 18.1-18.4, 19.4.4, 20.2.3, 26.7.
- PRML: Sec. 13.3.
- A Unifying Review of Linear Gaussian Models, S. Roweis & Z. Ghahramani, Neural Computation 1999.
- Bayesian Modeling of Uncertainty in Low-Level Vision, R. Szeliski, IJCV 1990.

- BRML: Sec. 27.1-27.2.
- MLaPP: Sec. 23.1-23.4.
- PRML: Sec. 11.1.
- EBS: Sec. 2.4.
- Introduction to Monte Carlo Methods, Sec. 1-3, D. MacKay, 1999.

- BRML: Sec. 27.6.
- MLaPP: Sec. 23.5.
- An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo, Cappe, Godsill, & Moulines, IEEE 2007.
- Nonparametric Belief Propagation, Sudderth, Ihler, Isard, Freeman, & Willsky, CACM 2010.

- BRML: Sec. 27.3-27.5.
- MLaPP: Chapter 24.
- PRML: Sec. 11.2-11.4.
- Introduction to Monte Carlo Methods, Sec. 4-8, D. MacKay, 1999.
- An Introduction to MCMC for Machine Learning, Andrieu, de Freitas, Doucet, & Jordan, Machine Learning 2003.

- BRML: Sec. 28.3-28.4.
- MLaPP: Chapter 21.
- PRML: Sec. 10.1-10.6.
- GEV: Chapter 5.
- EBS: Sec. 2.3.1.
- Variational Message Passing, J. Winn & C. Bishop, JMLR 2005.

- BRML: Sec. 28.7.
- MLaPP: Sec. 22.1-22.4.
- GEV: Sec. 4.1, Chapter 7.
- EBS: Sec. 2.3.2.
- Understanding Belief Propagation and its Generalizations, J. Yedidia, W. Freeman, & Y. Weiss, IJCAI 2001.

- MLaPP: Sec. 19.6-19.7.
- An Introduction to Conditional Random Fields, C. Sutton & A. McCallum, Foundations & Trends in ML 2011.
- Structured Prediction and Learning in Computer Vision, S. Nowozin & C. Lampert, CVPR 2012.

- MLaPP: Chapter 28.
- PRML: Chapter 5.
- Deep Learning, R. Salakhutdinov, KDD 2014.

- MLaPP: Sec. 25.2.
- EBS: Sec. 2.5.
- Modern Bayesian Nonparametrics, P. Orbanz & Y. W. Teh, NIPS 2011.