Lectures & Readings

Most course readings are taken from the course textbook, Machine Learning: A Probabilistic Perspective (MLaPP).

The specific schedule of topics and readings below is tentative, and may change as the course progresses.

Date Topic Primary Readings Materials
9/05 Course Overview MLaPP: 1.1-1.3 slides
9/10 Probability: Discrete random variables
Model complexity & validation
MLaPP: 2.1-2.3
MLaPP: 1.4
slides
9/12 Generative Models
Naive Bayes classifiers
MLaPP: 3.1, 3.3
MLaPP: 3.5
slides
9/17 Bayesian decision theory & ROCs
Continuous parameters & frequentist estimation
MLaPP: 5.7
MLaPP: 2.4
slides
9/19 Maximum likelihood & Bayesian learning
Smoothing: Beta & Dirichlet priors
MLaPP: 3.2
MLaPP: 2.5.4, 3.3-3.4
slides
9/24 Bayesian decisions & model selection MLaPP: 3.2, 5.2-5.3, 5.7 slides
9/26 Monte Carlo Methods
Multivariate Gaussian Distributions
MLaPP: 2.7
MLaPP: 2.5, 4.1
slides
10/01 Gaussian Discriminant Analysis
Linear Regression
MLaPP: 4.2-4.4
MLaPP: 7.1-7.2
slides
10/03 Linear Regression & Least Squares
Bayesian Linear Regression
MLaPP: 7.3
MLaPP: 7.5-7.6
slides
10/08 Bayesian Predictive Distributions
Logistic Regression & Probit Regression
MLaPP: 7.6
MLaPP: 8.1-8.3, 8.6
slides
10/10 Optimization for Logistic Regression MLaPP: 8.3 slides
10/15 Exponential Families
Robust Regression & Generalized Linear Models
MLaPP: 9.1-9.2
MLaPP: 7.4, 9.3-9.4
slides
10/17 Binary Feature Selection & Search
L1 Regularization & Sparsity
MLaPP: 13.1-13.2
MLaPP: 13.3-13.4
slides
10/22 L1 Regularization & Sparsity
Kernel Methods
MLaPP: 13.3-13.4
MLaPP: 14.2, 14.4
slides
10/24 Gaussian Process Regression
Gaussian Process Classification, GLMs
MLaPP: 15.1-15.2
MLaPP: 15.3
slides
10/29 GP Regression & Classification
Stochastic Gradients, Perceptron Algorithm
MLaPP: 15.2-15.3
MLaPP: 8.5
slides
10/31 Margins & Support Vector Machines MLaPP: 14.5 slides
11/05 Neural Networks
Clustering & K-Means Algorithm
MLaPP: 16.5
MLaPP: 1.3, 11.1-11.2
slides
11/07 Graphical Models
Mixture Models & EM Algorithm
MLaPP: 10.1-10.4
MLaPP: 11.2-11.4
slides
11/12 Expectation Maximization Algorithm MLaPP: 11.4 slides
11/14 Expectation Maximization Algorithm
Hidden Markov Models
MLaPP: 11.4
MLaPP: 17.1-17.3
slides
11/19 Viterbi & Forward-Backward Algorithms
EM for Hidden Markov Models
MLaPP: 17.4
MLaPP: 17.5
slides
11/21 Forward-Backward & EM for HMMs
Principal Components Analysis (PCA)
MLaPP: 17.4-17.5
MLaPP: 12.1-12.2
slides
11/26 Factor Analysis & Probabilistic PCA
EM Algorithm for Factor Analysis & PPCA
MLaPP: 12.1-12.3 slides
notes
11/28 NO LECTURE (Thanksgiving)
12/03 Continuous State Space Models
Topic Models
MLaPP: 18.1-18.3
MLaPP: 27.1-27.3
slides
12/05 NO LECTURE
12/10 Advanced Topics & Course Review slides

Recitations

Date Topic Readings Materials
9/10 Matlab Tutorial YAGTOM Matlab scripts
9/17 Continuous Random Variables MLaPP: 2.2, 2.4-2.6 notes
9/24 Linear Algebra Tutorial Stanford CS229 notes
10/01 Continuous Optimization Algorithms MLaPP: 8.3 notes
10/08 Continuous Optimization Software minFunc documentation
minFunc examples
Matlab scripts
PMTK3 installation
10/15 Lagrange Multipliers Klein tutorial notes
10/22 Midterm Exam Review Session notes
10/29 NO RECITATION
11/05 Dynamic Programming &
String Kernels
Wikipedia
Algorithms, Dasgupta et al.
notes
example
11/12 Markov Chains MLaPP: 17.2 notes
11/19 Principal Components Analysis (PCA) MLaPP: 12.2 notes
Matlab demo
11/26 NO RECITATION
12/03 Final Exam Review Session notes