DateTopicReadingsMaterials
09/10 Course Overview PRML 1.1, 1.2 Lecture Slides
09/15 Classification: Evaluation and ROC Curves PRML 1.5.1, Wikipedia
09/17 Classification: Naive Bayes PRML 4.2
09/22 Maximum Likelihood Estimation PRML 1.2.4, 2.1, 2.2
09/24 Frequentist and Bayesian Estimation PRML 1.2.3, 1.5
09/29 Bayesian Loss Functions, Dirichlet Priors PRML 1.5, 2.1, 2.2 Lecture Slides
10/01 K Nearest Neighbors, Cross-validation PRML 1.3, 2.5 Example: Color Constancy
10/06 Linear Regression: Maximum Likelihood PRML 3.1.1, 3.1.2, 3.1.4 Lecture Slides
10/08 Bayesian Regression, Multivariate Gaussians PRML 2.3.1-2.3.4, 3.3 Lecture Slides
10/13 Logistic Regression PRML 4.3
10/15 Logistic Regression, Exponential Families PRML 2.4, 4.3
10/20 Logistic Regression, Stochastic Gradient Descent PRML 4.3, 5.2.4
10/22 MIDTERM
10/27 Regularized Stochastic Gradient, Perceptron Algorithm, Kernels PRML 4.1.7, 6.1, 6.2 Lecture Slides
10/29 Kernels, Gaussian Process Regression and Classification PRML 6.2, 6.4.1, 6.4.2, 6.4.5
11/03 Clustering, K-Means PRML 9.1
11/05 Maximum Entropy Review, K-Means PRML 9.1
11/10 Rand Index, Expectation-Maximization Algorithm PRML 9.2, Wikipedia
11/12 Mixtures of Multinomials or Gaussians, EM Algorithm PRML 9.2, 9.3 Lecture Slides
11/17 EM Algorithm Theory, Hidden Markov Models PRML 9.4, 13.1, 13.2 Lecture Slides
11/19 HMMs: Viterbi, Forward-Backward, and EM Algorithms PRML 13.2 Lecture Slides
11/24 Monte Carlo: Importance Sampling, MCMC, Gibbs Sampling PRML 11.1.1, 11.1.4, 11.2.1, 11.3
12/01 Principal Component Analysis, Probabilistic PCA PRML 12.1.2, 12.1.3, 12.2
12/03 Probabilistic PCA, Factor Analysis, Directed Graphical Models PRML 12.2.1, 12.2.2, 12.2.4, 8.1

Supplemental Exercises

For Bishop's Pattern Recognition and Machine Learning text, a number of exercises (marked with "WWW") have solutions available online. Many of these WWW problems test conceptual issues likely to show up on exams, particularly those in the following chapters:

Supplemental Readings

An electronic edition of another good machine learning textbook, The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman, is available online. They provide clear explanations for many topics, from a more statistical perspective. The following sections are particularly recommended for those who would like an alternative presentation of some topics: