CS 143 Introduction to Computer Vision

Fall 2011, MWF 11:00 to 11:50, CIT 368.
Instructor: James Hays

TAs: Evan Wallace (HTA), Sam Birch, Paul Sastrasinh, Libin "Geoffrey" Sun, and Vazheh Moussavi.

Course Description

Course Catalog Entry
How can computers understand the visual world of humans? This course treats vision as a process of inference from noisy and uncertain data and emphasizes probabilistic, statistical, data-driven approaches. Topics include image processing; segmentation, grouping, and boundary detection; recognition and detection; motion estimation and structure from motion. This offering of CS 143 will emphasize the core vision task of recognition in particular. We will train and evaluate classifiers to recognize various visual phenomena.

The course will consist of five programming projects, two written quizzes, and a self-chosen final project. Students can earn graduate credit for the course but will need to meet higher requirements on all projects throughout the semester and need the instructor's permission. This course can satisfy the graduate A.I area requirement.

Prerequisites

This course requires programming experience as well as linear algebra, basic calculus, and basic probability. Previous knowledge of visual computing will be helpful. The following courses (or equivalent courses at other institutions) are helpful prerequisites:

• CS 123, Introduction to Computer Graphics
• CS 129, Computational Photography
• CS 195-F, Introduction to Machine Learning

Some of the course topics overlap with these related courses, but none of the assignments will.

Assignments	Winning projects	All Results
Hybrid images with Laplacian pyramids	Andy Loomis, Emanuel Zgraggen, Dylan Field	Project 1 results
pB Lite: boundary detection	Paul Sastrasinh, Li Sun, Hang Su	Project 2 results
Scene recognition with bag of words	Paul Sastrasinh, Chen Xu, Yun Zhang	Project 3 results
Face detection with a sliding window	Emanuel Zgraggen, Hang Su, Paul Sastrasinh	Project 4 results
Tracking and Structure from Motion or ...	Hang Su	Project 5 results
Your choice for final project	Seth Goldenberg, Emanuel Zgraggen	Final project results

It is strongly recommended that all projects be completed in Matlab. All starter code will be provided for Matlab. Students may implement projects through other means but it will generally be more difficult.

Textbook

Readings will be assigned in "Computer Vision: Algorithms and Applications" by Richard Szeliski. The book is available for free online or available for purchase.

Grading

Your final grade will be made up from

• 80% 5 programming projects
• 20% 2 written quizzes

You have three "late days" for the whole course. That is to say, the first 24 hours after the due date and time counts as 1 day, up to 48 hours is two and 72 for the third late day. This will not be reflected in the initial grade reports for your assignment, but they will be factored in and distributed at the end of the semester so that you get the most points possible.

Graduate credit is available and each project will specifiy the minimum requirements to earn such credit.

Important Links:

• Collaboration Policy
• Matlab Tutorial

Contact Info and Office Hours:

You can contact the professor or TA staff with any of the following:

• James: hays[at]cs.brown.edu
• HTA and Professor: cs143headtas[at]cs.brown.edu
• TAs and Professor: cs143tas[at]cs.brown.edu

James' office hours will be held in his office (CIT 445). TA office hours will be held in the Brindy Bowl (CIT 271).

• James Hays (hays), Monday and Wednesday 1:00-2:00
• Libin "Geoffrey" Sun (lbsun), Monday 7-9pm
• Paul Sastrasinh (psastras), Tuesday 7-9pm
• Sam Birch (sbirch), Wednesday 7-9pm
• Evan Wallace (edwallac), Thursday 7-9pm
• Vazheh Moussavi (vmoussav), Friday 5-7pm

Tentative Syllabus

Class Date	Topic	Slides	Reading	Projects
W, Sept 7th	Introduction to computer vision	.ppt, .pdf	Szeliski 1
Image Formation and Filtering
F, Sep 9th	Cameras and optics	.ppt, .pdf	Szeliski 2.1, especially 2.1.5	Project 1 out
M, Sep 12th	Light and color	.ppt, .pdf	Szeliski 2.2 and 2.3
W, Sep 14th	Pixels and image filters	.ppt, .pdf	Szeliski 3.2
F, Sep 16th	Thinking in frequency	.ppt, .pdf	Szeliski 3.4
M, Sep 19th	Image pyramids and applications	.ppt, .pdf	Szeliski 3.5.2 and 8.1.1
Machine Learning Crash Course
W, Sep 21st	Machine learning: overview	.ppt, .pdf
F, Sep 23rd	Machine learning: clustering	.ppt, .pdf	Szeliski 5.3
M, Sep 26th	Machine learning: classification	.ppt, .pdf		Project 1 due
Grouping and Fitting
W, Sep 28th	Edge detection and line fitting w/ Hough transform	.ppt, .pdf	Szeliski 4.2	Project 2 out
F, Sep 30th	Robust fitting (Hough Transform)	.ppt, .pdf	Szeliski 4.3
M, Oct 3rd	Robust fitting (RANSAC and others)	.ppt, .pdf	Szeliski 4.3
W, Oct 5th	Mixture of Gaussians and EM	.ppt, .pdf
F, Oct 7th	Gestalt cues, MRFs, and graph cuts	.ppt, .pdf	Szeliski 5.5
M, Oct 10th	No classes			Project 2 due
Recognition
W, Oct 12th	Recognition Overview and History	.ppt, .pdf	Szeliski 14	Project 3 out
F, Oct 14th	Image features and bag of words models	.ppt, .pdf	Szeliski 4.1.2, 14.4.1, and 14.3.2
M, Oct 17th	Interest points: corners	.ppt, .pdf	Szeliski 4.1.1
W, Oct 19th	Quiz 1
F, Oct 21st	Interest points and instance recognition	.ppt, .pdf	Szeliski 14.3
M, Oct 24th	Large-scale instance recognition	.ppt, .pdf	Szeliski 14.3.2	Project 3 due
W, Oct 26th	Detection with sliding windows	.ppt, .pdf	Szeliski 14.1
F, Oct 28th	Guest talk: Jim Rehg, Behavior Imaging and the Study of Autism
M, Oct 31st	Detection with sliding windows continued	.ppt, .pdf	Szeliski 14.2	Project 4 out
W, Nov 2nd	Context and Spatial Layout	.ppt, .pdf	Szeliski 14.5
F, Nov 4th	Guest talk: Gabriel Taubin, 3d photography
Multiple Views and Motion
M, Nov 7th	Feature Tracking	.ppt, .pdf	Szeliski 4.1.4
W, Nov 9th	Optical Flow	see above	Szeliski 8.4
F, Nov 11th	Guest lecture: Deqing Sun, Optical flow			Project 4 due
M, Nov 14th	Epipolar Geometry	.ppt, .pdf	Szeliski 11
W, Nov 16th	Stereo Correspondence	.ppt, .pdf		Project 5 out
F, Nov 18th	Structure from Motion	.ppt, .pdf	Szeliski 7	Final Project out
Advanced Topics
M, Nov 21st	Activity Recognition	.ppt, .pdf
W, Nov 23rd	No classes
F, Nov 25th	No classes
M, Nov 28th	Internet Scale Vision	.ppt, .pdf
W, Nov 30th	Guest lecture: Pedro Felzenszwalb, Object Detection	.pdf
F, Dec 2nd	Crowdsourcing	.ppt, .pdf
M, Dec 5th	Attributes and Course Summary	.ppt, .pdf
W, Dec 7th	Quiz 2
F, Dec 9th	No classes, reading period
M, Dec 12th	No classes, reading period			Final Project / Project 5 due
T, Dec 13th, 9:00 AM	Exam Period - final presentations

Acknowledgements

The materials from this class rely significantly on slides prepared by other instructors, especially Derek Hoiem and Svetlana Lazebnik. Each slide set and assigment contains acknowledgements. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgements.

Previous Versions of Course

Michael Black's most recent offering of CS 143 can be found here