CSCI 129 Computational Photography

Fall 2012, MWF 11:00 to 11:50, CIT 477 (Lubrano)
Instructor: James Hays
HTA: Sam Birch and GTA: Emanuel Zgraggen

Course Description

Course Catalog Entry
Computational Photography describes the convergence of computer graphics, computer vision, and the Internet with photography. Its goal is to overcome the limitations of traditional photography using computational techniques to enhance the way we capture, manipulate, and interact with visual media. In this course we will study many interesting, recent image based algorithms and implement them to the degree that is possible. We will cover topics such as

Cameras and image formation
Human visual perception
Image processing (filtering, pyramids)
Image blending and compositing
Image retargeting
Texture synthesis and transfer
Image completion / inpainting
Super-resolution, deblurring, and denoising.
Image based lighting and rendering
High dynamic range
Depth and defocus
Flash / no flash photography
Coded aperture photography
Single / multi view reconstruction
Photo quality assessment
Non photorealistic rendering
Modeling and synthesis using Internet data
... more interesting topics.

The course will consist of six programming projects, two written exams, and a student-chosen final project. Students can earn graduate credit for the course but will need to meet higher requirements on all projects throughout the semester. The graduate version can count towards a graphics specialization.

Prerequisites

This course requires programming experience as well as linear algebra, basic calculus, and basic probability. Previous knowledge of computer graphics or computer vision will be helpful. It is strongly recommended that students have taken one of the following courses (or equivalent courses at other institutions):

Some of the course topics overlap with these related courses, but none of the assignments will.

Assignments	Winning projects	All Results
Image alignment with pyramids	Hang Su	Project 1 Results
Gradient domain fusion using Poisson blending	Daniel Moreno	Project 2 Results
Image retargeting with Seam Carving	Jonathan Mace Bryce Aebi	Project 3 Results
Texture synthesis and transfer with Image Quilting	Hua Guo	Project 4 Results
High dynamic range and tone mapping	Yan Li	Project 5 Results
Automatic Panorama Construction	Daniel Moreno	Project 6 Results
Your choice for final project	Ian Strickman Yan Li	Final Project Results

It is strongly recommended that all projects be completed in Matlab. All starter code will be provided for Matlab. Students may implement projects through other means but it will generally be more difficult.

Equipment

Students are encouraged to capture their own photographic data for their projects. There's no need for anything fancy -- any digital camera with manual controls should work.

Textbook

Readings will be assigned in the free, online textbook "Computer Vision: Algorithms and Applications" by Richard Szeliski.

Grading

Your final grade will be made up from

65% 6 programming projects
15% final project
20% 2 written examinations

You have three "late days" for the whole course. That is to say, the first 24 hours after the due date and time counts as 1 day, up to 48 hours is two and 72 for the third late day. This will not be reflected in the initial grade reports for your assignment, but they will be factored in and distributed at the end of the semester so that you get the most points possible.

Graduate credit is available and each project will specifiy the minimum requirements to earn such credit.

Important Links:

Contact Info and Office Hours:

You can contact the professor or TA staff with any of the following:

Professor James: hays[at]cs.brown.edu
TAs and Professor: cs129tas[at]cs.brown.edu

Professor James' office hours will be held in his office (CIT 445). TA office hours will be held in CIT 227 or CIT 271.

James Hays (hays), Monday and Wednesday 1:00-2:00pm
HTA Sam Birch (sbirch), Sunday, 7:30-9:30pm, (CIT 227, Moonlab)
GTA Emanuel Zgraggen (ez), Monday 5:00-7:00pm, (CIT 271, Fishbowl)

Tentative Syllabus

Class Date	Topic	Slides	Reading	Projects
W, Sept 5	Introduction to computational photography	.ppt, .pdf	Szeliski chapter 1
F, Sept 7	Cameras and optics	.ppt, .pdf	Szeliski chapter 2, especially 2.1.5	Project 1 out
M, Sept 10	Capturing light, man vs machine	.ppt, .pdf	Szeliski chapter 2
W, Sept 12	Sampling and reconstruction	.ppt, .pdf	Szeliski chapter 2
F, Sept 14	Linear Filters	.ppt, .pdf	Szeliski 3.2
M, Sept 17	Frequency Domain	.ppt, .pdf	Szeliski 3.4	Project 1 due Project 2 out
W, Sept 19	Blending and compositing	.ppt, .pdf	Szeliski 3.3 and 9.3.4
F, Sept 21	Project 2 and Morphology	.pptx, .pdf .ppt, .pdf	Szeliski 3.3.2
M, Sept 24	Point Processing and Image Warping	.ppt, .pdf .ppt, .pdf	Szeliski 3.1 and 3.6.1
W, Sept 26	Image Retargeting	.ppt, .pdf
F, Sept 28	Image Retargeting, continued	see above		Project 2 due Project 3 out
M, Oct 1	Video Textures	.ppt, .pdf	Szeliski 13.5
W, Oct 3	Texture synthesis and filling	.ppt, .pdf	Szeliski 10.5
F, Oct 5	Image analogies, Graph cut, Scene completion	.ppt, .pdf .ppt, .pdf	Szeliski 9.3.2
M, Oct 8	No class
W, Oct 10	Exam review, Single image super-resolution	.pptx, .pdf		Project 3 due
F, Oct 12	Exam 1			Project 4 out
M, Oct 15	Invited Speaker: Connelly Barnes
W, Oct 17	Matting, Transparency, and Illumination	.ppt, .pdf	Szeliski 10.4 and 13.4
F, Oct 19	Recovering High Dynamic Range	.pptx, .pdf	Szeliski 10.2
M, Oct 22	Invited Speaker: John F. Hughes
W, Oct 24	Tone mapping	.pptx, .pdf	Szeliski 10.2	Project 4 due
F, Oct 26	Stereo	.pptx, .pdf	Szeliski chapter 11	Project 5 out
M, Oct 29	No class - Hurricane Sandy
W, Oct 31	Invited Speaker: Sylvain Paris. Fast Bilateral Filter	.ppt, .pdf
F, Nov 2	Project 5 and Stereo, continued	.pptx, .pdf .ppt, .pdf	Szeliski chapter 11
M, Nov 5	Modeling light and lightfields	.ppt, .pdf	Szeliski 13.3
W, Nov 7	Image-based lighting	.ppt, .pdf
F, Nov 9	Homographies and mosaics	.ppt, .pdf	Szeliski 9.1
M, Nov 12	Automatic image correspondence	.pptx, .pdf	Szeliski 4.1	Project 5 due Project 6 out
W, Nov 14	RANSAC and mosaic wrapup	.pptx, .pdf	Szeliski 6.1 and 9.1.6
F, Nov 16	Taking good photographs
M, Nov 19	Photo quality assessment	.pptx, .pdf
W, Nov 21	Project 6 and Final Project Discussion
F, Nov 23	No class, Thanksgiving break
M, Nov 26	Coded Aperture Photography	.ppt, .pdf	Levin et al., SIGGRAPH 2007	Project 6 due Final Project
W, Nov 28	Visual Data and the Internet	.ppt, .pdf	Szeliski 14.5.1 and 14.4.4
F, Nov 30	Visual Data and the Internet II	.ppt, .pdf
M, Dec 3	Invited speaker: Pierre-Yves Laffont. Intrinsic Images
W, Dec 5	Exam 2
F, Dec 7	Exam 2 recap, Final project discussion
M, Dec 10	No class, reading period
W, Dec 12	No class, reading period
F, Dec 21, 2pm	Final Presentations during exam period

Acknowledgements

The materials from this class rely heavily on slides prepared by other instructors. In particular, many materials are modified from those of Alexei A. Efros, who in turn uses materials from Steve Seitz, Rick Szeliski, Paul Debevec, Stephen Palmer, Paul Heckbert, David Forsyth, Steve Marschner and others, as noted in the slides. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgements.

Similar Courses at other Universities

Computational Photography (Derek Hoiem, UIUC)
Computational Photography SIGGRAPH Course (Raskar & Tumblin)
Computational Camera and Photography (Raskar, MIT Media Lab)
Digital and Computational Photography (Durand & Freeman, MIT)
Computational Photography (Efros, CMU)
Computational Photography (Essa, Georgia Tech)
Computational Photography (Levoy, Adams, Pulli, Stanford)
Computational Photography (Lazebnik, UNC)
Computational Photography (Fergus, NYU)
Internet Vision (T.Berg, SUNY)
Computer Vision (Seitz & Szeliski, UWashington)
Instroduction to Visual Computing and Visual Modeling (Kutulakos, UToronto)