Instructor: James Tompkin
Sign-up | Course | Schedule/Class Material | General Policy | Feedback | Acknowledgements
<-- Form for Paper Questions -->
James' office hour (Tues 1pm) appointment slot signups are here (top left).
How does computer vision enable new interactive graphical applications, and how can we improve them?
Computer vision strives to understand, interpret, and reconstruct information about the real world from image and video data. Computer graphics models dynamic virtual worlds to be synthesized in realistic or stylized ways. In visual computing, these fields are converging since both disciplines create and exploit models describing the visual appearance of objects and scenes. Interaction allows us to explore these worlds, and to use ourselves and our environments directly as interfaces. Machine learning and deep learning allows us to define mappings between domains across vision, graphics, and interaction, and to generate new data such as images from recombining existing databases. Combined, these disciplines enable applications from the seemingly simple, like semantic photo editing, to the seemingly science fiction, like mixed reality.
In this seminar course, we will discover the state of the art algorithmic contributions in computer vision which make these new applications possible. We will concentrate on recent research results that were published at top-tier conferences and journals from problem fields such as reconstruction of static and dynamic 3D scenes, computational photography and videography, multi-view camera systems, generative methods for image formation, and vision-based interaction devices. Each week, we will read state-of-the-art papers, present them, and discuss their contributions, impact, and limitations. Then, we will develop projects which implement and extend these ideas. Beyond computer vision, this course will help us learn how to quickly interpret and assess academic papers, and how to give effective and engaging presentations.
Please join us!
Upon completion of this course, students will have:
The course is spilt into two halves. In the first half, we will read research papers, present them, and discuss their contributions, impact, and limitations. We will build upon our analyses and discussions to propose projects which would further the state of the art. Then, in the second half, we will try and do it: we will break into teams and implement projects which further the state of the art.
This class can be taken as a capstone. Talk to James about the expected standard and additional work across the course.
This is a graduate course, but undergraduates are welcome! As a graduate class, we expect you to be somewhat self-guided; be prepared to read beyond the course material, and to explore and discover for yourself. Students should know something about visual computing before taking this course, e.g., having taken an existing vision, graphics, or deep learning course. Any other interested students should get in touch with James!
Due to the form of the class, there are no late submissions or late days. We expect you to attend every session, but let James know if you have any special requirements. For sickness and other issues of wellbeing, please obtain a note from health services and we will accommodate.
We expect you to read every paper in preparation for the upcoming presentation and discussion. Reading these papers may be difficult initially, and students are not expected to understand everything. However, students are expected to actively engage in discussions to further their understanding of the presented material, with the help of the instructor and the class, within a supportive and creative atmosphere. Ideas that are developed during the seminar discussion are intended to directly influence your projects.
Demos are welcome! Code or executables may be available for the techniques, and you should feel free to show them off. Likewise, for video material, but don’t just play it without providing any insight.
Each session, one student will be randomly selected as the discussion leader. They will receive a digest of the submitted questions before the seminar. Their goal is to briefly summarize the strengths and weaknesses of the technique, raise questions appropriately throughout the discussion, covers future work ideas, and keep order.
How to Read and Present Academic Papers:
Further reading material:
|Date||Topic||Reading / Slides||More info|
|First half—Review, Ideation, and Proposal|
|Mon||9||Sep||How to read and present; submitting questions; lab demos; interest declaration||Active Appearance Models, Cootes et al., ECCV 1998 (PAMI version 2001)|
|Wed||11||Sep||Paper session||A Morphable Model For The Synthesis Of 3D Faces, Blanz and Vetter, 1999|
|Mon||16||Sep||Paper session||Single-image Portrait Relighting, Sun et al., 2019|
|Wed||18||Sep||Paper session||Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer, Lasinger et al., 2019, arXiv preprint
|Mon||23||Sep||Paper session||KinectFusion: Real-Time Dense Surface Mapping and Tracking, Newcombe et al., ISMAR 2011|
|Wed||25||Sep||Paper session||Real-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera, Guo et al., SIGGRAPH 2017|
|Wed||02||Oct||Paper session||DC-GAN: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Radford et al., ICLR 2016
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets, Chen et al.
|Mon||07||Oct||Paper session||StyleGAN, Karras et al., CVPR 2019|
|Wed||09||Oct||Paper session||MeshCNN, Hanocka et al., SIGGRAPH 2019|
|Mon||14||Oct||Indiginous People's Day—no class|
|Wed||16||Oct||Paper session||PlatonicGAN, Henzler et al., arXiv 2019|
|Mon||21||Oct||Paper session||3D-Aware Scene Manipulation via Inverse Graphics, Yao et al., NeurIPS 2018|
|Wed||23||Oct||Final proposal session|
|Wed||30||Oct||Project review 1|
|Wed||13||Nov||Project review 2|
|Wed||04||Nov||Project review 3|
|Fri||06||Dec||New England Computer Vision Workshop @ Brown|
Our intent is that this course provide a welcoming environment for all students who satisfy the prerequisites. Our TAs have undergone training in diversity and inclusion, and all members of the CS community, including faculty and staff, are expected to treat one another in a professional manner. If you feel you have not been treated in a professional manner by any of the course staff, please contact any of James (the instructor), John Hughes (Dept. Chair), Tom Doeppner (Director of Undergraduate Studies), David Laidlaw (Director of Graduate Studies), or Laura Dobler (diversity and inclusion staff member). We will take all complaints about unprofessional behavior seriously. Your suggestions are encouraged and appreciated. Please let James know of ways to improve the effectiveness of the course for you personally, or for other students or student groups. To access student support services and resources, and to learn more about diversity and inclusion in CS, please visit http://cs.brown.edu/about/diversity/resources/.
Prof. Krishnamurthi has good notes on this area.
This class runs quiet hours from 9pm to 9am every day. Please do not expect a response from us via any channel. Likewise, we won't ask you to do anything between these times, either, like hand in projects.
Feel free to talk to your friends about the concepts in the projects, and work through the ideas behind problems together, but be sure to always write your own code and perform your own write up. You are expected to implement the core components of each project on your own, but the extra credit opportunties often build on third party data sets or code. Feel free to include results built on other software, as long as you credit correctly in your handin and clearly demark your own work. In general, if you use an idea, text, or code from elsewhere, then cite it.
Brown-wide, academic dishonesty is not tolerated. This includes cheating, lying about course matters, plagiarism, or helping others commit a violation. Plagiarism includes reproducing the words of others without both the use of quotation marks and citation. Students are reminded of the obligations and expectations associated with the Brown Academic and Student Conduct Codes.
Brown University is committed to full inclusion of all students. Please inform me if you have a disability or other condition that might require accommodations or modification of any of these course procedures. You may email me, come to office hours, or speak with me after class, and your confidentiality is respected. We will do whatever we can to support accommodations recommended by SEAS. For more information contact Student and Employee Accessibility Services (SEAS) at 401-863-9588 or . Students in need of short-term academic advice or support can contact one of the deans in the Dean of the College office.
Being a student can be very stressful. If you feel you are under too much pressure or there are psychological issues that are keeping you from performing well at Brown, we encourage you to contact Brown's Counseling and Psychological Services. They provide confidential counseling and can provide notes supporting extensions on assignments for health reasons.
We expect everyone to complete the course on time. However, we certainly understand that there may be factors beyond your control, such as health problems and family crises, that prevent you from finishing the course on time. If you feel you cannot complete the course on time, please discuss with James Tompkin the possibility of being given a grade of Incomplete for the course and setting a schedule for completing the course in the upcoming year.
Laptops are discouraged, please, except for class-relevant activities, e.g., to help answer questions and show items relevant to discussion. No social media, email, etc., because it distracts not just you but other students as well. Read Shirky on this issue ("Why I Just Asked My Students to Put Their Laptops Away"), or Rockmore ("The Case for Banning Laptops in the Classroom").
We will release course lecture material online. In considering laptop use for note taking, please be aware that research has shown note taking on paper to be more efficient than on a laptop keyboard (Mueller and Oppenheimer), as it pushes you to summarize the content instead of transcribe it.
We would appreciate any feedback on how to improve the course. We have created an anonymous form to collect feedback, which is accessible through your Brown Google account (but we do not collect your identity). We will read the feedback regularly. If there is something which you do not feel comfortable addressing with this anonymous form, please consider contacting one of the parties listed in the general policy.
Portions of this seminar design are from Stefanie Tellex's CSCI 2951-R course, from Christian Theobalt and his CVfCG course @ Max-Planck-Institute for Informatics, with special thanks to Christian Richardt. Thanks also to James Hays and CSCI2951-T Data-Driven Computer Vision course @ Brown University, with special thanks to Genevieve Patterson.
Thanks to Karras et al. (StyleGAN, CVPR 2019) and to Dmitry Nikitko, whose software I used to make the Brown CS faculty bust teaser.
Thanks to Tom Doeppner and Laura Dobler for the text on accommodation, mental health, and incomplete policy.
Thank you to the previous students who helped to improve this class. Previous course runs: