SIGGRAPH 2005 Abstract. Video matting is the process of pulling a high-quality alpha matte and foreground from a video sequence. Current techniques require either a known background (e.g., a blue screen) or extensive user interaction (e.g., to specify known foreground and background elements). The matting problem is generally under-constrained, since not enough information has been collected at capture time. We propose a novel, fully autonomous method for pulling a matte using multiple synchronized video streams that share a point of view but differ in their plane of focus. The solution is obtained by directly minimizing the error in filter-based image formation equations, which are over-constrained by our rich data stream. Our system solves the fully dynamic video matting problem without user assistance: both the foreground and background may be high frequency and have dynamic content, the foreground may resemble the background, and the scene is lit by natural (as opposed to polarized or collimated) illumination.
Defocus Video Matting Morgan McGuire, Wojciech Matusik, Hanspeter Pfister, John F. Hughes, and Fredo Durand
Brown University and MERL and MIT [PDF] [Video] [PPT Talk] [Bibtex] [Acknowledgements] [Media Pack]
The data for this project are too large to post online. Please contact Hanspeter Pfister at MERL if you would like to work with our video streams for research purposes.
Articles about Defocus Matting
Third party articles describing our technique have appeared in:
- Print and on-line issues of Millimeter, August 2005
- CGWORLD, September 2005
- c't Magazine (German)
- Noriko Kurachi's new book on videography published by Ohmsha
@article{1073231, author = {Morgan McGuire and Wojciech Matusik and Hanspeter Pfister and John F. Hughes and Fr\&\#233;do Durand}, title = {Defocus video matting}, journal = {ACM Trans. Graph.}, volume = {24}, number = {3}, year = {2005}, issn = {0730-0301}, pages = {567--576}, doi = {http://doi.acm.org/10.1145/1073204.1073231}, publisher = {ACM Press}, address = {New York, NY, USA}, URL = {http://www.cs.brown.edu/people/morgan/DefocusVideoMatting} }
AcknowledgementsWilliam Yerazunis and John Barnwell of MERL worked with us to design and manufacture the camera. We thank Shree Nayar (Columbia), Doug Roble (Digital Domain), Bill Freeman (MIT), and Michael Black (Brown) for their advice, and NVIDIA Corporation for Morgan's fellowship. The novel backgrounds are photographs by Shriram Krishnamurthi.
Media PackThe following downloads are available for authors writing articles about defocus matting. Copyrights are all held by Morgan McGuire or by the ACM (for materials that appear in the SIGGRAPH paper). Using these materials for review or news purposes with attribution is fair use and explicitly allowed by copyright law in the United States.
Explanation Movie
SIGGRAPH 2005 Paper
SIGGRAPH 2005 Presentation (includes more movies)
Images