"Extraction of layers with opacities from images"

Yagiz Aksoy, PhD student at ETH Zurich with Marc Pollefeys/currently a visiting student at MIT

Thursday, September 21, 2017 at 4:00 P.M.

Room 368 (CIT 3rd Floor)

Photographs have naturally occurring soft transitions between image colors as well as between semantically meaningful spatial segments. Decomposing images into layers that represent such soft transitions allow applications like image editing and compositing. In this talk, I will talk about three methods that generate layers with opacities for three separate application scenarios. First part covers green-screen keying, separation of actors and objects in the foreground that are shot with a controlled background. Keying is widely utilized in movie post-production industry, but the task requires specialized artists and creates a bottleneck in terms of time. We introduce color unmixing, which decreases the interaction time by an order of magnitude while achieving higher quality results when compared to commercial tools. In the second part, I will generalize color unmixing to soft color segmentation. Soft color segments are layers of homogeneous colors that can be exported to common photo editing tools such as Photoshop to achieve realistic color editing and compositing results effortlessly. The proposed fully automatic method is orders of magnitude faster than the state-of-the-art while giving higher quality layers. In the last part, I will be talking about natural image matting, estimation of the foreground in images where the background content is not controlled. We approach this problem via a graph-based formulation, in which we carefully examine the main challenges in natural matting and define pixel-to-pixel connections that effectively represent the transitions between spatial segments. Our energy function, which can be formulated as a linear system and solved in closed-form, is also generalized to related applications.

Host: Professor James Tompkin