"Understanding 3D Shape Perception using the Morse Smale Complex

Ben Kunsberg, Applied Math, Brown University

Wednesday, April 25, 2018 at 12:00 Noon

Lubrano Conference Room (CIT 4th Floor)

What are the visual structures that encode 3D perception? How can we see reliable 3D shape in such a diverse range of cues like texture, shading, and line drawings? The usual attempt to recover 3D shape proceeds by isolating the cue, then applying inverse optics based on that cue and a cost function minimization. Although this may sometimes lead to decent reconstructions, it requires a source separation and precise knowledge of the cue and the way it renders an image from the surface. For example, in Lambertian shading, the intensity field (cue) is modeled by the dot product of the light source and the normal field. If this Lambertian shading model is wrong, as it invariably will be in natural scenes, a cost function minimization will be using inappropriate penalty terms and thus may end up with an arbitrarily wrong minimizer. However, many psychophysics studies show that qualitative shape constancy is perceived under diverse illumination conditions.

We are developing a theory that can formally deal with this conundrum while also explaining electrophysiological and psychophysical data. We believe the key lies in analyzing the topological properties of the orientation field of the image. We believe the Morse Smale complex (calculated via the orientations and consisting of multiple contours) provides the image contours that ‘anchor’ the surface perception. First, we will show that these contours are computable from the image with the same model under textured, specular, and shaded images. Second, we will show that these contours generically delineate the ‘bumps and valleys’ of the surface. In this way, our model gives a qualitative surface description independent of the rendering and may explain shape constancy.

Host: Professor James Tompkin