(with Ruth Rosenholtz)
Rosenholtz and Malik (ARVO '93, '94) have developed a computational model for shape from texture. This model assumes that the image consists of only a single textured surface. What if a textured surface is occluded by another object? What if there is more than one texture in the image? What if one views one textured surface through another texture; e.g., when viewing a grassy field through the slats in a picket fence? All of these situations are common in natural images. Can we modify the basic shape from texture model to perform shape from texture in these more complicated cases?
Malik and Rosenholtz drew an analogy between shape from texture and structure from motion: two image patches give one two views of the same surface in one image, just as two frames do in motion. In both cases the multiple views carry information about the surface shape and orientation. If this analogy is useful, we should be able to apply techniques from computational models of structure from motion for multiple motions to solve the problems mentioned above.
We present a computational model for shape from texture for multiple textures with occlusion, along with results on real and synthetic images. The model is based on both previous multiple motion work and Rosenholtz and Malik's shape from texture model. We demonstrate that the multiple texture case is analogous to the multiple motion case (and thus the problem is easy to solve) when (1) the contrast between the two textures is large compared to the contrast within either texture; (2) one can see ``enough'' of each texture to perform shape from texture. These conditions seem to correspond well to conditions under which humans can perform the task, which suggests some interesting psychophysics.
Black, M. J. and Rosenholtz, R. A computational model for shape from texture for multiple textures, Investigative Ophthalmology and Visual Science Supplement, Vol. 36, No. 4, March 1995, p. 2202.
Black, M. J. and Rosenholtz, R. Robust estimation of multiple surface shapes from occluded textures, International Symposium on Computer Vision, Miami, FL, Nov. 1995, pp. 485-490. (abstract)