A Probabilistic Framework For Matching Temporal Trajectories: Condensation-based recognition of gestures and expressions

(with Allan Jepson)

The recognition of human gestures and facial expressions in image sequences is an important and challenging problem that enables a host of human-computer interaction applications. This paper describes a framework for incremental recognition of human motion that extends the ``Condensation'' algorithm proposed by Isard and Blake (ECCV'96). Human motions are modeled as temporal trajectories of some estimated parameters over time. The Condensation algorithm uses random sampling techniques to incrementally match the trajectory models to the multi-variate input data. The recognition framework is demonstrated with two examples. The first example involves an augmented office whiteboard with which a user can make simple hand gestures to grab regions of the board, print them, save them, etc. The second example illustrates the recognition of human facial expressions using the estimated parameters of a learned model of mouth motion.

This figure shows the gestures that are recognized by the system.

The left figure shows a montage of a "Cut Region" gesture. The dots are tracked locations of the "phicon". The figure on the right shows the extracted image region.

Related Publications

Black, M. J. And Jepson, A. D., A Probabilistic Framework For Matching Temporal Trajectories: Condensation-based recognition of gestures and expressions to appear: European Conf. on Computer Vision, ECCV-98, Freiburg, Germany, 1998.

Black, M. J. and Jepson, A. D., Recognizing temporal trajectories using the condensation algorithm, to appear: Int. Conf. on Automatic Face and Gesture Recognition, Nara, Japan, 1998. (postscript).