Tech Report CS-99-15

Whole-Hand and Speech Input in Virtual Environments

Joseph J. LaViola Jr.

December 1999

Abstract:

Recent approaches to providing users with a more natural method of interacting with computer applications have shown that more than one mode of input can be both beneficial and intuitive as a communication medium between humans and computers. Two modalities in particular, whole-hand and speech input, represent a natural form of communication that has been ingrained in our physical and mental makeup since birth. In this thesis, we investigate the use of whole-hand and speech input in virtual environments in the context of two applications domains: scientific visualization and interior design. By examining the two modalities individually and in combination, and through the creation of two application prototypes (Multimodal Scientific Visualization Tool and Room Designer), we present a number of contributions including a set of interface guidelines and interaction techniques for whole-hand and speech input.

(complete text in pdf or gzipped postscript)