Tech Report CS-05-04

Look-That-There: Exploiting Gaze in Virtual Reality Interactions

Robert C. Zeleznik, Andrew S. Forsberg, Jurgen P. Schulze

March 2005


We present a suite of interaction techniques that fundamentally leverages the user's gaze direction to provide a range of potential benefits over existing techniques such as reduced arm fatigue, more powerful interaction, and more specialized interaction. Because measuring true gaze direction is problematic, we instead approximate gaze with a non-linear mapping of head orientation that reduces neck strain when looking up or down.

Given the immaturity of gaze-assisted VR interaction, we chose to prototype interaction designs across a variety of fundamental VR tasks that includes 3D point specification, 3D movement, and environment navigation. For each basic task we created a range of exemplary gaze-based techniques that populate three classifications: "Lazy" interactions that minimize or obviate hand movement, "Helping Hand" techniques in which gaze augments conventional interaction as if with an extra hand, and "Hands Down" manipulations in which gaze offloads the hands so that they can operate specialized devices such as a tablet.

Specifically, this paper presents Look-That-There, a technique for moving objects in a virtual environment that does not require hand movement, in addition to gaze-based techniques for selecting menu items, specifying arbitrary 3D points or regions, and orbiting and flying.

(complete text in pdf)