CS 1430 Project 5: Tracking and Structure from Motion

Kumud Nepal
December 13, 2011

Overview

In this assignment the 3D shape of an object will be created from a series of observations (a video). There are three essential components to this: keypoint selection, feature tracking, and structure from motion. Keypoints are selected using a Harris corner detector. These features are tracked along the motion and an optical flow model is generated using the Lucas Kanade algorithm. These keypoint tracks are used as input to determine the affine structure from motion. Finally, the predicted 3D locations of the tracked points are plotted from 3 different viewpoints. Also, the predicted 3D path of the cameras is plotted.

Results

Fig. 1: 500 strongest keypoints detected in the first frame by the Harris Corner detector

Fig. 2: Optical flow visualization on the last frame

Fig. 3: Displacement shown by 20 random keypoints from the first frame to the last (shown on the first frame)

Fig. 4: All discarded points, that go out of the frame some time along the sequence

Fig. 5: Structure of the hotel recovered from motion. This is the predicted 3D locations of the tracked points. Predicted 3D path of cameras shown in red

Fig. 6: Viewpoint2

Fig. 7: Viewpoint3