Project 5 Report

By Andersen Chen

Algorithm: Tracking and Structure from Motion

First, we use the Harris corner detector to select keypoints in the first frame to track. Then, we compute the optical flow between video frames using least squares, and move the selected keypoints from the first frame along the flow field. We discard any keypoints that move out of the picture frame. Now that we've tracked keypoints throughout the video, we can implement structure from motion. We center the feature coordinates for each frame, and factorize the measurement matrix using singular value decomposition. Then, we create the motion and shape matrices. We use least squares and the Cholesky decomposition to retrieve the appropriate transformation matrix Q to eliminate affine ambiguity.

Results

Image of selected keypoints overlayed over the first video frame:

2D path over the sequence of frames for 20 random keypoints:

Points which have moved out of frame at some point along the sequence:

3D locations of tracked points for 4 different viewpoints:







3D path of the cameras, for dimensions x, y, and z (in order, from top to bottom).