|
| The results of my point tracking in the final frame |
This webpage displays some of my results for the Brown University CS 1430: Introduction to Computer Vision Project 5: Tracking and Structure from Motion. In this project, I implemented a structure from motion algorithm by finding the optical flow across a series of images and reconstructing the 3D structure .
The structure from motion pipeline that I implemented goes as follows:
First, I generated key points using the Harris Corners detector and selected 500 of the strongest points. Then I tracked these points across a series of images. If a point's predicted location left the image at any point, I discarded it. In order to do this I had to calculate the optical flow. I did this using some matrix manipulation and looking up a points predicted next location assuming that the movement between frames would be small and brightness would be constant. For information on the math involved, see the project assignment: Project 5: Tracking and Structure from Motion.
 |
| Above: The initial locations of the key points |
 |
| Above: The points which move off screen at some point |
 |
| Above: The predicted future locations 20 key points across the frames |
 |
| Above: The predicted future locations all key points across the frames |
 |
| Above: The optical flow across 2 frames |
Once I had the optical flow and the predicted locations of the key points, I could calculate the structure of the 3D object. To acquire this structure, I centered all of the key points. Then I created the measurement matrix that includes the data for how all points are seen in all 2D images. I then factorized this matrix by using a matlab function to compute the SVD. This allowed me to calculate the motion(affine) and shape(3D structure) matrices. There remained one problem, however, affine ambiguity. To eliminate this ambiguity, I followed a the metric transformation process in the Morita-Kanade 1992 paper.
Here are 3 3D views of the construction:
 |
| View 1 |
 |
| View 2 |
 |
| View 3 |