In this project, structure from motion is accomplished in the following process:

1. Track interest points: Given N images, get interespoints in the first frame and track them along subsequent frames. The process is done by computing the optical flow at every 2 consecutive frames and get the displacements of the interest points at the later frame.
2.Structure from Motion: From the flows obtained in step one, apply the procedure described in Shape and Motion from Image Streams under Orthography: a Factorization Method (Tomasi and Kanade 1992), then eliminate the affine ambiguity described inSection 2.3 of A Sequential Factorization Method for Recovering Shape and Motion from Image Streams, Morita and Kanade, 1997.

In the procedure of finding the affine motion, we need to get the Singular Value Decomposition of the measurement matrix D. The paper assumes 2F > P, with F is the number of frames and P is the number of interest points. In our test case, we only have 50 frames whereas the number of points is 500. The paper suggest calculating the SVD of the D transpose.

The result in fact does not differ much from each other, since we can get the same U, W, V matrices by calling either:

[ U W V ] = svd (D) or [V W U] = svd(D')

As displayed in result images, the structure and camera appear correctly in both cases, but are flipped.

1. First and last frames.

The followin images show the first and last frames of the image sequence, with the interest points plotted. Red points survive through the frames, blue points move out of the picture at some frames.

2. 2D paths.

The left image shows 2D path of all points, while the right image shows only the first 20. For some reason the points are flipped horizontally. I use the same code for both of them.

3. 3D points and camera.

The following images show 3D points (triangulated) and the camera (red).

Using measurement matrix D:



Using transpose measurement matrix D':