Oh La La~~~ The Final Project

1. Intro

So basically we detect a bunch of interest points and tract them over couple of frames to get their 3D world coordinates. Simply as that.

2.Finding interest points and tracking them

Using harris corner detection, we could get 500 interest points like this:

Clearly, you can see we only want the points to be on the hotel but not on the ground, as this would result in some really ugly deformed shape that is not a hotel. So what I did was pretty much an ugly hack but ehhh, it works. Since it is an intensity image, I just dropped the intensity under a certain threshold and make them zero, the image along with the points would look like this:

Now we begin tracking, yay~~~
This is done by optical flow, nothing fancy. Here are three pictures of randomly chosen 20 points and their paths along the 51 frames over the first frame. The little circles are the ending points of each tracked points.

There are some points that are gonna fall off the frame bounds at some points, we have to toss them away. Here are all the points, points that stay and points that are tossed away:

3.Structure from Motion

Now that we have the points and their movements along the frames, let's build the model, so much fun!
Here is the structure:

The red fan-shaped object is actually the vectors of the cameras of each frame, they are so close together that, they appear this way. Since we constructed and orthogonal camera, it really doesn't have a precise location. What matters is just the direction.