Project 6 Writeup

David Dufresne (ddufresn)
April 19, 2010

Algorithm

The goal of this project was to perform automated panorama stitching.

A projective transformation can be computed from four pairs of corresponding points using a system of linear equations. Choosing the correct points to define this homography is non-trivial.

First, Harris corners are detected in each image. Nonmaximal suppression is performed to ensure the interest points are distributed fairly evenly. Then, for each Harris corner, a feature vector is defined. To avoid aliasing, the a region of approximately 40x40 pixels centered at the corner is sampled and scaled to 8x8. Feature matching is performed by associating each feature of one image with the closest match in the other image. Feature pairs are only kept if the ratio of the error of the best match over the error of the second best match is less than a threshold, in this case 0.3.

A Visualization of Corresponding Points

The projective transformation is then determined by the RANSAC algorithm. For 1000 iterations, four pairs of corresponding points are chosen, and a transformation constructed. Errors values are obtained by transforming the remaining points of one image using this transformation, and calculating the distance to the corresponding points in the other image. If the distance is less than 0.5 pixels, the pair is added to the consensus set. If the consensus set is 10 elements long or larger, then the transformation is a good one, and its total error is compared to the best error of previous iterations of the algorithm.

I made an additional change to the algorithm. If no suitable transformation is found in 1000 iterations, I decrement the minimum consensus set size and run RANSAC again. The minimum size of a consensus set is 4, which merely includes the pairs of points from which the transformation is generated.

An Example Where No Suitable Transformation is Found for a Consensus Size of 10

Results Images

Rectified Bus

Discussion

I noticed that I got better results when images were warped to the middle image of the sequence instead of the first one. The panorama is generated by iteratively adding each picture from the middle image to the first image to the panormama, and then adding images from the middle image to the last image.

Warping to First Image	Warping to Middle Image

Sometimes, this causes the a dark outline to appear within the panorama, which I believed was a result of a rounding error. I found the simplest way to fix this was to only add pixels that were not NaN to the mosaic. I have not tested this on image sequences of sizes greater than 3, but in theory it should still work.