I implemented these three main parts of the panorama algorithm:
1. Adaptive non-maximal suppression fits into the pipeline right after the feature points are detected. I tinkered with how many points to suppress and discovered that keeping only a third of the points had the best results—keeping anything more didn't really provide any benefit over the unsupressed version, and keeping less often meant there wouldn't be enough matches in the next step. The two images below show the difference between unsuppressed and suppressed, on the left and right, respectively.
2. The next step is to extract patches that surround these feature points and then compare the patches from the first image with those from the second image, subsequently discarding the points which have a ratio of their best match to their second best match which exceeds 0.2 (an arbitrary value that I've found works nicely). These are the remaining points in the image above on the right look like after undergoing this step:
3. The final step recovers the homography between the two images with a RANSAC algorithm. I implemented this as described on the assignment page, with the code that actually calculates the transform taken from Paul Heckbert's "Projective Mappings for Image Warping".
Here're the final panoramas. The final two were created from my own photos.
|
|
|
|
|
|
|
|
|
|
A big limitation is that my feature descriptors aren't rotationally invariant; if two images are very much rotated from each other, the algorithm breaks down, as you can see below. The obvious solution to this problem is to implement the invariant descriptors described in "Multi-Image Matching using Multi-Scale Oriented Patches" by Brown et al.
I unfortunately didn't implement anything above and/or beyond for extra credit.