CS129 / Project 6 / Automated Panorama Stitching
Example of a panorama generated from 3 separate images of a scene.
In this project, I implement automated panorama stitching. This involves 4 steps: (1) defining correspondences between two images; (2) recovering the homography using RANSAC; (3) warping one image using the recovered homography; (4) compositing the images. In particular, I implemented algorithms to execute tasks (1) & (2); (3) & (4) were already provided to me in the support code.
I will be describing how I performed steps (1) & (2) in this report, as well as showcasing some of the panoramas I've stitched together.
(1) Defining Correspondences Between 2 Images
This consists of 4 sub-steps: (A) Harris interest point detection, which is provided in the support code; (B) Adaptive non-maximal suppression; (C) Feature descriptor extraction; (D) Feature matching.
(B) Adaptive Non-Maximal Suppression: This is described in Brown, et al, section 3. Interest points are sorted based on their suppression radius, which is the radius in which that particular point is strongest. This allows us to recover interest points spread out across the entire image, which is important because the Harris detector may return a large number of interest points clustered around certain points in the image.
(C) Feature Descriptor Extraction: There is a pre-determined patch size which we look at whenever we look at an interest point. However, instead of looking at all the points in the patch, we only sample one pixel for every five pixels, a method that increases the reliability of feature matching. The intensity of these sampled pixels are used to "describe" the interest point.
(D) Feature Matching: Features are matched based on the feature vectors computed in (C). Using Lowe's ratio test (ratio of distance to 1st nearest neighbor to distance to 2nd nearest neighbor), we can match an interest point in picture 1 to an interest point in picture 2.
(2) Recovering the Homography Using RANSAC
We want to recover a projective transformation H which is a 3x3 matrix. This means we need to solve a system of equations with 8 unknowns -- so in reality, we only need 4 matching points in our two images of interest. Using RANSAC, we can isolate the points that are truly matching, instead of running into a problem where we might be using false matches as one of our matching points to compute the homography.
To do this, pick 4 random points, and then compute the homography. Test the other points on the matrix we have obtained, and record the number of "inliers" (ie points which are transformed to a position not more than 0.5 units away from their corresponding matches). Repeat this many times (I repeated this sequence 1000 times per picture), and take the homography which gives us the greatest number of inliers. This will be the most accurate transformation that we desire.
Results
The last two images in this section are my own images.
|
|
|
|
|
|
|
|
|
Discussion of Results
Results were generally decent, but there were some locales that didn't correspond perfectly, resulting in a low-quality panorama with artifacts like blurred regions. These varied across images - some images look almost like a single image, while some were very obviously semi-poorly stitched together. One way to improve this might be to increase the number of repeats for RANSAC so that a better homography is found. Another way might be to tweak some of the parameters, such as the Lowe's ratio or the number of interest points after adaptive non-maximal suppression.
By Kai Herng Loh (Computer Science-Economics, Brown '14)