Automatic Panorama Stitching
CS129 Computational Photography Project 6
November 26, 2012
Background
A panorama is any wide-angle view or representation of a physical space, here a photograph. Panoramas can be generated by stitching number of overlapping images seamlessely. This project performs image stitching to generate a panorama
Goal
The goal of this project is to generate panoramic images. The panorama construction pipeline involves following things
- To define corespondence between images to be stitched i.e. to find corresponding points in both images.
- To recover homography using RANSAC (RANdom SAmple Consensus).
- Warp the image using the transform obtained from RANSAC.
- Composite the images seamlessely.
 |
 |
 |
Algorithm
Defining Correspondences between images
Finding the best corresponding points is an important part of this process. To be able to stitch images we need these corresponding interest points. The procedure used to find corresponding points is based on paper from Brown, et al.
- Harris corners: Harris corner detector is used to generate the interest points with some corner strength. The following images shows the interest points generated by the Harris corner detector.
Input Image Harris Corners 
- Adaptive Non-Maximal Supression: The Harris Corner Detector provides large number of interest points that has some local non-maximum supression in a 3x3 grid. But we want the points to be spatially diverse. Adaptive Non-maximal Supression algorithm developed by Lowe is used to get feature points which are evenly distributed throughout the image. The output is 500 interest points well distributed spatially.
Strongest 500 harris corners Adaptive Non-maximal Suppression 
- Feature Descriptors: The next step in the process is to compute features of the interest points that encodes color information around the interest point. For my descriptor I used 8x8 grid around the interest point with spacing of 5 pixels. Each descriptor is normalized by subtracting mean and dividing it by standard deviation.
Image 1 
- Feature Matching: After computing feature descriptors, they are compared with features of the other image. The features with the least SSD were selected. From these features, those with the ratio of the best match and the second best match greater than certain threshold were rejected. I used 0.3 as the threshold as it gave me points that perfectly matched with each other. The following figure shows the comparision of simple SSD with SSD and Lowe's ratio test. The Lowe's ratio test removes the false matches pretty well.
SSD matches SSD with Ratio 
Recovering Homographies
- Recovering projective transformation: To recover projective transformation H which is a 3x3 matrix with 8 degrees of freedom, we need atleast 4 pairs of corresponding points. I followed the paper projective mappings to compute the transformation matrix.
- The feature pairs obtained from feature matching can still contain outliers (zero in my case as threshold is very less). To remove the outliers and compute the best homography the RANSAC algorithm is used. In this algorithm four feature pairs are randomly selected to compute homography. This homography is applied to all feature pairs and inliers are stored. The inliers can be computed by SSD(p',H*p) < 0.5. This process can be repeated until most inliers are obtained. I used 1500 iterations of RANSAC. Then the projective transformation is computed from these inlier points.
- Given the transformation the images are composited to create a panoramas. In compositing two images instead of averaging the both images in the overlap part I took the max of them. The difference is shown below.
Compositing with averaging (Blurred Output) 
Compositing with maximum (Less Blurring) 
Extra Credit : "Panorama Recognition"
Detecting panoramas is really easy. I computed the features for each image from the set of images. The features of each image is compared with features of every other image in the input set. Those images with number of feature matches greater that 30 were selected and used to generate panoramas. The following images were given as input and the output panorama after auto detection is shown below:
Results : Own Images
Image 1
Image 2
Image 3
Image 4
Image 5
Results : Given Images
Image 1
Image 2
Image 3
Image 4
Image 5
Image 6
Image 7
Image 8
Image 9
Back to TOP