Name: Chen Xu
login: chenx
The algorithm mainly has two parts:
1. We first need to use harris interest points detector to detect interest points for both images as our matching pair candidates. Before extracting features, we do adaptive non-maximum suppression to mine points with high strength, the adaptive part of this algorithm makes the points selected close to uniformly distribution. Features are extracted as 8 by 8 patches.
2. Use RANSAC to discover pairs and homography. RANSAC assumes there are good pairs exsiting and we find the homography with most number of inliers.
Table 1 shows results (40x40 patch based)of given test cases, two parameters should be carefully tuned, one is the ratio of distances from 1NN and 2NN, the other is the error tolerant value of RANSAC.
| Patch based |
|---|
|
|
|
|
|
|
|
|
Table 2 shows more results
| Patch based | Source |
|---|---|
 |
   |
 |
  |
I try to add rotation invariance to the feature descriptor, what I use is SIFT feature from the VLFeat library, it gives better results when the images are slightly rotated. But projection will accumulate when more and more images are stitched together, and results will become very bad quickly. So it's very hard to create global panorama, especially for stitching the last piece.
Table 3 shows results of using SIFT as feature.
| SIFT based |
|---|
    |
 |
 |
 |
 |
When detecting interesting points from warped image, sometimes the algorithm will detect points on borders, which are artifacts due to warping. These wrong detections will greatly affect the correctness of homography. One way to avoid this is that when using distance ratio of 1NN and 2NN to filter matches, we'd better for each point in next source image, looking for matches in the warped and composited images, rather than for each point in warped and composited image, looking for matches in the following source image. The reason is that those border false positive points descriptors are alway similiar and result in similar distances to points in the other image, if we do the ratio test in the correct direction, those matches can be filtered out easily. Of course, a better way is to do the ratio test bidirectionally.
Table 4 shows how ratio test directions affect results. We can see that, if we look for matches in new images for points in warped image, the wrong detections (on the border above sky) cannot be filtered out by ratio test. However, looking for matches in warped image for points in new image helps ratio test to filter out most of the false positives on borders that above sky.
| Find Matches in New Image | Find Matches in Warped Composited Image | |
|---|---|---|
| Before Ratio Test |  |
 |
| After Ratio Test |  |
 |