CS 143 / Project 2 / Local Feature Matching
The goal of this project was to develop a feature dection and matching pipline modeled on the SIFT pipeline. A Harris corner detector is used to find possible feature locations, then a 128 dimension feature is created at each point using directional gradient values. Features are matched between images by finding the most distinct k-nearest neighbor matches between features. MATLAB built-ins and matrix operations were used when possible to try to decrease computation time. Most of the optimization of the pipline was performed in get_interest_points, where many different non-maximum suppression techniques were attempted. The first was finding the most intense >1% (more than 3.5 standard deviations above mean) of interest points globally however, this left interest point clusters that slowed the get_features step without adding much information. The next implementation was a sliding window filter that would return only the maximum value in a 5x5 pixel square. This implementation fixed the feature clustering problem, but was very slow. Sequential column-wise and row-wise non-maximum supression was implemented and was fast but left many weak interest points that degraded matching performance. In the end, a hybrid approach was used in which the most intense ~2% of interest points was taken globally, then filtered with a 5x5 sliding window maximum filter. If more than 2000 interest points are returned by that process the filter window is iteratively increased to reduce the number of interest points that get_features must consider. Below is a demonstration of the pipeline matching features on two different images of the Notre Dame. The pipeline achieves 88% feature matching accuracy when considering the 100 most distinct feature matches. This pipeline is intensity-invariant because feature values are normailized, however it is not scale or rotation invariant, therefore the matching accuracy will decline when considering two more disparate images.
Matched features outlined in green, unmatched in red |
The following are features found in other sets of image pairs. Though there is no ground truth to compare to, by looking at the features picked out of the images one can guess that the more the images differ in scale and rotation the worse the feature matching will be. Here are additional images in descending order of feature matching accuracy.
Capricho de Gaudi, probably good matching |
Sacre Coeur, probably OK matching |
Gaudi Episcopal Palace, probably bad matching |