Wonnie Sim (ssim)

The top 100 most confident local feature matches from a baseline implementation of proj2. In this case, 72 were correct (highlighted in green) and 28 were incorrect (highlighted in red).

1. Algorithm - Interest Point Detection

To detect distinctive points, I implemented the Harris Corner Detector. Harris Corner Detector has 5 big parts to be implemented. First, get image derivatives (blurring first can be done optionally to get rid of some boundary noise). Second, calculate squares of derivatives. Third, apply Gaussian filter to each squares of derivatives. Fourth, Calculate cornerness function (har=g(I_x^2)g(I_y^2)-[g(I_x*I_y)]^2-alpha*[g(I_x^2)+g(I_y^2)]^2). Lastly, suppress non-maximum values. Before moving on to non-maxima suppression step, I changed too-small values to 0 and changed the boundary values as 0 for easier getting-features step.

2. Algorithm - Local Feature Description

The next step is to get features of each interest point that we got in step1. To do this, I took Gaussian filter first, and filtered according to each direction (we have 8 directions here). Then, I set a 16*16 window for each point and cut the window into 16 4*4 small pieces. After, I calculated mean value and set it as a feature of each block in each direction. After this step, I got 1*128 feature vector; 128 came from 8(directions)*16(4*4 size small pieces in each window).

3. Algorithm - Feature Matching

Last step is to compare features in image1 and image2, and find the matching points. The distances between two features are calculated by summing up of a square result of differences between features in image1 and those in image2. (sum((features1-features2).^2)). After getting distances from a set of features in image1 and the whole features in image2, sorting the distances and their indices to find which features in image2 is the most lookalike as those in image1. While finding the matches, I calculated the ratio as well by dividing the shortest distance to the second shortest distance to calculate its confidence. In this project, I assumed confidence is equal to 1/ratio.

4. Result

I first used the stencil code as it was given at the first and I got about 1000 points which were most likely incorrectly matching (204 were correctly matching and 838 were incorrectly matching.) So I decided to take top 100 most confidences feature matches instead. We know the confidences for these 100 matching points are high, which means feasible, so it explains more about the final result instead of seeing all of the matching points, which have low confidences.

Below two pictures are the results of all matching points. Among two, upper ones are the matching points, and the bottom ones are highlighted according to its correctness (correct matching is green and incorrect matching is red).

Below two pictures are the results of top 100 most confidences matching points. It is distinguishable that the results are more recognizable (how accurate this algorithm is) when I took the top 100 confidences matching points. Again, top two are showing matching points and the bottom two are reflecting their correctness (correct ones are green and incorrect ones are red).

5. More Images

Additionally, I tried two more sets of images to see if how my algorithm works well. Upper ones are Capricho Gaudi and the bottom ones are Episcopal Gaudi. Unfortunately, I could not evaluate my results but with naked eye, I could find some correctly matching points based on small points' colors and their spots as well as which points have focused distinctive points.