Flora Jin fjin

Harris Corner Detection

The Harris Corner Detector consisted of the following steps (Slide 32 in Lecture 8):

1. Take image derivatives (as dx and dy)
2. Square the derivatives (dx_2, dy_2, and dx_dy)
3. Use a Gaussian filter on each (gdx_2, gdy_2, gdx_dy)
4. Use the cornerness function to find spots where both eigenvalues are strong:
   
   har = gdx_2*gdy_2-[gdx_dy^2]-alpha*[gdx_2+gdy_2]^2
   where * and ^ operators indicate elementwise operations

From these corners, I was able to find the interest points by using a 3x3 pixel sliding window and saving the coordinates of the most intense pixel in this sliding window. I also used a threshold parameter to specify the minimum intensity of each saved pixel. Many of the parameters for this portion can be tuned accordingly. These include:

• Gaussian window size
• Gaussian parameter
• Alpha in the Harris Corner Detection Equation
• Intensity threshold

These resulting pixels get passed to the next step, Feature Extraction.

Feature Extraction

Here is an outline of the steps for coming up with a way to represent features of each interest point:

1. Get image gradient.
2. For each interest point:
1. Generate a window of pixels (of length and width = feature_width) from the image gradient
2. For each window, split into 16 blocks of equal size (split window into 4x4)
3. Create 8 bins, which will each hold a range of angles (or directions)
4. Put each of the pixels in each block into a bin based on its direction in the gradient
5. Aggregate the list of bins for each block. This set of 128 (4*4*8) values represents information about the interest point
3. Pass the list of feature descriptors (the 128 values from step 2) to the Point Matching step.

Point Matching

To match points in one image to points in another, do the following:

1. For each point in image1, get the 2 closest points to this point from image2. Let's call them im2_pt1 and im2_pt2 for the closest and second closest points, respectively.
2. The ratio we are interested in will be the ratio of the distance of im2_pt1 from our interest point to the distance of im2_pt2 from our interest point. That is,
   
   ratio = dist(point-im2_pt1)/dist(point-im2_pt2)
   
   
3. If the ratio is below a certain threshold (setable), then we return the two points as a match.

Reasoning for Design Decisions

I originally decided to choose interest points by looking at connected components. However, this was yielding very poor results so I switched to the sliding window implementation. It seemed like the connected components should've worked better, but I think the reason why it didn't was because it ended up returning less interest points than the sliding window method. Another change that boosted my program accuracy was that during the feature extraction step, I take the gradient of the image before splitting each image into smaller windows and smaller blocks. Previously, I took the gradient for each block. This doesn't work as well though because the gradient values near the edges will not be as accurate. By taking the gradient for the entire image first, you avoid needing to fill in the edges for each block.

Sample Results

Matches	Accuracy
	24 good matches, 4 bad matches 85.7%
	Features matched in the correct general areas.
	A couple points match exactly, but most are in the correct general areas.
	Matched one point (blue) perfectly, but the others are not exactly in the correct place. This shows the limitations of a basic SIFT implementation.