CS 143 / Project 3 / Scene Recognition with Bag of Words

I created a basic implementation of the tiny image/nearest neighbor pipeline, the SIFT feature/nearest neighbor pipeline, and the SIFT feature/SVM classifier pipeline. The tiny image/nearest neighbor method resulted in an accuracy within the expected range: Accuracy is 0.205.

The SIFT feature representation and nearest neighbor classifier performed slightly worse than expected. Without any of the extra credit methods, its accuracy was 38.8%, which is still significantly better than the tiny image method. I created the vocabulary that bag of SIFT uses to form a histogram with a sampling of SIFT features from all 1500 images, with a step size of 50, also using the fast parameter. This resulted in roughly 200 features per image, for a total 300,000 features in the vocabulary. This data was then used to create 400 kmeans clusters. My bag of SIFT method sampled from the test images with a step size of 20, and compared each new feature to the vocabulary and create a normalized histogram for each image. The histogram results were then run through the nearest neighbors classifier.

Finally, the SIFT feature representation and SVM classifier performed better than the previous two methods, though still slightly out of the 60-70% range. Fiddling with parameters improved the result up to 53-56% accuracy. This example had an Accuracy of 54.7%, using a lambda value of 0.001. Interestingly, these accuracy values seemed to vary slightly between trials, without any change of parameters.