The algorithm I used was to scale the image down with an image pyramid. Each image in the pyramid is scaled down to half the size of the previous step, until it gets to 10 pixels on a side. Then images are then compared at the smaller scale to find the shift that best matches the images. The best match is determined by the sum of squares of the two images, selecting the shift that has the lowest value. The images are shifted 3 pixels in each direction, for a total of 7x7, or 49, comparisons per layer. The borders of the images are removed from the images for the comparisons, because they tend to be unrelated to the content of the images (noise). To do that, 1/8 of the image is removed from each side (after applying the circshift). The best shift is then passed back (the algorithm is recursive), where it is adjusted for scale, and the process repeats. Once back to the full-size image, the program returns the best shift.
When running on single-scale, the program just compares the full-size images with a 5%-pixel shift range (in each direction). This method is significantly slower.
However, this one image in particular was troublesome. This is because the three color channels did not have values that match well. As you can see in the image to the left, the man's clothes range from having very high values (top) to having very low values (bottom). For this image, sum of squares isn't effective, because the differences can be greatly reduced, for example, by pushing the high-valued clothes from the top channel into the high-valued door of the bottom channel.
This problem only happened when I did the multi-scale comparisons (probably because the man gets shifted when the resolution shows him as just a light/dark blob). When I did the comparisons using the slower, single-scale comparisons, the picture came out ok.
Here's how the composited picture turned out:
For extra credit, I implemented a simple white-balance algorithm. In each image, I scaled each channel so that the average value would be 0.5, and this had mediocre results. For some images, the white-balance output is an improvement, and for some it's about the same or worse. In general, I noticed that blue-ish pictures would get redder, and red-ish pictures would get bluer.
Here are some examples which I think are improvements:
Without white-balance
|
With white-balance
|
|
|
|
|
And here are some examples which I think are not improvements:
Without white-balance
|
With white-balance
|
|
|
|
|
