Project 1 : Image Alignment with Pyramids

Valay Shah (valayshah)
September 15, 2012

Goal

The goal of this project is to extract the three color channel images from the digitized Produkin-Gorskii glass plate image and automatically align them to form a single RGB color image with the least number of visual artifacts.

Background

Sergei Mikhailovich Prokudin-Gorskii (1863-1944) was a photographer ahead of his time. He saw color photography as the wave of the future and came up with a simple idea to produce color photos: record three exposures of every scene onto a glass plate using a red, a green, and a blue filter and then project the monochrome pictures with correctly coloured light to reproduce the color image; color printing of photos was very difficult at the time. Due to the fame he got from his color photos, including the only color portrait of Leo Tolstoy (a famous Russian author), he won the Tzar's permission and funding to travel across the Russian Empire and document it in 'color' photographs. His RGB glass plate negatives were purchased in 1948 by the Library of Congress. They are now digitized and available on-line.

Algorithm

The algorithm used in this process finds the best shift vector which rightly aligns two color channels together. The algorithm is divided into following steps :

The main idea of this algorithm is to align images using image pyramids. The color channel images are smoothed and subsampled. This process is followed until the coarser image (ont the top of pyramid) reaches a certain minimum image size threshold.
In Next part, 'edges' are detected in each image followed by applying SSD (sum of squared distances) image matching metric in order to find the shift vector and this process is followed every time we travel down in the pyramid. The shift vector returned by this method is the shift needed to align the three color channel images.
Though aligned, the color image has these ugly borders of different colors (Mostly black and white). In order to check this artifact 'Sobel' filter is used with very less threshold which gives us all the horizontal and vertical edges (Mostly between border and actual image). Then by searching over a specified portion (relative to image size) near the border, false borders are cropped through edges.
The border cropped image is then color adjusted in order to improve the appearence of the image.


1. Without alignment	2. Alignment using Image Pyramid (with canny edge).

3. Border cropped.	4. Color Adjusted.

Some example images produced by applying above mentioned algorithm.

Extra Credit

Extra Credit Extension #1 Better Features

Applying SSD or NCC on the edge detected images gives better alignment then directly applying SSD or NCC on the intensities of three channel images. The "Canny" edge detector is applied on the images from all three color channels and then the image matching metric SSD is applied on them inorder to find the best possible alignment. The diffrence can be seen between the following images (Not Border Cropped).
Extra Credit Extention #2 Automatic Border Cropping

As channels are shifted relatively from each other border artifacts appear in the aligned image. 'Sobel' filter (with very less threshold) is used to detect vertical and horizontal borders . These borders are removed/cropped from the image by searching relatively (to image size) near image border.
Extra Credit Extension #3 Automatic Color Adjustment

The algned color image need some color adjustment as the average intensities of three color channel images differ by a greater amount. So by taking the average of the intensity values in each color channel image and then comparing it to the average intensity across three color channels give much better results. As shown in figure the red channel has more effect on aligned image then green and blue which makes the image slightly orange. The above process corrects this artifict.