Overview

As far as first projects go, this one was really interesting. We got to take three monochrome pictures and colorize them.

Sergei Prokudin-Gorskii pioneered color photography1 in a time when no color film was produced. By taking three exposures with red, green, or blue filters, he was able to project the negatives with their respective filters and generate a color image.

While extremely innovative, Prokudin-Gorskii's method has its flaws. This technique requires no movement of the camera or the subject. While it's relatively easy to stabilize a camera, ensuring that the subject remained stationary proved not to be so simple. People have a tendency to shift in their seat, so portraits using three different exposures can look blurry even with the best alignment.

Furthermore, his glass plate negatives have degenerated over the years, yielding various "holes" in the images (regions where one color channel is exceptionally represented/underrepresented). This can be a result of various stages of decay of the negatives, chips in the glass, poor scanning of the negatives, dust specks, or a variety of other mishaps that may have resulted in damaged negatives.

Our assignment was to align these images as best we could and make a color image out of three monochrome ones. The algorithm I used is outlined below, and the results of the algorithm follow thereafter.

Algorithm

To align the images, the blue color channel remain stationary while the green and red channels are individually aligned to the blue channel.

A simple approach would be to exhaustively loop over a region of offsets (say, -15:15 px) on the x- and y-axes, and at each possible alignment, test the alignment against some sort of metric (e.g. sum of squared differences). This would result in a fairly slow program, but it would undoubtedly do the job.

A better approach is to use a "coarse-to-fine" strategy by creating Gaussian pyramids. A pyramid approach is relatively simple:

  1. Blur the image
  2. Subsample the image (usually by a factor of two)
  3. Align the images starting from the smallest image first, multiplying your displacement by the subsample factor

My algorithm utilized a Gaussian filter to blur the image, with the filter doubling in size for every iteration. Once the image was blurred and subsampled by a factor of two, I passed it along to the aligning algorithm. The alignment process searched over an area of displacements (-15:15 px for the smallest image, -1:1 px for each subsequent iteration), and scored each displacement using a sum of squared differences metric. Before each alignment was scored, the image was cropped by 20% on each side so that extraneous artifacts on the edges would be ignored, and only the desired content of the photo would be tested. The best displacement was returned, the original red and green color channels were aligned to the blue channel using the returned values, and the final color image was created.

While not necessary for a Gaussian pyramid, I decided to first treat the images to a Prewitt filter. Prewitt filters emphasize the horizontal edges of an image. This helped the matching metric better determine which displacements best aligned the images.

Results

Here's what the program produced!

The images may have artifacts including holes, colorized borders, and not-quite-perfect alignment. Alignment may have been better had rotation been included in the algorithm. Extra credit opportunities could have removed the artifacts.










Extra Credit

So I attempted some extra credit, and despite it not working the way I hoped, it still resulted in a pretty cool pictures.

Using NASA2 photos, I tried to recreate a few color images. Descriptions of the images follow with the results.







A coronal mass ejection occurring on August 31, 2012.

Wavelengths used: 171, 131, 335 angstroms

Source: Goddard Space Flight Center










M101: Pinwheel Galaxy

Wavelengths used: Infrared, Optical, and UV

Source: Chandra X-Ray Observatory









M82: Starburst Galaxy

Wavelengths: Low, Medium, and High

Source: Chandra X-Ray Observatory