CSCI1430 : Project 1 Hybrid Image

yunzhang

The goal of this assignment is to create hybrid images whose interpretation can be changed as a function of thew viewing distance.

Algorithm

  1. Image alignment
    2. Two pairs of corresponding points are manually specified to indicate parameters used in image transformation.
  2. Laplacian pyramids creation
    3. We first construct the Gaussian pyramids of level N for two input images, using matlab builtin functions imresize(), which uses bicubic filter as smoothing function by default. Two Laplacian pyramids of level N-1 are then constructed based on their corresponding Gaussian pyramids. When calling imresize() function to perform the blurring by scaling down and up, we choose the value of scalar case by case, ranging from 0.5 to 0.7. We found tuning this value carefully is important because it affects where our later cutoff is on the frequency map.
  3. Image Hybriding
    4. The hybrid images are created by summing some low-pass levels of the first Laplacian pyramid and high-pass levels of the second. Again we choose the cutoff case by case to gain the best visual effect of the final images.

Extra Credit

We use color to enhance the visual effect of result images. We experiemented with following approaches:
  1. Construct a gaussian pyramid and its corresponding laplacian pyramid for each channel of two input images, do the blending as described above for RGB channels separately and combine them tegether to form the final results.
  2. Convert from RGB space to CIE L*a*b color space first, then create the hybrid images only on the intensity channel. We combine the other two color channels from the low frequency image with the hybrid intensity channel to recreate a color image in L*a*b channel and convert back to RGB space.
The images we created by these two approaches looks basically the same.

Results

For each pairs of images, we run our algorithm on different settings of scalar, cutoff and number of pyramid level, and show only best compositing results here. Please click the thumbnails to view images in original sizes.

Cat & Dog ( salar=0.7, cutoff=5, N=8 )

Marylin & Einstein ( scalar=0.6, cutoff=3, N=8 )

Derek & Cat ( scalar=0.6, cutoff=3, N=8 )

Discussion

As per Oliva et al., there are two primary mechanisms can be exploited to create compelling hybrid images. The first is maximizing the correlation between edges in the two scales so that they blend. The second resides in the fact that the remaining edges that do not correlate with other edges across scales can be perceived as noise. Based on these principles, we can further enhance our results in follow ways:
  1. Do better alignment: We can specify more corresponding points so that a more complex image warp marix can be developed to align some strong edges in two inputs. For example, in Marylin-Einstein test, we can stretch one of the faces to make sure their eyes, eyebrows, noses and mouth align well, then other weak edges will be treated as noises when we look at the image.
  2. Do local blending: Our algorithm so far does the blending merely based on a global cutoff of frequency. We can devise a smarter algorithm which performs the cutoff for each local regions. Doing this will give us an overall better decision boundary. Taking Marylin-Einstein test as instance again, one reason our result looks way from perfect is that Einstein's beard, which is not supposed to appear in the low frequnecy bands, still dominate our visual perception when we stand away from the image. For such regions, we may want to set the cutoff to favor more to low frequency information from Marylin's image.