Project 1: Hybrid Images
CS 143: Introduction to Computer Vision

Introduction

In this project, our goal is to produce a hybrid image from two input images, fusing the low frequencies from one with the high frequencies from the other. Low frequencies in images generally correspond to color and large structures, whereas high frequencies carry information about textures and details. By combining these bands from different images, we can potentially create some interesting hybrids which cause different visual impression depending on the viewing distance. In particular, the low frequencies will be more visible when viewed from afar, whereas the high frequencies will dominate our visual experience when up-close.

Implementation

Technically, this can be easily implemented in the frequency domain or a simple filter+combine procedure. As required by the project, we will use a pyramid-based representation to achieve the same effect, and the cutoff frequency will correspond to the level at which the pyramids are being mixed.
Gaussian and Laplacian pyramids provide a lossless over-complete representation of an image, by capturing the low-passed and band-passed frequency bands respectively. If we successively build the Gaussian bands and record the Laplacian bands at all levels, we can reconstruct the original image by collapsing the Laplacian bands plus the final Gaussian band, which represents the remaining low frequencies.
If we specificly blur the image and take the difference to get the Laplacian bands, we will end up not able to recover all the signals because upscaling a low-pass image will never recover the Gaussian blurred image: the downsampling operation is irreversible. AS a result, I form the Laplacian bands by taking the difference between the current image I by Up(Down(I)), where Up is the upsampling operator, Down is the downsampling operator.
Some image pairs are hard to mix by nature. By setting the downsampling factor to 0.8 (instead of 1/2), I build taller pyramids which provide more choices for the mixing level, hence producing more visually pleasing results.

Colors

colors are often a low-frequency phenomenon. Hence, if an image/object can be easily recognized just by color (not because of its texture/shape), it can be used as a good candidate for the low frequency image. For example, if we use the cat image as the low frequency in the hybrid and DerekPicture.jpg as the high frequency, the effect is less recognizable than the opposite mixing order, for example, it is somewhat hard to recognize Derek in the third image below.

Cat as high freq, Derek as low freq, without color

Cat as high freq, Derek as low freq, with color

Derek as high freq, cat as low freq, with color

Results

The following are some results I have collected using the provided images and images over the Internet. The hybrid image the large image, with the low frequency visualization next to its top-right corner. In general, having faces to align is relatively easy and produce decent results. However, most image pairs do not provide easy ways for alignment and hence unpleasant hybrid image output. In some cases I have to reduce the size of hybrid image significantly to visualize the low frequency.