CSCI 1430 Project 1 : Image Filtering and Hybrid Images
Goal
The goal of this project was to write an image filtering function, and then use it to create hybrid images. This was done by using a more basic version of the methods and theory presented in the 2006 SIGGRAPH paper by Oliva, Torralba, and Schyns.
This is a catdog. They are not found in the wild, but you can find them by extracting the high frequencies from a picture of a cat, the low frequencies from a picture of a dog, and then adding the extractions together.
Explanation
The project can be divided into two parts. The first part was to create our own version of MATLAB's imfilter(). This was done by taking as input an image and a filter, padding the image appropriately (in my case using symmetric padding with the help of padarray()'s option), and then applying the filter to the resulting image matrix. This was done by taking the padded image, putting it through im2col() which turned every filter_height by filter_width tile of the image into a column in a separate matrix, and then taking the dot product of that matrix and another matrix that had each column as the values of the filter. Finally, col2im() was used on the result. For color images, this process was done on each color channel. The returned object was the now-filtered image. Because we only used Gaussian filters in this project, my imfilter implementation only returned the low frequencies of the input image (which is what Gaussian filters do). A high frequency image was found by taking the original image, and then subtracting the low frequency image which was found with a Gaussian filter. This approximates the Laplacian of the Gaussian filter, resulting in an image representing the high frequencies of the original image. The second part was to simply add the high and low frequency images together, to form the 'hybrid image'.
Notice that the doggy has undergone a Gaussian filter (blurred/smoothed, low frequencies left in the image), while the kitty is representing the Laplacian of a Gaussian filter (high frequencies exposed).
Discussion of Hybrid Images
Hybrid images take advantage of the fact that high frequencies are much more prevalent to human vision when an object is physically close, while low frequencies become more prevalent when an object is far away. Thus, hybrid images work by displaying one object when the image is seen up close, and another object when the image is further away from the eyes (or when the image becomes smaller, which has the same effect as increasing the distance between the receptors and the image).
The method of forming a hybrid image is shown below in MATLAB pseudocode.
Image1 = [n n];
Image2 = [n n];
low_frequency_image = imfilter(Image1, Gaussian_filter);
high_frequency_image = Image2 - imfilter(Image2, Gaussin_filter);
hybrid_image = low_frequency_image + high_frequency_image;
Results
The following two tables each contain a row of hybrid images, followed by a row of their low frequency component and their high frequency component.
|
|
|
|
|
|
The following are the cutoff frequency settings for each hybrid image. Check them against the hybrid image file name if you are confused. fish and sub = 8; dog and cat 1 = 7; dog and cat 2 = 5; marilyn and einstein 1 = 3; marilyn and einstein 2 = 10; bird and plane = 4; bike and motorcycle = 6; motorcycle and bike = 3;
Selected Examples
Notice that the above Einstein to Marilyn transformation is much more believable and observable than the below. This is because the cutoff frequency for the top image was 3, and the below image was 10. Thus, much more of the high frequencies in the original Einstein picture and much fewer of the high frequencies in the Marilyn picture made their way into the hybrid below, so you cannot really see Marilyn at any size. However, for the top image, even at maximum size, you can still see some of Marilyn's features in what looks like a very glamorous Albert Einstein. Of course, the specific cutoff settings are image-dependent, but the pattern will hold.
The following two hybrids came out well. This is likey because the images are properly aligned, and they are rather similar in shape and inner-filling.
The following two hybrids did not come out as well. I suspect this might be because the motorcycle has a large section that is filled in the center of it (the engine and its components), while the bike simply has a large hole there. They are also not very similar in shape besides the wheels. Perhaps they would work better if I used different settings for them.