CS 143 / Project 1 / Image Filtering and Hybrid Images

A hybrid image of Marilyn Monroe and Albert Einstein.

As the name of the project indicates, the task of creating hybrid images involves two main steps. The first of which is to filter two images using Gaussian filters to grab the low frequencies from one image and the high frequencies from the other. While this is normally accomplished quickly with the built in function imfilter(image, filter) in MatLab, writing it on your own is results in a much slower function due to the speed of for loops in Matlab. Finally after applying the filter's (with different cutoff frequencies) the images need to be combined. This is relatively simply and follows the formula H = I1 * G1 + I2 * (1 - G2) where the I's are images and the G's are filters. The reason for the subtraction is to remove the low frequencies from an image so that the high frequenices remain.

Code and Design Decisions

When completeting the project I quickly figured out how slow the process of filtering was. As a result, I ended up trying with three different ways of running the algorithim, one of which is commented out in the program:

  1. Looping Over the Filter: I loop through the image for each pixel i,j and then each color field k, and then use two more for loops to grab neighboring pixels and perform the convultion with the filter.
  2. Subsections Using im2Col: I grabbed subsections of the image corresponding to the size of the filter. Then I ran im2Col on both the filter and the image subsection which allowed for an extremely easy multiplication of the elements using .* and also made it easy to sum them all up using sum(). This makes the convultion require far less lines of code, but appears to be just as slow.
  3. Simplifications: The final change I made to speed up the my_imfilter was to remove im2col since I realized you can just .* for any matrix, not just vectors, and I removed the average function since the multiplication already took that into account.
  4. Grayscale: I had to add an if statement at the start of my_imfilter to account for black and white images. Since black and white images do not need a third dimension in their matrices I just used the same code for filtering but elimated looping over the different color fields (RGB) by checking to see if the length of the image dimension array equaled 2.

Method 1: Looping over neighboring pixels and the filter


%Method 1
matrix_sum = 0;                   
for l=1:1:filter_size1
   for m=1:1:filter_size2
       multiply = padded_image(i+l-1,j+m-1,k) * filter(l,m);
       matrix_sum = matrix_sum + multiply;
   end   
end
output(i,j,k) = matrix_sum;

Method 2: Image Subsections and Im2Col()


%Method 2
sub_image = padded_image(i:i+filter_size(1)-1,j:j+filter_size(2)-1,k);
image_collumn = im2col(sub_image,[filter_size(1),filter_size(2)],'distinct');
final_image = image_collumn .* filter_collumn;
conv_sum = sum(final_image) ;               
average = (conv_sum );
output(i,j,k) = average;

Method 3: Simpifications


%Method 3
sub_image = padded_image(i:i+filter_size1-1,j:j+filter_size2-1,k);
final_image = sub_image .* filter;
conv_sum = sum(final_image(:) ); 
output(i,j,k) = conv_sum;

Selecting the Right Image and Cutoff Frequency

Having properly functioning filtering functions is only half the task. A hybrid image only meets the standard of appearing to switch between the two images at different distances if a proper cutoff frequency is used when applying the gaussian filters. Furthermore, more often than not there is an image that would be a good high frequency image and one that would be a good low frequency image and making this distintion is important. Here are some of the results below.

In the table above the the image marilyn.jpg was run through my_imfilter(image, filter) as both the high and low frequency images. The top row is her in the high frequencies and the bottom is the low frequency images. On both rows the cutoffs for the Gaussian filters started at 1 and progressed by 2 each time (1, 3, 5 ,7 ,9 ,11) to see how cutoff frequency effects a fitlered image. Its also important to note that at low frequencies we a lot of the details of an image making it important to pick the proper image that is easier to see when "blurred".

Hybrid Image Results

Low Cutoff Frequency: 7 High Cutoff Frequency: 3
Here I choose the cat as the high frequency image because of the whiskers and the dog as the low.

Low Cutoff Frequency: 5 High Cutoff Frequency: 2
Here I choose the high frequency image as the plane because of its sharp geomtric shape and because the bird is easy to see when blurred and is less recognizable up close.

Low Cutoff Frequency: 4 High Cutoff Frequency: 2

Low Cutoff Frequency: 4 High Cutoff Frequency: 2
For the Marilyn Monroe and Albert Einstein I thought that both served reasonably well as both high and low frequency images. In the case where einstein is the low frequency it is very easy to see Marilyns sharp features but still see Einstein clearly in the smaller image. On the other hand, with Einstein as the high frequency you get to see the details in his hair, but his features do tend to carry over at a distance making Marilyn lips, nose and eyes a little less clear.

Low Cutoff Frequency: 5 High Cutoff Frequency: 2
Here the submarine was the flow freuqncy image and the fish was the high frequency image because the submarine did not have a lot of detail worth keeping or that was necessary to comprhend the image.

Low Cutoff Frequency: 8 High Cutoff Frequency: 3
Similiar to the Marilyn Monroe and Einstein image, I though eitheir the bike or the motorcycle could have been the high/ low frequency pass images because making eitheir the low frequency pass images results in a loss of detail. However, I choose the motorcycle because it had all the detail around the motor whereas the bicycle was more recognizable when blurred because of its unique shape.