CS 143 / Project 1 / Image Filtering and Hybrid Images

The Algorithm

I chose to use the im2col and col2im functions to transform the blocks within the image into columns. This allowed me to avoid manually looping through the pixel matrices when doing multiplication with the filter, and simplified my code.

My general approach was as follows:

  1. Reshape the filter into one column.
  2. Pad the image. This required that I determine how many zeros must pad the image on each side. For each dimension, this is the size of the filter in that direction, minus the center pixel, divided by two (as half of the filter must extend off of the original image if we consider a pixel on the edge).
  3. For each channel, use the im2col function to transform each block of the filter's size in the image into one column of a matrix with n x m rows, where n and m are the dimensions of the original image.
  4. Multiply each column with the filter.
  5. Sum each column to one value (this is now the intensity of the corresponding pixel) to produce a row vector of length n x m.
  6. Use the col2im function to map each element of the row vector back to its location in the image.
The code: 

function filtered_image = my_imfilter(image, filter)
	image_size = size(image);
	filtered_image = zeros(image_size);                                         % allocate new image

	filter_reshape = reshape(filter, size(filter, 1) * size(filter, 2), 1);     % reshape filter into one column for element-wise multiplication

	x_pad = (size(filter, 1) - 1) / 2;                                          % determine proper x and y padding
	y_pad = (size(filter, 2) - 1) / 2;

	for channel = 1 : size(image, 3)
	    padded_channel_image = padarray(image(:, :, channel), [x_pad, y_pad]);  % pad image with zeros
	    
	    cols = im2col(padded_channel_image, size(filter));                      % convert image blocks the size of the filter to columns
	    filtered_cols = cols .* repmat(filter_reshape, 1, size(cols, 2));       % multiply each column by the filter elementwise

	    sum_vector = sum(filtered_cols, 1);                                     % sum each column of the matrix together
	    filtered_image(:, :, channel) = col2im(sum_vector, ...                  % convert values back to image blocks 
	        [1 1], image_size(:, 1:2));                                         % (now mapping each value to a single pixel)
	end
end


Filtering Results

This table shows the results of applying my filtering function to the cat image with various types of filters.
identity-image blur-image (gaussian) large-blur-image (large gaussian)
sobel-image laplacian-image high-pass-image


Hybrid Image Results

Combination #1: Cat/Dog

cutoff_frequency = 7
Original Image #1 Original Image #2
High Frequencies Low Frequencies Hybrid Image
Hybrid Image Scales


Combination #2: Bike/Motorbike

cutoff_frequency = 6
Original Image #1 Original Image #2
High Frequencies Low Frequencies Hybrid Image
Hybrid Image Scales


Combination #3: Bird/Plane

cutoff_frequency = 8
Original Image #1 Original Image #2
High Frequencies Low Frequencies Hybrid Image
Hybrid Image Scales


Conclusions

Although my algorithm was able to filter images relatively quickly (depending on the filter size, of course), it was not nearly as fast as matlab's built in imfilter.

Choosing which image would provide the low or high frequencies (and turning the cutoff frequency) was probably the hardest part of this project -- it was difficult to get any pair of images to match up as well as the dog/cat combination.