CS129 / Project 3 / Seam Carving

Steps

  1. Compute the energy matrix using gradient magnitude
  2. Pass this monochromatic image off to another function (image_energy) to compute the x-coordinates of the ideal (lowest energy) seam.
  3. Using these x-coordinates, remove that seam from the image.

Computing the Energy Matrix

We apply a Sobel filter using fspecial and imblend per color channel and sum across the three color channels

Finding the Minimum Path

First we copy the input energy matrix into a variable called M. We also initialize another array of all zeroes of the same dimensions as M for holding back pointers.

Now, looping through the rows starting at the second-to-top, we calculate the minimum value in M of the current pixel's neighbors at the above-left, above, and above-right locations. At the same time we get its relative x-index to the current pixel (-1, 0, or 1) and adjust it appropriately. We have to edge case for when the current pixel is on the left edge (i.e. it only has a pixel above and to the above-right of it) and the right edge(above-left and above). The relative index of the minimum value is being stored in the back pointers (bP) array and the minimum value at the same location in M.

Now that we have computed the back pointers array, we can generate the path of x-coordinates. We initialize a vector of zeroes with a length with as many rows as there are in the image. We set the last item in the vector to be the coordinate of the argmin of the bottom row in M. Now we can calculate the backpointers as follows:


%example code
for y = num_rows-1:-1:1 % (y = num_rows - 1; y >= 1; y--)
	path_x_coords(y) = path_x_coords(y+1) + bP(y+1, path_x_coords(y+1));
end

Removing the Seam

We create a new_img matrix initialized as zeros and give it identical dimensions to the input image with the exception that the column-dim is one fewer for the removal of the seam. Then, using the path_x_coords we can copy the pixels per row into new_img up to path_x_coords(y) and after it.

Results

The results are fairly good for most cases. The last three images are from outside sources, and the last (a painting by waterhouse) was resized to have dimensions 100 less on the width and the height. The picture of the group of people is an example of where the algorithm fails, although this failure isn't quite dramatic. The edges of the faces are surprisingly well preserved, but the limbs are torn apart. The algorithm fails when items of importance are smooth.