Jonathan Koh (jvkoh)

CS129 / Project 3 / Seam Carving

The goal of this project was to write a program that would resize a photo by taking "seams" out. A vertical seam is a set of pixels (one from each row) that are adjacent to eachother, such that the pixels are touching at least diagonally. A horizontal seam is identical except that it takes one pixel from each column.

In order to figure out which seams to remove, we calculate an "importance" or "energy" for each pixel. Then we find the seam with the lowest overall energy and remove that, and repeat the process until the image is the desired size. The idea is that least energy seams are the least noticable when you remove them.

In order to calculate pixel energy I used a pixels "edge energy" as it's energy. Edge energy is calculated simply by looking at it's neighbors and seeing how similar they are. This is done by looking at both the x neighbors and the y neighbors.

The purpose of removing seams rather than simply scaling or cropping the image is that it forces you to lose less information. Specifically it allows you to make an image smaller without changing the objects with the most detail. In the example on the right (and the first in the results) you can see that ultimately the girl and the houses are moved closer together and the image is significantly thinner with little change to the picture as a whole.

Results Key

Resulting Images

Failures and Artifacts

Image 1 (Farm with Girl): One of the better images. Her legs are deemed not significant by the algorithm and my brain interprets her as having her legs folded more in the resulting image.

Image 2 (Boat): Also one of the better images. The only real problem with this seam carved image is the island in the background has a pretty obvious slice through it that indicates that the algorithm has been slicing one to many seams through the mountain. This would likely not be a problem if the image were not made as thin as it is.

Image 3 (Dali):The first artwork image, which has brings up the simple problem that art is often made to be seen in it's entirety. Obviously that is not true of all art, but generally the whole piece is the "focus" of the image. That being said, neither the fuzzy version of Abraham Lincoln or the image itself end up looking all that bad. Unfortunately, the skin tones on the nake woman are so uniform that they are deemed "unimportant" by the algorithm when really they are an important center to the image, making her look rather silly in the carved image.

Image 4 (Grassland): In my opinion, this is the best resulting image.

Image 5 (Castle): I think of this as being the second best resulting image, only second because I like the grassland image better.

Images 6 (Kandinsky): This image again suffered from the art problem. There really isn't any specific focus, and at the same time the whole image is the focus of the image. It would be very hard to draw seams out of here that don't mess with some of the lines. However, there are other slightly more advanced methods of seam carving that factor in if the seam is going to create an awkward line cut. Unfortunately, I did not get that far in this project.

Image 7 (Mona Lisa): As for the Mona Lisa, while it is part of the "focus" of the drawing, Mona Lisa's hair and clothing are the lowest "energy" pixels simply because they are the most uniform. As you can see pretty clearly, those are the first to go, leaving only her face and the scenic background. Also clearly suffering from the art problem in seam carving.