The goal of this to implement seam carving and demonstrate the results of the implementation. Seam carving is a technique to 'retarget' images, or, 'smart resize' images. The technique preserves content in order of its importance. The importance of content in an image is calculated according to some energy function. For this project, I implemented the standard gradient-based energy function. I also implemented a forward- gradient based energy function, as described in Rubinstein 07.
For this part of the project, I implemented seam carving using a backwards energy function. This is a simple method, whereby the energy for each pixel is calculated as being the gradients from its upper- and left- neighbours to itself. Then to carve out a low-energy seam is simply removing the lowest-energy connected line of pixels from the image.
The following images show, from left to right: the original image; the image with the selected seams for removal; the image with the seams removed.
One downside to seam carving using a backwards energy function is that, whilst it removes the lowest energy pixels, it does not account for the fact that removal of the pixels could add energy to the image by increasing the energy values of the neighbouring pixels. The next technique, seam carving using a forwards energy function, addresses this.
For this part of the project, I implemented seam carving using a forwards energy function. This is a slightly more complicated method, and the energy function calculation is outlined in section 5 of Rubinstein 07. Briefly, the cost of removing a pixel is calculated to be the sum of the gradients between the pixels that would subsequently become aligned after the seam was removed.
The following images show, from left to right: the original image; the image with the selected seams for removal; the image with the seams removed.
The effect of using this method is that large amounts of energy are not reinserted into images after seam removal. Visually, the effect of the image is that lines and shapes are more preserved than in the backwards energy case.
The following images illustrate the seams that are selected for removal using forwards- and backwards- energy formulation. For most of the images, the forward- energy approach produces much better results than the backwards-energy results. In a couple of instances, the backwards- and forward- energy approaches produce similar results. The final picture produces quite bad results using both approaches. This is for two reasons - first, the image is very busy, and so the energy of the image is relatively similar for every pixel. Second, the regions of interest are the 2 people in the photo. However, the colours on the clothing of the handsome person is very dark with little gradient change - thus making those pixels good candidates for removal. Therefore the approach produced a strange stick-figure like result.
| Original | Controls | Backwards Energy | Forward Energy |