On this page:
1 Introduction
2 Energy
3 Seams
4 Support Code
5 The Assignment
6 Built-Ins
7 Analysis
8 Template Files
9 Handing In

Fluid Images

    1 Introduction

    2 Energy

    3 Seams

    4 Support Code

    5 The Assignment

    6 Built-Ins

    7 Analysis

    8 Template Files

    9 Handing In

1 Introduction

Suppose that you are given an image and wish to shrink its width by a given number of pixels without changing its height. The simplest strategy is simply to scale the image, but this may introduce undesirable distortions. Another option is to crop the edges of the image, but this is unacceptable if the edges contain important information.

In this assignment, you will implement (in two ways) a more advanced algorithm to intelligently decide what parts of the image to remove.

2 Energy

Given an image we first compute the “energy” of each pixel, which measures how much that pixel stands out from its surroundings. This gives us a rough idea of its importance. In other words, an “unimportant” pixel blends in with its surroundings, and can thus be removed with, hopefully, the least distortion to the image.

Each pixel is represented as a Color, which is defined by its red, blue, and green values. To compute the energy of a pixel E surrounded by pixels A through I, arranged as follows,

A B C

D E F

G H I

use the formula

\[energy(E) = \sqrt{xenergy^2 + yenergy^2}\]

where

\[xenergy = a + 2d + g - c - 2f - i\]

\[yenergy = a + 2b + c - g - 2h - i\]

and each lowercase letter represents the brightness (sum of the red, blue, and green values) of the corresponding pixel.You are welcome to experiment with different energy functions, but the solution you hand in must use the formula above.

To compute the energy of edge pixels, you should pretend that the image is surrounded by a 1 pixel wide border of black pixels (with 0 brightness).

3 Seams

A vertical seam is a set of pixels, one from each row of the image, that are “connected” vertically. That is, any two pixels on adjacent rows are either vertically or diagonally adjacent. Note that this means seams are not always vertical lines. The energy of a seam is the sum of the energies of the pixels in the seam.

For example, in the diagram above, where each square represents one pixel, the blue (1), red (2), orange (3), and yellow (4) sets of pixels are each a seam. Note that the pixel where the blue (1) and red (2) seams overlap is in each of these seams.

We will shrink the width of the image by removing vertical seams from the picture. Our algorithm consists of repeatedly finding the lowest-energy vertical seam and removing it from the image. Sometimes multiple seams may tie for the lowest energy. If this happens, remove the leftmost of those seams. The leftmost seam refers to the seam with the leftmost pixel in the top row. For instance, in the diagram above, the blue seam (1), is the leftmost seam. If the seams overlap, the leftmost seam will be the one with the leftmost pixel in the topmost row where the seams diverge. Note that once a seam has been removed, the image has changed, so the energies of the remaining pixels must be recomputed.Think about why we remove lowest-energy seams rather than simply removing the lowest-energy pixel from each row. Better still, try it out, and you’ll see the difference for yourself! But this isn’t part of the assignment. (In principle some of these energies can just be updated, but at an introductory level, simply recomputing is sufficient.)

4 Support Code

You will be able to access the following types and functions:

5 The Assignment

For this assignment, implement the following two functions:

liquify-memoization :: Image, Number -> Image

carves the number of seams given by the second parameter.

Your solution must use memoization and should be reasonably efficient: it should not take exponential time! If you cannot liquify 10 seams in a few minutes from the test images we provide, you should rethink your implementation. You do not need to be this fast to receive full credit, but it is a good benchmark.

You need to define an explicit memoize function, of the right number of parameters, and use it, to separate the core idea from the notion of memoization.

liquify-dynamic-programming :: Image, Number -> Image

This must use dynamic programming instead. Again, it must be sub-exponential.

For both of these functions, you can assume that the images will never be empty. The number of seams to carve will always be strictly less than the width of the image. This means that you never have to (and never should) deal with the “empty” image case; the image will always have pixels, since we will always leave at least 1 column of pixels remaining.

6 Built-Ins

You should be able to do this assignment with no use of mutation beyond that needed for memoization or dynamic programming. Arrays may be used only for those purposes.

7 Analysis

Give a big-O analysis of each solution. First find an upper bound on the total number of possible vertical seams for an image in terms of its width and height. Use this bound to find the time complexity of the naive solution, which computes the energy for every possible seam. Then determine the complexity of your two solution functions.

In your analysis, you may assume that generating the pixels from images and vice versa, for an image with width \(w\) and height \(h\), runs in \(O([w, h \rightarrow wh])\).

8 Template Files

Implementation

Examplar

9 Handing In

Solution