Kai Herng Loh (kloh)

A very toasty face.

Texture Synthesis

The first part of the project, Texture Synthesis, is a method to generate a larger image of repeating patterns based on a smaller image of a texture. For example, we have a few apples but would like to generate an image of many apples (one of the examples illustrated below).

To achieve this, I sample random patches from the input texture, with a margin of overlap, calculating the smallest squared difference (SSD) each time. This works by iterating through all the pixels in the overlap region, and calculating the absolute difference between each color channel in each pixel on each patch the algorithm is looking at, and the corresponding pixel on the image in the overlap region that is already filled by an earlier patch. I take the patch with the smallest SSD as the patch that will fit in most seamlessly with the rest of the image that we have already synthesized.

However, just choosing our patches based on SSD alone will result in observable "boundaries" between patches that reduce their aesthetic quality. To make the transition even more natural, I used the same algorithm for "seam carving" to carve out a natural-looking boundary between the patch with the lowest SSD and the rest of the image on the region of overlap.

Texture Transfer

In the second part of this project, I explored transferring a texture onto an image, applying it as the "base" of the second image. An example is the face on toast picture shown at the top of this page.

The patch we choose at each offset must correspond not only to the overlap regions of existing patches already on the image, but also to the intensity of the target image. In addition, my algorithm is run multiple times, decreasing the tile size of the patch each time to get better results. Therefore, we modify the error term from Texture Synthesis to be the following:

error = (alpha) * (overlap_error + previous_synthezised_error) + (1 - alpha) * correspondence_error

Texture Synthesis Results

Here are the results of my algorithm in table form for Texture Synthesis. The leftmost image corresponds to the naive-transfer case, where we simply take random patches and paste them next to each other. The middle image in each row, which looks much more natural, shows the result after choosing the patch that has the lowest SSD at the margin of overlap with existing patches. Finally, the rightmost image shows the result after applying minimum error boundary cut ("seam carving").

Discussion:The algorithm generally gives good results. However, in some cases, we get repeating sequences of patches. In future, perhaps a tolerance "threshold" can be set, and a random patch that fulfills the threshold selected, because selecting the patch with the lowest SSD may result in the same patch being selected each time.

Texture Transfer Results

Here are the results of my algorithm in table form for Texture Transfer. The image on the left is the texture that we want to represent the image in the middle, and the image on the right is the result.

Discussion:The patches look somewhat disjoint - perhaps "seam carving" to find the minimum error boundary cut could also be implemented so the patches fit better with one another. In addition, some results of texture transfer don't look all that nice - perhaps this might be a choice of textures. I noticed that textures with a greater range of color intensities seem to work better in texture transfer, maybe because it is easier to map the color tone of the picture to a pixel in the source texture.

By Kai Herng Loh (Computer Science-Economics, Brown '14)