cs129 / Texture synthesis and transfer with Image Quilting

CS129 / Project 4 / Texture synthesis and transfer with Image Quilting

Example of a min-cut algorithm on texture synthesis.

Example of a texture transfer

1. Background

In this project I implemented Image Quilting for Texture Synthesis and Transfer, a SIGGRAPH 2001 paper by Alexei A. Efros and William T. Freeman. The paper presents a simple image-based method to do texture synthesis and texture transfer. Texture synthesis is the process of creating an image of arbitrary size from a small sample( like the image above). Texture transfer means re-rendering an image in the style of another one( Abraham Lincoln).

2. Implementing Algorithm:

The algorithm is intuitive, choose a patch from source texture and do SSD metrics to get the nearest match. Using seam carving to make the scene seamless. I listed some decisions on implementing the algorithm.

2.1 Step

When implementing the SSD match for the texture patch, I came with a question how to choose the step for searching the whole texture. If the step is too small, than the searching time will be long, and if the step is too large, I may not get the comfortable result that matches well. So after several trials, I decided to use 1/12 of the tile size.

2.2 Flag

Flag has been used in the implementation as a signal on whether it is the first time of iteration, because in the first time iteration, the source layer is not yet come, so do not need to compare with the source layer,which is the operation that must do in next iterations

3. Extra Credit:

Graph Cut:

In this assignment, I implement graph cut as extra credit. The algorithm is simple, what need me to do is generate the sparse matrix which shows the edge information, generally I think graph cut's result is as good as seam finding.

4. Discussion on the results:

From the result below we can see that the min_cut algorithm is good for the texture synthesis and texture transfer. But the random works badly. Pure SSD algorithm is not as good as min_cut, but is much better than random, in SSD I can still see a lot of seams.

On texture transfer, the execution time is really long, it takes minutes to get the transfer results.