Overview

Given a texture, how does one extrapolate that into a full size image? This project tackled that problem and a bit more: using image quilting to overlay a texture against another picture.

A SIGGRAPH 2001 paper by Alexei A. Efros and William T. Freeman outlines the method we used in this project. Our project involved three steps (outlined below): a random image generated by quilting patches at random, an SSD image that tried to quilt similar patches, and an SSD with a minimum error cut for more seamless edges.

Texture transfer required a bit more analyzing. We had to make sure that the image quilt remained seamless, but also adhered well to the underlying image. This required a trade-off between quilting seamlessly and corresponding to the image we wanted to re-render in the new texture. The process through which this was achieved is outlined below.

Algorithm

To generate an SSD image without a minimum error cut, 1000 random patches were compared to the current generated image. Only the portion of the image and the patch that overlap were compared via SSD.

To add a minimum error cut to the mix, we had to compare the image and the patch in the overlapping region, and then find the minimum seam through that region. Once that seam was found, only pixels to the right or below that seam on the patch were added to the quilt.

In order to apply this texture quilt to a new image and re-render the image to have the new texture, we had to make sure that the quilt somehow corresponded to the underlying image. This naturally results in a trade-off between seamlessness and image correspondence. To balance this, an α value was used.

Furthermore, we used multiple iterations with decreasing patch size. In this case, the α value has to be adjusted. Agreement with the overlap region and the already-synthesized image must be taken into account. The error term then becomes

error = (α)*(overlap_error+previous_synthezised_error) + (1-α) * correspondence_error

.

Results