Project 4: Image Quilting for Texture Synthesis and Transfer
Vibhu Ramani
October 24th, 2012
Texture Synthesis
Steps
- Given a texture patch we select a tile of a fixed size.
- We scan through the whole texture trying to find another tile of the same size with the least SSD on a small overlap region with the previous selected tile.
- For a Min cut: in the overlap region we try to find a path with least energy and use that line to transtion to the new tile. This makes the transition at the edges smooter.
- To calculate the min energy line we calculate the diff of the images at the overlap region and then apply dynamic programming to find the least enegy path.
Extra Credit
Calculating SSD via image filter operations: To calculate the SSD of a patch over an source image region we look at what the SSD expands to:
SSD = for every pixel i Sum (Si-Pi)^2
= sum (Si^2 - 2*Si*Pi + Pi^2)
sum(Si^2) : Square every Pixel value on the source image. Sum of the pixels on region the patch is applied to. Can be done by doing an image filter operation with a filter of values(1);
sum(Pi^2): sum of the square of the pixel values in the Patch.
sum(2*SiP1): twice the value of an image filter operation of the patch on the source image.
Computing it this way gives us the SSD at every pixel on the source image and thus we can pick up the best one. But to avoid similar patch sets we choose at random from the top 10.
Poisson Blending: in the overlap region after finding the min cut we can do a poisson blending on this transition line so that the intensities do not change drastically.
Results
| Texture Patch | SSD | SSD+min_cut | SSD+min_cut+Poisson Blending |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Results for the other test images are available here
Texture Transfer
Steps
- Similar to Texture systhesis, but we add a contraint to the patch that we are trying to add, to be similar to the original image thus giving context of the original image.
- To get a comparison with the original image we use luminance diff at that point.
- We use a parameter "alpha" to determine the weight of the texture match vs original image.
- We perform several interations reducing the tile size to initially get coarse image similarity matching patches and then small patches to make the textures join up seamlessly.
| Texture | Image | Texture Transferred Image |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fun:
Generated from the Texture synthesis pipeline. Move 5px up each iteration.