CS123 Project: Image Blending

Author: Yan Li

Login: yanli

Email: yan_li@brown.edu

 

Goal of Project

The goal of this project is to implement Poisson Blend described in P¨¦rez, et al.

The source image with a specific mask and the target image can be blended seamlessly by solving the Poisson Equation.

 

My Works

1 Optimizing the algorithm for solving Poisson Equation.

2 Implementing the Poisson Blend.

3 Implementing Blend with mix gradient. (Extra Credit)

4 Implementing Blend with Laplacian Pyramid. (Extra Credit)

5 Implementing Texture Flattening. (Extra Credit)

 

IMPLEMENTATION

1 Algorithm

Instead of calculating the gradient for each pixel by convolution, I use ¡°diff¡± operator in Matlab and compute the X,-X,Y,-Y gradient, then mix them together and get the result which is same with convolution.

Instead of building sparse matrix by carefully index the matrix and retrieve each element that is not zero, I pre-compute the indices and the values.

For the default six test cases, the current algorithm for Poisson Blend will cost less than 10 seconds.

 

2 Poisson Blend

   Here I show my results in the following form. The columns includes the source image, the target image, the mask, the result of Naïve Blend and the result of Poisson Blend,.

 

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_naive01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_naive02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_naive03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_naive04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_05.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_05.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_05.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_naive05.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_05.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_06.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_06.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_06.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_naive06.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_06.jpg

 

3 Poisson Blend with mix gradient

     Instead of always using the gradient of the source image, I use the larger gradient from either the source image or the target image. This could generate some transparent effect between the source image and the target image. The following form illustrates the difference between Blend with mix gradient and without it. As you can see, some blend results are enhanced, like the sixth image, the rainbow has less halo effect. But some blend results are not realistic such as the third one, the car looks really transparent.

 

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Documents:ComputerPhotography:proj2:stencil:code:results:result_01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Documents:ComputerPhotography:proj2:stencil:code:results:result_02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Documents:ComputerPhotography:proj2:stencil:code:results:result_03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Documents:ComputerPhotography:proj2:stencil:code:results:result_04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_05.jpg

˵Ã÷: Macintosh HD:Users:yanli:Documents:ComputerPhotography:proj2:stencil:code:results:result_05.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_06.jpg

˵Ã÷: Macintosh HD:Users:yanli:Documents:ComputerPhotography:proj2:stencil:code:results:result_06.jpg

 

4 Blend with Laplacian Pyramid

I also implemented the Laplacian Pyramid Blend. This kind of blend can preserve the original intensity in source image perfectly, but the results are not seamless. Obvious seam in the result image can be seen. Here is the difference:

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_laplacian03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_laplacian01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_laplacian04.jpg

 

5 Texture Flattening

Possion equation cannot only be used for blending. Indeed, we can use those more important gradient information from the source image to wash out some relatively useless details. Texture Flattening is such a technique that can use generate a non-photorealistic effect.

I extracted the useful gradient information by using edge detector (In my project, I use Sobel Operator to extract edge ). Those guidance gradient on the edge should be taken into account, while others we just set the guidance gradient to zero.

Here is the result of Texture Flattening:

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_texture01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:source_mask04.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_texture01.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_texture02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:source_mask02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_texture02.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_texture03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:source_mask03.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_texture03.jpg

 

6 Other Images

   Here are some other test cases of Poisson Blend:

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_07.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_07.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_07.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_07.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_08.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_08.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_08.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_08.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_09.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_09.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_09.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_09.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:source_10.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:target_10.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:data:mask_10.jpg

˵Ã÷: Macintosh HD:Users:yanli:Library:Containers:com.apple.mail:Data:Library:Mail Downloads:proj2 2:stencil:code:results:result_poisson_10.jpg