Yipin Zhou (zyp)

CS129 / Project 2 / Image Blending

Poisson blending

The primary goal of this session is to blend a part of source image in to a target image seamlessly. To achieve this, we try to let output pixels under the hand made mask preserve the gradients of the source region maximally without considering their absolute intensities.

Algorithm

For the output pixels which outside the mask, we make them directly equal to the according target image pixels. For pixels under the mask, we try to make their gradients(4*v(i,j) - v(i-1, j) - v(i+1, j) - v(i, j-1) - v(i, j+1)) closest to the according source gradients(4*s(i,j) - s(i-1, j) - s(i+1, j) - s(i, j-1) - s(i, j+1)). During the implementation, I made coefficient matrix A a sparse matrix at very beginning and indexed into it in a for loop. That would take about 2 minutes to finish 6 given images. To be efficient, I got A matrix values in advanced and constructed A a sparse matrix in one operation, which takes 5 seconds to blend all images.

%Pseudocode
if output(x,y) outside mask
   output(x,y) = target(x,y)
else (n is the number of neighbors)
   n*output(x,y)- output_neighbors(x,y) = n*source(x,y)- source_neighbors(x,y)
end

Result images

Naive version	Poisson blending

Extra Credit

Mixing Gradients

To implement mixing gradients, in Poisson equations I used either source gradients or target gradients which has a larger absolute value instead of only accessing source gradients. The mixing gradient way works pretty well in the 'rainbow' image by adding transparency to the rainbow, which seems more reasonable. However, this approach does not work well when the objects should be opaque.

%Pseudocode
(for pixel p, q is one of its neighbors, N is the set of the neighbors)
for(each q belongs to N)
    if(|target_p - target_q| > |source_p - source_q|)
       Vpq = target_p - target_q
    else
       Vpq = source_p - source_q
    end
    Mix_gradient_p += Vpq;
end

n*output_p - output_neighbors_p = Mix_gradient_p;

Result images

Naive version	Poisson	Poisson with mixing gradients

		more realistic

		ghost!!

Laplacian pyramid blending

In Laplacian pyramid blending, we need to build Laplacian pyramid for source and target images by building Gaussian pyramid first and using former levels minus latter levels (eg.Laplacian_level1 = Gaussian_level1-Gaussian_level2 and Laplacian_lastlevel = Gaussian_lastlevel).

Here is the method

1. Build laplacian pyramid for source and target image
2. Build Gaussian pyramid for the mask
3. Blend every level: output_level_n = mask_level_n * source_level_n + (1-mask_level_n) * target_level_n
4. Collapse every output level by adding them all together to get the final blending output image

During implementation, some output images may have sharp edges where should have been blurred when the radius of Gaussian filter is too large. Because in function fiximages(), one part of source images were cut directly and pasted on black backgrounds to build new source images which is not continuous.

Level 1	Level 2	Level 3	Level 4

	source laplacian

	target laplacian

	mask gaussian__

Result images

Naive version	Poisson	Laplacian

		result with sharp edges

My images

Cut and paste	Poisson blending

Cut and paste	Poisson blending

Cut and paste	Poisson blending with gradient mixing

Thank my friend Rui for being willing to be a potato.