Introduction

The objective of this "Poisson Blending" algorithm is to compose a source image and a target image in the gradient domain. The reason "Poisson Blending" achieves a more realistic looking composition than naively pasting two simarily colored images together is because the human visual system is more sensitive to contrast than intensity values. The technique described here is an implementation of the algorithm presented in (Pérez, et al.)


Algorithm Overview

Prior to running the algorithm, a mask needs to be manually generated that indicates the overlapping region between the source (image being inserted) and the target (image the source is being inserted into).

  1. Preprocess Source, Target, and Mask images
  2. The mask and source images are first padded so that they are the same size as the target image. Next, they are translated using parameterized row/column offsets. If the valid pixels in the mask are on the edge of the source image, a 1 pixel symmetric buffer is added to all 3 images to make processing easier. This 1 pixel buffer is then cropped off later from the result.

  3. Generate a Sparse Matrix
  4. A linear system of equations is required to compute the resulting image from the source and target gradients. This system is represented by Ax=b, where A is the sparse coefficients matrix, x is the output image, and b is the desired gradient matrix. The size of sparse matrix A is NxN, where N is target image rows multipled by target image columns.

    For pixels outside the masked region, the output image pixel is simply the same as the target image. For these pixels, the row in the sparse matrix A is simply the same as the identity matrix. Also for these pixels, the corresponding value in the desired pixel gradient matrix b are also the same pixel value as the target image.

    For pixels inside the masked region, the output image pixel x at (row, col) depends on its neighbors according to the following equation:

    4*x(row, col) - x(row+1, col) - x(row-1, col) - x(row, col+1) - x(row, col-1) = desired pixel gradient

    For these pixels inside the masked region, the row in the sparse matrix then contains these coefficients at the corresponding indices. The value of the desired pixel gradient is in matrix b and is described further in the Mixing Gradients section.

    The creation of this sparse matrix was also optimized by eliminating most of the for loops, avoiding calls to MATLAB's sub2ind and ind2sub functions, and only calling MATLAB's sparse function once.

  5. Blend Each Color Channel Separately
  6. Finally, each image's red, green, and blue color channels are blended separately. While the same sparse coefficients matrix A may be used for each color channel, the gradient matrix b is different for each color channel.

    The final output is calculated using the matrix operation x = A\b in MATLAB.


Mixing Gradients

Mixing gradients is performed using a linear combination of the source gradient and target gradient for pixels inside the masked region. A parameterized coefficient alpha is used to control how much the source and target influence the gradient according to the following equation:

gradient = (alpha)*source_gradient + (1 - alpha)*target_gradient

Therefore, an alpha value of 0.5 would be the same as averaging the two gradients together. A comparison of two blended images processed with and without this technique is shown below.

No Mixing(left) vs Mixing Gradients(right)


Results from Original Images

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)


Failure Case

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)

This is clearly a failure case, because the green grass is showing through the lava instead of the black rocks. Also, the coloring in the transition between the rocks and the grass looks wrong. This is because the algorithm depends on the pixels on the border of the masked region to calculate the entire masked region. For example, the transition between the black rocks and the gray tiled road looks a lot more reasonable.


Results from Default Images

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)

Inputs (Target / Mask / Source)

Outputs (Naive / Poisson Blending)