CS129 Project #2: Gradient Domain Fusion Using Poisson Blending

By: Bryce Aebi

An image generated by blending a dinosaur head onto a hippo

Objective

The purpose of this project was to seamlessly blend an object or texture from a source image into a target image. Simply pasting the source image into the target image results in obvious seams. To circumvent this problem, the source image is altered using Poisson blending so that it can seamlessly integrate into the target image. Since people are more sensitive to gradients than absolute image intensities, the aim was to find values for the output source pixels that would preserve the gradients of the source region.

Method

To quantify gradients, the Laplacian is used on each pixel in the source image. In general, a pixel with small gradients between it and its neighbors will have a value near zero when filtered by a Laplacian. This is because the brightness values of the four neighbor pixels multiplied by -1 will cancel out with the brightness value of the target pixel multiplied by 4.

The Laplacian Filter
(-4 centered on the target pixel.
I actually used this matrix multiplied by -1)

To blend two images there must be a target image that will recieve inserted pixels, a source image from which the inserted pixels will come, and a mask that denotes which pixels from the source will be transferred onto the target. The output image will be the same size as the target image.

For the output image, any pixel index that is not "allowed" by the mask will be taken directly from the target image. The pixels allowed by the mask will be placed in the output image in the remaining space. These pixels are altered so that their gradients are the same as they were in their original image. The problem arises from the borders which may or (more likely) may not match the gradients that existed in the source image. To solve this, the program sets the gradient value of the source image (created with the Laplacian) equal to the gradient value of the output image pixel. Since the output brightness value of the pixel is as yet unknown, it can be solved for. To clarify, the gradient value of a given source pixel equals

4(source-pixel) - (north-pixel) - (south-pixel) - (west-pixel) - (south-pixel)

This can be set equal to the output pixel at the same index in the image

4(output-pixel) - (north-pixel) - (south-pixel) - (west-pixel) - (south-pixel)

All of the pixel values in source are known so this becomes a constant. However, in the output pixel expression, the value of each pixel is variable. There are many output pixels that need to be solved for, resulting in a system of equations that can be solved for to determine the values of all of the output pixels. (Remember the output pixels not indicated by the mask automatically assume the values of the correspondingly located target pixels.

Target Image

Source Image

Mask

Naive Pasting

Blended Image

Results

Generally, this algorithm produced relatively seamless image composites. For well chosen pairs of images, the seam is practically undetectable. For others, the edges may appear blurry if the textures of the target and source images differed greatly. The most sigificant issue with Poisson blending is the unnatural color changes that the source image can undergo. This occurs when the preservation of gradients along the borders forces the inner pixels to assume drastically different brightness levels. In conclusion, if the source and target images are carefully chosen the source image pixels are well-placed, Poisson blending can effectively merge two images together without leaving a large number of undesirable artifacts.