CS 195-G: Image Blending
The goal of this project was to implement the seamless image blending algorithm of Pérez, et al., which determines the final image using a discrete Poisson equation.
Algorithm
Basic Algorithm
 The source image |
 The target image |
This algorithm copies regions from a source image to a target image. In this example it will use the face from the source image to replace the face of the destination image. The face in the source image has been aligned manually to fit the target. To select the regions from the source image that will be used, a third mask image is needed. It is white in the region of interest and black elsewhere.
The mask image
The pixels in the final image are computed by solving a Poisson equation. In code, each final pixel is defined as:
if mask(x,y) is 0:
final(x,y) = target(x,y)
else:
final(x,y) = 0
for each neighbor (nx,ny) in (x-1,y), (x+1,y), (x,y-1), (x,y+1):
final(x,y) += source(x,y) - source(nx,ny)
if mask(nx,ny) is 0:
final(x,y) += target(nx,ny)
else:
final(x,y) += final(nx,ny)
However, all masked pixels (where the mask is nonzero) depend on each other for their final value. We have to compute everything simultaneously, which can be done by solving the matrix equation Ax = b. The size of both vectors x and b is the number of pixels in the target image. The vector x contains all pixels in the final image (what we are solving for). The vector b is the guiding gradient plus the sum of all non-masked neighbor pixels in the target image. The non-masked neighbor pixels define the value of the pixels at the boundary of the mask, which are blended across the mask area. The guiding gradient defines the second derivative of the final pixels in the mask area, and helps to "shape" what is blended in the mask area. The matrix A is a sparse matrix with a 4 for each pixel and a -1 for each neighbor pixel under the mask, and computes the gradient of the masked pixels in the final image. To get the final image, simply solve the equation for x. If the guiding gradient is zero, we are just solving a Laplace equation and the values at the mask boundary are blended smoothly across:
The final image with no guiding gradient
With the gradient of the source image as the guiding gradient, the mask area takes on the appearance of the source image, but is smoothly blended at the boundary:
 The gradient of the source image |
 The final image |
Mixing Gradients
It is sometimes better, especially for transparent objects, to retain some of the gradient of the target image in the mask area. The two gradients are mixed by picking the gradient to each neighbor either from the source image or the target image, whichever gradient has a higher absolute value.
 The gradient of the source image |
 The gradient of the target image |
 Both gradients mixed |
 The final image using gradient mixing |
Results
Project Images
Regular Poisson blending didn't work that well for the jet image because the area around the jet ended up blurry. This was almost completely fixed using gradient mixing, although the background can be seen slightly through the right of the jet.
 The source image |
 The target image |
 Poisson blending |
 Poisson blending with gradient mixing |
The pool image worked better with regular Poisson blending. Gradient mixing let some of the water behind the bear show through.
 The source image |
 The target image |
 Poisson blending |
 Poisson blending with gradient mixing |
The rainbow image worked better with gradient mixing because rainbows are transparent. Regular Poisson blending blurred out the area behind the rainbow.
 The source image |
 The target image |
 Poisson blending |
 Poisson blending with gradient mixing |
The whiteboard image also worked better with gradient mixing because the texture of the bricks needs to be preserved under the marker. Regular Poisson blending took the texture from the flat whiteboard.
 The source image |
 The target image |
 Poisson blending |
Poisson blending with gradient mixing |
The Mona Lisa image worked better with regular Poisson blending because Mona Lisa's actual face doesn't show through.
 The source image |
 The target image |
 Poisson blending |
 Poisson blending with gradient mixing |
My Images
In this image the Rubik's cube completely replaces the plate. The differences in color temperature are mostly compensated for, but the shadow on the bottom is slightly blue.
 The source image |
 The target image |
 Poisson blending |
|
In this image the pepper shaker is placed on the snow, although it is slightly transparent because gradient mixing was used.
 The source image |
 The target image |
 Poisson blending with gradient mixing |
|
In this image the washer door is grafted onto the wall. The boundary is perfectly seamless.
 The source image |
 The target image |
 Poisson blending |
|
In this image the apple laptop logo is applied to the side of the truck, with no visible boundary.
 The source image |
 The target image |
 Poisson blending |
|
In this image lettering from the Watson Institute is put on a Sunlab wrist rest. The brushed metal behind the lettering is still slightly visible.
 The source image |
 The target image |
 Poisson blending with gradient mixing |
|