CS129 / Project 2 / Image Blending
The Walter Lisa
In this project, we do seamless image blending via Poisson blending. It is covered in detail by Perez, et al., but here's a brief description. Doing image blending based on absolute pixel values is very hard in the general case, because many images have very different color palattes. In this algorithm, instead of relying on the absolute pixel values of a part of a source image, instead we get information about the gradient for each pixel in that part of the source image, and then in the target image, we apply the same gradient, so that the colors seamlessly blend into the target image along the edges of the selection. The key here is that since the gradient remains the same between source and target images, the edges and other contrast features in the modified area stay consistent, even though the colors might be slightly different. This makes the image very believable.
Algorithm Details
The specifics of our algorithm make use of linear algebra and sparse matrices. We have a mask which is the same size as both the source and target images. The mask's values are either 0 or 1. In our final image, if the mask's value is 0, we want to just take the target's pixel value at that point. If it is 1, we want to set up a linear equation such that the gradient (as measured by a discrete laplacian filter) for a given pixel is the same in both the source and final images. This means measuring neighbor pixels as well as the pixel under the mask. We go through the entire image and create a vector with one entry for each pixel which takes the actual values of either the target image, if the mask was 0 at that point, or the gradient value from the source if the mask was 1 at that point. We also set up a sparse matrix which has an entry for every pair of pixels. This matrix is a coefficient matrix, which allows us to theoretically multiply by the output image to get our target/source hybrid image. Once we have the coefficient matrix, A, and the target/source hybrid image, b, we can find our output image: output = A/b. Thanks to matlab and the speed of sparse matrices, we can do this for fairly large images in reasonable amounts of time.
Discussion
This algorithm works very well in certain situations. Results 1-6 are all pretty good, but my personal attempts (7-10) aren't as great. The main problem with the algorithm is that it relies on a somewhat uniform placement in the target image. For instance, the fact that the bear becomes more blue in #1 is not a problem, because the pool behind him is also very blue, so we figure it's normal. Contrast that to #8, in which the russian hat blends with the brown of the bear's fur and the green of the background so that it is barely black anymore. One other issue with the russian hat is that it is so black that ithas very little change in signal over the image, and so it is more vulnerable to edge bleeding kinds of effects. You can see a similar issue in #9, where the blue paint from William Wallace's face almost uniformly blends through Andy's. The algorithm is quite successful, though, if you are placing a high contrast source object onto a low contrast target, as illustrated especially by of 1, 2, and 3
Results
| # | Target | Mask | Source | Naive Result | Algorithm Result |
| 1 |
|
|
|
|
|
| 2 |
|
|
|
|
|
| 3 |
|
|
|
|
|
| 4 |
|
|
|
|
|
| 5 |
|
|
|
|
|
| 6 |
|
|
|
|
|
| 7 |
|
|
|
|
|
| 8 |
|
|
|
|
|
| 9 |
|
|
|
|
|
| 10 |
|
|
|
|
|
As we can see, the test images (1-6) seem to work better, on average, than my images, although I am proud of my "Shia LaCage". With a bit more fiddling and some modification of the source image backgrounds to match the target image backgrounds, my images (7-10) might work better.