Project 4: Scene Completion
Ben Cohen (btcohen)
March 13, 2010
For this assignment, I modified my image pyramid code from Project 1.
To align the images, I computed the SSD between the source image and the target
image at various offsets, ignoring the area under the mask.
To speed up the alignment, I limited the function to four recursive calls.
Initially I was having problems with the images being shifted too much, causing
seams to be visible in the resulting images. To fix this, I bounded the amount
that an image could be shifted so that no seam would appear underneath the mask.
Although this greatly improved the results, there are still seams visible in some
images since the graphcut expanded the masks past their original boundaries.
When the images are aligned, they are passed to the graphcut solver. Since there
is only a mask for the source image, I had to generate a mask for the target as
well. To do this, I expanded the source mask using imdilate and then inverted it;
this ensured that there was no overlap between the two masks, while forcing some
of each image to be present in the composite.
To rank the images, I used both the flow computed by graphcut and the ssd computed
by the image alignment. Since I wanted to pick images that have high flow, but a
low ssd, I ranked images by flow/log(ssd).
When I initially ran my algorithm, I noticed that the images with the highest values
of flow/log(ssd) weren't very good; every more puzzling was that the images with a
very low score seemed to be some of the best. Although it seems counter-intuitive to me,
I reversed the ranking so that images with the lowest scores are ranked the highest; I've
found that with this ranking metric, the scenes ranked highest by the algorithm are
most often the same images I would have chosen.
Once all the images are ranked, the top 5 images have a Poisson blend applied. Since
blending is fairly slow, processing all the results would be very time consuming.
(click image for results)