Final Project: Face Morphing

Vibhu Ramani
December 20th, 2012

Steps:

Face morphing involves 3 major steps.

Defining Correspondences, and triangulation

Given a source and target face images (imgA, imgB), pick corresponding points on both the images by hand. More points results in better transition. We choose 36 points.

For each corresponding point in imgA, you now know the spatial transition it needs to match the similar corresponding point in imgB.

For an intermediate warp you know the line on which each of the point would lie.
Pick the points for your intermediate state based on some factor [0-1] where 0 means source while 1 would mean target.
We need to get a triangulation of these points. This can be done manually or using a triangulation technique. Delaunay triangulation is a good method since it doesn't produce over skinny triangles.

Compute Warping Transormations

For each triangle in the source and intermediate image we need to calculate a transformation matrix such that T*x = y
where T is a 3x3 matrix say
|t1 t2 t3|
|t4 t5 t6|
|t7 t8 t9|
x is the input point (x1, x2, 1) and y is the corresponding point in the transformation. (y1, y2, 1)
Since our warping is in 2d space we can assume t7=t8=0 and t9=1. For a triangle we have 3 points and thus with the above constraint reduces our system of equations to 6 unknowns and 6 equations which is solvable.
Each triangle in the source would have a corresponding T matrix to transform to the corresponding triangle in the intermediate image.
Similarly we find transformations for Each triangle in the target to the corresponding triangle in the intermediate image.

{imgA -> intermediate <- imgB}

Inverse Warping using interpolation, and alpha matting

Ideally since we now have the transformations of every triangle, in the source and target image to the triangle in the intermediate image, we should be able to fill up the intermediate image.
Most of the times you will find that the points from the source, and target dont map to a particular pixel on the intermediate image, but somewhere in between. One way to fix this is to contribute that pixel value to the neighbouring pixels. This is known as splatting.
Another way would be to do inverse warping. ie take every point on the intermediate image and find its corresponding value in the source,target images. If the value falls between pixels we can compute that using interpolation.
Once you have the 2 values of the pts in intermediate image (1 from source 1 from target) based on a alpha value we can get the final value.
alpha*target + (1-alpha)source.

Homography transformations.

If we smoothly vary the alpha and warp ratio, we can get a series of images showing a smooth transition.

What if we change the alpha to 0, ie the source image changes homographies based on the target, but the target never becomes visible.
The same face just keeps taking different shapes based on the target images on the right.

There are some artifacts since the source image I started with doesn't have a proper right side view to begin with.

Further Extensions

Mean face: If find the mean value of the corresponding points across the images and transform every image to that and then take a mean value of the pixels we can get the mean face.

Creating Baby faces: http://www.morphthing.com/ Take an image of a average baby.

Transform the images of the parents to that homography.

and take a 0.5 alpha while combining the images, which results in an estimated look of their offspring.

Autocomputing the correspondence pts. The correspondence points on the face are quite unique and based on features we can find similar points in a new image. This way we dont have to enter the correspondence pts manually.
I only tried basic experimentation with this technique by taking patches of each point from all the images, and then computing an average of the patch.
Using this mean patch I did an SSD over a new image to see which point matched the best. The technique was rarely able to find the equivalent point.

Credits

This projects was based on http://graphics.cs.cmu.edu/courses/15-463/2012_fall/hw/proj4-morph/
Some starter code was also taken from http://www.cs.brown.edu/courses/csci1950-g/asgn/proj5/