Final Project Writeup

Generating Novel Faces through Nonuniform, Nondeterministic Face Morphing

David Dufrense (ddufresn)
May 17, 2010

Goal

The goal of this project was to implement a nonuniform, nondeterministic face morpher that could approximate what the child of two adults might look like.

Algorithm

Face Morphing is done here by solving a system of linear equations to compute the displacement at each pixel. Correspondence points are selected beforehand, and the displacement values for these points are constrained. Correspondence points are selected in pairs across the face's axis of symmetry to avoid having an asymmetric cross-fade. Pixels that are not correspondence points are constrained to have the average displacement of neighboring pixels.




I found that the displacements generated by this method led to localized effects near defined correspondence points. This produced a "droopy" effect. This was particularly unsettling near the eyes. To remedy this, I filtered the displacement values with a Gaussian filter of size 20 and sigma 5. This way, whole regions around the defined correspondence points would move with them.

The first step of my algorithm is to generate an intermediate image from the two "parents." I did not want to simply produce a uniform cross-fade or morph between them, as children often have features that resemble those of one parent, but not another. The correspondence points of the intermediate image are a linear interpolation of those of the parent image. A value from 0 to 1 is generated that determines how closely a correspondence point will resemble one parent as opposed to another. A value of 0 means the point is identical to one parent, a value of 1 means it is identical to the other. The value is sampled from a normal distribution with mean 0.5 and standard deviation 0.2, with values that exceed the range [0,1] scaled to be with that range.

Similarly, I wanted the amount the two parent images are cross-faded to be nonuniform. For each pair of correspondence points, I uniformly choose a number from 0 to 1. If the number exceeds 0.5, the crossfade value is set to 1. Otherwise, it is set to 0. This emphasizes which features the child inherits from each parent.

To generate a baby image, the difference between the baby and mean correspondence points is calculated, and added to the image.

difference = ptsbaby-ptsmean;
newpts = round(pts+difference);

Generating correspondence points for "male" and "female" adult images is similar, although since the differences here are more subtle, I exaggerate them.

difference = ptsfem-ptsmean;
newpts = round(pts+1.2*difference);

The mean and baby images are warped to match the blended image. I originally added 0.3 times the difference between the baby and mean faces, as shown below.

+0.3*( - )=

However, after some research and testing, I found I liked the image produced by blending with the infant image better.

Once the image and points have been generated, I warp the images based on these correspondence points.

Results

In one trial, the method for generating the baby, masculine, and feminine images involved blending them towards the mean baby, male, and female images. In the other trial, I instead added the difference between the baby and mean faces, the difference between the mean male and mean female, and the difference between mean female and mean male.

Mean FaceMean MaleMean Female
Baby Minus MeanMale Minus FemaleFemale Minus Male
Blending with Means Image Subtraction
Basic Blend - Baby - Masculine - Feminine Basic Blend - Baby - Masculine - Feminine

Conclusion

There is not much visible difference between the different methods attempted. However, in principle I feel that blending a third image with the "parent" images dilutes the unique features of the parent faces. The methods for producing infant and female variations on a image are largely successful, as they cause noticeable changes. Infant images have a distinct facial shape, and have lighter hair and skin. Female images are more flushed, with elongated features and larger pupils. The masculine images are not noticeably different from the images they are generated from. Careful comparison shows that the face does become more squat, and less colorful. This minimal change may be due to the fact that I used 23 male and 6 female images to generate the mean, thus making the male and mean face more similar

The infant morph produced some images that could conceivably be real children.

And others that were obviously not.

This is in part due to features such as facial hair, or facial orientations that do not transform well into baby proportions. Correcting these is currently beyond the scope of this project.

Unfortunately, due to the nondeterministic nature of this algorithm, there is a chance that "child" images will be very similar to a parent, which is not an interesting result.

Bloopers