Image-Based Shaving (Nguyen, M. et al)

Vazheh Moussavi (vmoussav)


Introduction

Do you wonder how you would look like without facial hair but afraid of commitment and don't want to let go of that sweet mustache? Let's say you're a detective chasing down one of America's Most Wanted and in the only photo you have of him, you can barely see half his face because its covered by his huge beard. Image-Based Shaving tries to solves those problems by "shaving" the photo for you.

Procedure

My implementation of Image-Based Shaving began by obtaining a set of 1140 images of "canonical faces" (ie. fit onto a 94x92 rectangle such as the middle two images above). These images were NOT labeled as with/without beards, due to ownership rights. After labeling all 1140 images by hand, I was ready to proceed with an implementation of Image-Based Shaving.
The basic idea of the paper is to impose the influence of the non-bearded images (the "non-beard subspace") in the training set on the beard pixels of the given test image. This reconstruction is done through Robust Regression, which differs from a basic, or "naive" regression in that it tries to reduce the influence of outliers of a dataset, specifically the beard pixels of the test image. Below is a description of the matematics of naive and robust regression.
In the paper, there are also two additional steps. First, the principal components (used in EigenFaces) of the "beard subspace" are used as a negative reinforcement to the beard pixels. The desired effect for this is for facial characteristics such as moles/scars to not be lost in this process. There is also a GraphCut post-processing step to help re-blend the face into the original image. However both of these steps, (especially the GraphCut), are somewhat superfluous and are not required to shave someone. Because of this and time constraints, I chose not to implement these add-ons.


alt : test.html

Results

I was able to obtain fairly decent results for Image-Based Shaving. This was the case
especially when the facial hair in question was not overly dense, or too extreme of a color.
Original Face Naive Reconstruction Robust Reconstruction


One important observation about my results is that they sometimes tend to fade the
beard, as opposed to actually shaving the individual. The next two cases display this clearly.

Failures

In addition to/as a result of the "fading" phenomenon observed in some of these images, the paper describes
a "beard threshold" after which their algorithm does not work. Clearly, my beard threshold is lower than
that of the original paper, and the following photos show where my algorithm was very much off.