Applications of Face RecognitionAs a computer vision problem, face recognition is one of the most soughtafter unsolved problems in the industry. There are dozens of different implementations (a simple, albeit long, description of a lot of different approaches can be found here. Face recognition is interesting as a computer vision problem, cognitive neuroscience problem, and in various industries. Areas that would benefit from robust face recognition include video surveillance, humanmachine interaction, photo tagging, virtual reality, or law enforcement. Brown Cognitive Science classes teach that humans have special areas of the brain designated for face recognition, separate from other visual process. Others still think this is an open question.  
The first face recognition engineers used semimanual feature selection which computers could then use for recognition. This was in the early 1960's. Even today, the same problems with still plague even the best implementations: differences in posture, illumination, occlusion, age. You may be able to look at a picture of your grandmother on her wedding day and recognize her, but computers still can't. 
The approach I decided to use was using eigenfaces and prinicple component analysis as the feature space for faces, and knearest neighbors for the machine learning classification part. I started with a large database of face images (courtesy of Dr. Libor Spacek). The theory behind eigenfaces is that using linear algebra techniques, we can represent the different "important features" of faces with the eigenvectors. Eigenfaces can be built for individual faces, or for a set of faces  I chose to do the latter.

Given a set S of m images:
Create a matrix M where each row is an individual image (i.e. concatenate the rows of each image into a single row). Compute the average face M_avg, and subtract that face from each row in M. M = M  M_avgC = (1/m) * SUM(Ai*Ai')(A' is the transpose of matrix A) 
Where Ai is the vectorized image from row i in M, and the SUM is over all images for i=[1,m]. So if the size of each image is n = height * width, then the size of the covariance matrix is n x n. Calculating the eigenvectors directly from this matrix is computationally infeasible, so instead I used this neat linear algebra trick (courtesy of Wikipedia):
Given the number of images m and meansubtracted image matrix M, we notice that if ui is an eigenvector M*M', then vi = T' * uiWe can get the eigenvectors of C and their associated eigenvalues using Matlab's handy eigenvector function eig: T = M*M'
(gets you a matrix U of ui eigenvectors and diagonal matrix D of their associated eigenvalues)

Compute a difference matrix L for each of the m training images, where L is defined as a vector of scalars:
Face Detection Run the images through a face detection algorithm as preprocessing step (i.e. perform eigenface computation on a set of images that look more like the examples to the left, rather than figure 1). Using training data where the faces were more centralized would hopefully result in eigenfaces that defined actual facial features, and were more invariant to hair, background, pose, head size, etc. 

Autoalignment Align the images so that the eyes are in the same place in each figure as a preprocessing step; this would result in a more meaningful average face, and the resulting eigenfaces would be more representative of facial features rather than "image" features (i.e. if someone's head was more to the left of the picture frame rather than centered). The example on the right is an average face from Drexel's implementation of eigenfaces. The example on the left is my resulting mean face from the male directory. 

