Face Recognition
Hamid Dehghani April 2021
Plan
• Eigenfaces: the idea
• Eigenvectors and Eigenvalues
• Co-variance
• Learning Eigenfaces from training sets of faces • Recognition and reconstruction
Eigenfaces: the idea
• Think of a face as being a weighted combination of some
“component” or “basis” faces
• These basis faces are called eigenfaces
-8029 2900 1751 1445 4238 6193
…
• •
Eigenfaces: representing faces
These basis faces can be differently weighted to represent any face
So we can use different vectors of weights to represent different faces
-8029 2900 1751 1445 4238 6193
-1183
-2088
-4336
-669
-4221
10549
Learning Eigenfaces
Q: How do we pick the set of basis faces? A: We take a set of real training faces
…
Then we find (learn) a set of basis faces which best represent the differences between them
We’ll use a statistical criterion for measuring this notion of “best representation of the differences between the training faces”
We can then store each face as a set of weights for those basis faces
Using Eigenfaces: recognition & reconstruction
• We can use the eigenfaces in two ways
• 1: we can store and then reconstruct a face from a set of weights
• 2: we can recognise a new picture of a familiar face
Learning Eigenfaces • How do we learn them?
• We use a method called Principle Components Analysis (PCA)
• To understand this we will need to understand – What an eigenvector is
– What covariance is
– what is happening in PCA qualitatively
Eigenfaces
• All we are doing in the face case is treating the face as a point in a high- dimensional space, and then treating the training set of face pictures as our set of points
• To train:
– Takeyourimages,andre-arrangeasa2Dmatrix
• Rows: Each image
• Columns: Each pixel value
– Wecalculatethecovariancematrixofthefaces
– Wethenfindtheeigenvectorsofthatcovariancematrix
• These eigenvectors are the eigenfaces or basis faces
• Eigenfaces with bigger eigenvalues will explain more of the variation in the set of faces, i.e. will be more distinguishing
Eigenfaces: image space to face space
• When we see an image of a face we can transform it to face space w =xi.v
• There are k=1…n eigenfaces vk
• The ith face in image space is a vector xi
• The corresponding weight is wk
• We calculate the corresponding weight for every eigenface
kk
Recognition in face space
• Recognition is now simple. We find the euclidean distance d between our face and all the other stored faces in face space:
2 d(w,w)= å(w-w)
n
• The closest face in face space is the chosen match
1212
ii i=1
Reconstruction
• The more eigenfaces you have the better the reconstruction, but you can have high quality reconstruction even with a small number of eigenfaces
82 70 50
30 20 10
Summary
• Statistical approach to visual recognition
• Also used for object recognition
• Problems
• Reference: M. Turk and A. Pentland (1991). Eigenfaces for recognition, Journal of Cognitive Neuroscience, 3(1): 71–86.