CIS 5543 – Computer Vision Preface Face Recognition Haibin Ling Many slides revised from K. Grosse, R. Fergus, S. Lazebnik Karl Grosse Preface Motivation Face recognition Given a test face and a set of reference faces Application Demands in a database find the N closest reference Nonintrusive identification faces to the test one. Nonintrusive verification Nonintrusive access control Face authentification Identification for law enforcement Given a test face and a reference one, decide if the test face is identical to the reference one. 3 4 Karl Grosse Karl Grosse Challenges in face recognition Outline Many variations Holistic face recognition, intensity based Pose variation Eigenfaces Illumination conditions M. Turk and A. Pentland, Face Recognition Scale variability using Eigenfaces, CVPR 1991 Age difference Expression Modeling texture and geometry Elastic Bunch Graph Matching Varied image conditions Occlusion Shape and appearance Low resolution Noise Active Appearance models 5 1
Principal Component Analysis Principal Component Analysis • Direction that maximizes the variance of the • Given: N data points x 1 , … ,x N in R d projected data: N • We want to find a new set of features that are linear combinations of original ones: Projection of data point u ( x i ) = u T ( x i – µ ) N ( µ : mean of data points) • What unit vector u in R d captures the Covariance matrix of data most variance of the data? The direction that maximizes the variance is the eigenvector associated with the largest eigenvalue of Σ Eigenfaces: Key idea Eigenface examples • Assume that most face images lie on a low- Training dimensional subspace determined by the first k images ( k < d ) directions of maximum variance x 1 ,…, x N • Use PCA to determine the vectors u 1 ,… u k that span that subspace: x ≈ μ + w 1 u 1 + w 2 u 2 + … + w k u k • Represent each face using its “face space” coordinates (w 1 ,…w k ) • Perform nearest-neighbor recognition in “face space” M. Turk and A. Pentland, Face Recognition using Eigenfaces, CVPR 1991 Eigenface example Eigenfaces example • Face x in “face space” coordinates: Top eigenvectors: u 1 ,… u k Mean: μ = 2
Eigenfaces example Summary: Recognition with eigenfaces Process labeled training images: • Face x in “face space” coordinates: Find mean µ and covariance matrix Σ • Find k principal components (eigenvectors of Σ ) u 1 ,… u k • Project each training image x i onto subspace spanned by • principal components: = (w i1 ,…,w ik ) = ( u 1 T ( x i – µ ), … , u k T ( x i – µ )) Reconstruction: Given novel image x : Project onto subspace: • (w 1 ,…,w k ) = ( u 1 T ( x – µ ), … , u k T ( x – µ )) Optional: check reconstruction error x – x to determine • = + ^ whether image is really a face Classify as closest training face in k-dimensional subspace • ^ x = µ + w 1 u 1 +w 2 u 2 +w 3 u 3 +w 4 u 4 + … Limitations Limitations • PCA assumes that the data has a Gaussian • Global appearance method: not robust to distribution (mean µ, covariance matrix Σ ) misalignment, background variation The shape of this dataset is not well described by its principal components Other Component Analysis Outline • Is principle component the right one? Holistic face recognition, intensity based • Direction of maximum variance good for classification? Eigenfaces Shape and appearance Active Appearance models Cootes, Edwards, and Taylor, “Active Appearance Models”, ECCV 1998 • More subspace methods: Modeling texture and geometry • Fisherfaces (LDA, Belhumeur et al. 1997) • Independent Component Analysis (ICA, Bartlett et al. 2002) Elastic Bunch Graph Matching • Nonlinear embedding • Laplacian face (LPP, He et al. 2005) 3
Essence of the Idea: Recognition by Synthesis So what’s a model Model Explain a new example in terms of the model parameters “ texture” “ Shape” Slide: Dhruv Batra Slide: Dhruv Batra Shape Vector Active Shape Models training set = Provides alignment! 43 Slide: Dhruv Batra Slide: Dhruv Batra The Morphable Face Model Texture Models The structure of a face Shape vector S = (x 1 , y 1 , x 2 , … , y n ) T , containing the (x,y) coordinates of vertices of a face, Appearance vector T = (R 1 , G 1 , B 1 , R 2 , … , G n , B n ) T , warp to mean shape containing the color values of the mean-warped face image. Shape S Appearance T Slide: Dhruv Batra Cootes, Edwards, and Taylor, “Active Appearance Models”, ECCV 1998 4
The Morphable face model Active Appearance Model Search (Results) Again, assuming that we have m such vector pairs in full correspondence, we can form new shapes S model and new appearances T model as: m m T T S a S b model i i model i i i 1 i 1 If number of basis faces m is large enough to span the face subspace then: Any new face can be represented as a pair of vectors ( 1 , 2 m ) T and ( 1 , 2 m ) T ! Slide: Dhruv Batra Overview Playing with the Parameters Holistic face recognition, intensity based Eigenfaces Shape and appearance Active Appearance models First two modes of shape variation First two modes of gray-level variation Modeling texture and geometry Elastic Bunch Graph Matching First four modes of L. Wiskott, J.M. Fellouse, N. Krüger, C.v.d.Malsburg Face appearance Recognition by Elastic Bunch Graph Matching, PAMI variation 1997 Slide: Dhruv Batra EBGM Overview Gabor wavelets Human faces share a similar topological structure Shape of plane waves restricted by a Gaussian envelope Labeled graph as basic object representation function Nodes positioned at fiducial points (eyes, nose…) Hence good results in practice Jets at each node Biologically motivated Edges labeled with distance information Pro: Stored model graph matched to new images Image graph (can become model graph) Invariant to changes in brightness Robust against translation or distortion Model graphs easily translated, scaled, orientated Con: Dependent on the background of the image Karl Grosse Karl Grosse 5
Gabor wavelets Jets Family of Gabor kernels Wavelets for different frequencies and orientation Jet describes a small patch of grey values Defined as the set of complex coefficients In the shape of plane waves with wave vector J={J i } restricted by a Gaussian envelope function. for a given pixel 5 different frequencies 8 orientations Width of Gaussian controlled by Family of kernels is self-similar and generated from one mother wavelet by dilation and rotation! Karl Grosse Karl Grosse Image graph Bunch graph Image Graph G: N nodes, E edges Constructing a Bunch graph B from M Image graphs Labeling of nodes: G BM : Jets J n at positions x n , n = 1,…, N Summarize the jets from a node Labeling of edges: Set of jets “Bunch” Distances Label nodes with Bunches between nodes n and n ´ Label edges with average distance Graph is not complete Karl Grosse Karl Grosse Matching Matching Goal: Calculate an Image graph for an image Four stages: Find approximate position 1. Refine position and size 2. Refine size and find aspect ratio 3. Local distortion 4. Initial Graph: Structure of Bunch Graph Image graphs found by probing with the Bunch graph Karl Grosse Karl Grosse 6
Matching Comparison of results Results of face recognition using the FERET db: Different poses: frontal, halfprofile, profile Top row: Image graphs manually marked Bottom row: Image graphs found by the system Karl Grosse Karl Grosse Conclusion Holistic face recognition Assuming faces are aligned Subspace approach Active shape/appearance model Separate shape and appearance Landmark based face warping Elastic Bunch Graph Matching Modeling topological with a graph Modeling local appearance with Gabor Open problems Alignment Occlusion and cluttering Expression, aging, glasses, facial hair 7
Recommend
More recommend
