Landmark Labelling for 3D Faces Clement Creusot, Nick Pears, Jim Austin Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 1
Table of Content ■ Motivation Motivation ■ Problem Problem Solution ■ Solution Results ■ Results Conclusion ■ Conclusion Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 2
Motivation ● Non-cooperative Recognition at a distance ● Modality ● Difficult Cases ● Review ● Assumptions Motivation Problem Solution Results Conclusion Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 3
Non-cooperative Recognition at a distance Motivation ● Non-cooperative Recognition at a distance ■ Application: ● Modality ● Difficult Cases ◆ Surveillance ● Review ● Assumptions ◆ Human-Machine Interaction Problem Solution Results Conclusion Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 4
Non-cooperative Recognition at a distance Motivation ● Non-cooperative Recognition at a distance ■ Application: ● Modality ● Difficult Cases ◆ Surveillance ● Review ● Assumptions ◆ Human-Machine Interaction Problem ■ Problems: Solution ◆ Pose From [Savran et al., 2008] Results ◆ Occlusion Conclusion ◆ Speed From [Savran et al., 2008] Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 4
Modality ■ Non-Cooperative ⊃ Anti-cooperative Motivation ● Non-cooperative Recognition ■ Proved possible for big database at a distance ● Modality ● Difficult Cases ● Review ● Assumptions Problem Solution Results From [Proenca, 2008] From [Yan and Bowyer, 2007] From [Phillips et al., 2005] From [Havasi et al., 2007] Conclusion Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 5
Modality ■ Non-Cooperative ⊃ Anti-cooperative Motivation ● Non-cooperative Recognition ■ Proved possible for big database at a distance ● Modality ● Difficult Cases ● Review ● Assumptions Problem Solution Results From [Proenca, 2008] From [Yan and Bowyer, 2007] From [Phillips et al., 2005] From [Havasi et al., 2007] Conclusion ■ 2D or 3D ? From [Liu et al., 2007] Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 5
Difficult Cases Motivation ● Non-cooperative Recognition at a distance ● Modality ● Difficult Cases ● Review ● Assumptions Problem Solution Results Conclusion ■ Recognition: ◆ Holistic method → Need for good Registration ◆ Feature based method → Need for good Feature Localisation Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 6
Difficult Cases Motivation ● Non-cooperative Recognition at a distance ● Modality ● Difficult Cases ● Review ● Assumptions Problem Solution Results Conclusion ■ Recognition: ◆ Holistic method → Need for good Registration ◆ Feature based method → Need for good Feature Localisation ■ Will often fail at preprocessing ◆ Naive methods for feature detection ◆ Strong assumptions Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 6
Difficult Cases Motivation ● Non-cooperative Recognition at a distance ● Modality ● Difficult Cases ● Review ● Assumptions Problem Solution Results Conclusion ■ Recognition: ◆ Holistic method → Need for good Registration ◆ Feature based method → Need for good Feature Localisation ■ Will often fail at preprocessing ◆ Naive methods for feature detection ◆ Strong assumptions ■ Recquire better feature detection Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 6
Review ■ Almost all paper expect non occluded frontal face Motivation ● Non-cooperative Recognition ■ A few that don’t: at a distance ● Modality ● Difficult Cases ◆ Some orientation change: ● Review ■ [Colbry et al., 2005]: Curvature + ICP + Relaxation ● Assumptions ■ [Lu and Jain, 2006]: Directional Maximum Problem ■ [Faltemier et al., 2008]: Rotated Profile Signature Solution Results Conclusion [Lu and Jain, 2006] [Faltemier et al., 2008] [Colbry et al., 2005] ■ Almost all papers expect the nose will be present ■ Most papers recquire two well defined inner corners of the eyes Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 7
Assumptions ■ The ones we needed to make: Motivation ● Non-cooperative Recognition ◆ At least half of the face is visible at a distance ● Modality ◆ There exist features repeatable across individual ● Difficult Cases ● Review ■ The ones we did not make: ● Assumptions ◆ All landmark are present and will match there descriptor Problem ◆ Candidates for one landmark descriptor are rare Solution Results ■ The ones we made (only in post-processing) Conclusion ◆ The face is roughly convex ◆ Faces are not too flexible ( � = hand) ◆ Only 1 face per scene Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 8
Motivation Problem ● The landmark Detection Problem ● Input Generation Problem Solution Results Conclusion Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 9
The landmark Detection Problem Motivation Problem ● The landmark Detection Positions + Labels Problem ● Input Generation Solution Input Mesh Landmarks Results Landmarking Conclusion ■ Landmark = Position + Label ■ Two Approaches: ◆ Select One Label + Find Corresponding Position ◆ Find All Positions + Find Corresponding Labels Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 10
The landmark Detection Problem Motivation Problem ● The landmark Detection Problem ● Input Generation Solution Input Mesh Points Landmarks Results Repeatable Point Detection Labelling Conclusion ■ Landmark = Position + Label ■ Two Approaches: ◆ Select One Label + Find Corresponding Position ◆ Find All Positions + Find Corresponding Labels Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 10
The landmark Detection Problem Motivation Problem ● The landmark Detection Problem ● Input Generation Solution Input Mesh Points Landmarks Results Labelling Conclusion ■ Landmark = Position + Label ■ Two Approaches: ◆ Select One Label + Find Corresponding Position ◆ Find All Positions + Find Corresponding Labels Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 10
Input Generation Motivation Problem ● The landmark Detection Problem ● Input Generation Solution Results Conclusion Input Points Mesh Automatic Points Hand-Placed Points + Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 11
Motivation Problem Solution ● Our Strategy ● Graph Generation Solution ● Graph Matching ● Elimination ● Post-Processing Results Conclusion Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 12
Our Strategy Motivation Problem Solution ● Our Strategy ● Graph Generation ● Graph Matching ● Elimination ● Post-Processing Results Conclusion Input Position Graph Matching Post-Processing Results • Multi-attribute seeding • Threshold on scores • Unit-Quaternion clustering • Relaxation by elimination Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 13
Graph Generation Motivation Problem Neighbourhood Mesh Scalars & Vectors Solution ● Our Strategy ● Graph Generation ● Graph Matching ● Elimination ● Post-Processing Input Points H,K,SI,Vol,LC Results Node Attributes Conclusion Graphs Edge Attributes Eucl.Dist., Geod.Dist., Ratio, ∆ H, . . . ■ Graph Properties: ◆ Complete Graph (for now) ◆ 5 attributes per Node ◆ 7 attributes per Edge Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 14
Graph Matching Motivation Problem ■ Structure Solution ◆ list of candidates ● Our Strategy ● Graph Generation ◆ Associated scores ● Graph Matching ● Elimination ● Post-Processing Results Conclusion Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 15
Graph Matching Motivation Problem ■ Structure Solution ◆ list of candidates ● Our Strategy ● Graph Generation ◆ Associated scores ● Graph Matching ● Elimination ■ Objective: ● Post-Processing ◆ Reduce correspondence Nb Results Conclusion Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 15
Graph Matching N properties Motivation Query . . . 1.1 120.0 5.6 × 0 . 2 × 0 . 5 × 0 . 3 Problem . . . Model ■ Structure Solution ◆ list of candidates ● Our Strategy 0.5 × 0 . 2 0.65 × 0 . 5 0.9 × 0 . 3 Σ . . . = Score ● Graph Generation ◆ Associated scores ● Graph Matching ● Elimination ■ Objective: ● Post-Processing ◆ Reduce correspondence Nb Results ■ Seeding Conclusion LDA ◆ Partial scores → Score Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 15
Graph Matching Motivation Problem ■ Structure Solution ◆ list of candidates ● Our Strategy ● Graph Generation ◆ Associated scores ● Graph Matching ● Elimination ■ Objective: ● Post-Processing ◆ Reduce correspondence Nb Results ■ Seeding Conclusion LDA ◆ Partial scores → Score ■ Relaxation on hyperedges ( � = [Christmas et al., 1995]) Clément Creusot, October 25th, 2010 3D Object Retrieval, Florence - p. 15
Recommend
More recommend
