Automatic Keypoint Detection on 3D Faces Using a Dictionary of Local - - PowerPoint PPT Presentation

▶

May 08, 2023 421 likes •600 views

Automatic Keypoint Detection on 3D Faces Using a Dictionary of Local Shapes Clement Creusot, Nick Pears, Jim Austin Advanced Computer Architecture group Department of Computer science 3DIMPVT, Hangzhou, China, May 2011 Aim What Why How

SLIDE 1

Automatic Keypoint Detection on 3D Faces Using a Dictionary of Local Shapes

Clement Creusot, Nick Pears, Jim Austin Advanced Computer Architecture group Department of Computer science

3DIMPVT, Hangzhou, China, May 2011

SLIDE 2

What Why How Results Conclusion

Aim

Keypoints detection (NOT LANDMARKS) Similar to any of 14 learnt features (Dictionary of local shapes)

Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 2 / 8

SLIDE 3

What Why Long Term Objective Gap in Research How Results Conclusion

Part of a bigger project

Landmarking Positions + Labels Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 3 / 8

SLIDE 4

What Why Long Term Objective Gap in Research How Results Conclusion

Part of a bigger project

Keypoint Detection Labeling Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 3 / 8

SLIDE 5

What Why Long Term Objective Gap in Research How Results Conclusion

Gap in Research

Most literature: 3 points max or single-point-of-failure design Weak features often discarded Almost no work on combining more than 2 descriptors Little literature that examine multiple descriptors over multiple scales Most people focused on landmarking, without giving the intermediate results on candidate detection (keypoints)

Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 4 / 8

SLIDE 6

What Why How Results Conclusion

Workflow

OFFLINE

Train Meshes ×D Descriptor Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

SLIDE 7

What Why How Results Conclusion

Workflow

OFFLINE

Statistical Distributions Landmarks Train Meshes ×D Descriptor Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

SLIDE 8

What Why How Results Conclusion

Workflow

OFFLINE

Statistical Distributions Landmarks Train Meshes ×D Descriptor Maps ×D×14 Score Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

SLIDE 9

What Why How Results Conclusion

Workflow

OFFLINE

Statistical Distributions Descriptor Weights . . . . . . . . . . . . Landmarks Train Meshes ×D Descriptor Maps ×D×14 Score Maps LDA Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

SLIDE 10

What Why How Results Conclusion

Workflow

ONLINE

Statistical Distributions Descriptor Weights . . . . . . . . . . . . Test Meshes ×D Descriptor Maps ×D×14 Score Maps Dictionary of local shapes Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

SLIDE 11

What Why How Results Conclusion

Workflow

ONLINE

Statistical Distributions Descriptor Weights . . . . . . . . . . . . Test Meshes ×D Descriptor Maps ×D×14 Score Maps ×14 Mixed Maps Dictionary of local shapes Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

SLIDE 12

What Why How Results Conclusion

Workflow

ONLINE

Statistical Distributions Descriptor Weights . . . . . . . . . . . . Test Meshes ×D Descriptor Maps ×D×14 Score Maps ×14 Mixed Maps ×1 Final Map Keypoints Dictionary of local shapes Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

SLIDE 13

What Why How Results Results Examples Conclusion

Results

Sparse selection (max 1%) Reapeatable (same subject registration)

∼75% (at 10 mm)

Close to human hand-placed landmarks

average All: ∼85% (at 10 mm) average Nose: ∼99% (at 10 mm) average Eyes: ∼90% (at 10 mm)

High proportion of the local shapes retreived

∼11.88/14 (at 10 mm)

80 85 90 95 100 5 10 15 20 25 30 Percentage of Match Matching Acceptance Radius (mm) 00 01 02 03 04 05 06 07 08 09 10 11 12 13 80 85 90 95 100 5 10 15 20 25 30 Percentage of Match Matching Acceptance Radius (mm) 00 01 02 03 04 05 06 07 08 09 10 11 12 13

Configuration 1 Configuration 2 Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 6 / 8

SLIDE 14

What Why How Results Results Examples Conclusion

Examples

0.224 0.224 0.612 0.612 1.00 1.00 0.265 0.265 0.632 0.632 1.00 1.00 0.270 0.270 0.635 0.635 1.00 1.00

Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 7 / 8

SLIDE 15

What Why How Results Conclusion

Conclusion

Good points:

Detects ”weak” features No single-point-of-failure design

Limitations:

Can be time consuming article: 7s, now: 0.5s (8 desc.) Linear combination of scores

Future Work:

Non linear methods (boosting, kernel methods) Structural matching to deduce correspondences Comparison with a new clustering technique for keypoint detection

Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 8 / 8

SLIDE 16

What Why How Results Conclusion

Conclusion

Good points:

Detects ”weak” features No single-point-of-failure design

Limitations:

Can be time consuming article: 7s, now: 0.5s (8 desc.) Linear combination of scores

Future Work:

Non linear methods (boosting, kernel methods) Structural matching to deduce correspondences Comparison with a new clustering technique for keypoint detection

Thank You For Listening!

http://www.cs.york.ac.uk/~creusot

Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 8 / 8