Face detection, local features face alignment, and Face detection - - PowerPoint PPT Presentation

▶
face detection
SMART_READER_LITE
LIVE PREVIEW

Face detection, local features face alignment, and Face detection - - PowerPoint PPT Presentation

Nov 27, 2022 127 likes •242 views

12/6/2013 Lecture overview Brief introduction to Face detection, local features face alignment, and Face detection http://www.ima.umn.edu/2008-2009/MM8.5-14.09/activities/Wohlberg-Brendt/sift.png face image parsing

slide-1
SLIDE 1

12/6/2013 1

Face detection, face alignment, and face image parsing

Brandon M. Smith Guest Lecturer, CS 534 Monday, October 21, 2013

12/6/2013 1 CS 534: Computation Photography

Lecture overview

  • Brief introduction to

local features

  • Face detection
  • Face alignment and

landmark localization

  • Face image parsing

12/6/2013 CS 534: Computation Photography 2

http://www.ima.umn.edu/2008-2009/MM8.5-14.09/activities/Wohlberg-Brendt/sift.png http://www.noio.nl/assets/2011-03-01-stitching-smiles/violajones.png http://www.mathworks.com/matlabcentral/fx_files/32704/4/icaam.jpg http://homes.cs.washington.edu/~neeraj/projects/face-parts/images/teaser.png

Local features: broad goal

  • What are local features trying to capture?

The local appearance in a region of the image

12/6/2013 CS 534: Computation Photography 3

David G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004)

Local features: motivation

What are their uses?

  • Matching

12/6/2013 CS 534: Computation Photography 4

http://docs.opencv.org/_images/Featur_FlannMatcher_Result.jpg

slide-2
SLIDE 2

12/6/2013 2

Local features: motivation

What are their uses?

  • Matching
  • Image indexing and retrieval

12/6/2013 CS 534: Computation Photography 5

Shen et al., CVPR 2012

Local features: motivation

What are their uses?

  • Matching
  • Image indexing and retrieval
  • Aligning images, e.g., for panorama stitching

12/6/2013 CS 534: Computation Photography 6

http://www.leet.it/home/lale/files/Garda-pano.jpg

Local features: motivation

What are their uses?

  • Matching
  • Image indexing and retrieval
  • Aligning images, e.g., for panorama stitching
  • Video stabilization

12/6/2013 CS 534: Computation Photography 7

Local features: motivation

What are their uses?

  • Matching
  • Image indexing and retrieval
  • Aligning images, e.g., for panorama stitching
  • Video stabilization
  • 3D reconstruction

12/6/2013 CS 534: Computation Photography 8

http://www.nsf.gov/news/special_reports/science_nation/images/virtualrealitymaps/d uomopisa500.jpg

slide-3
SLIDE 3

12/6/2013 3

Local features: motivation

What are their uses?

  • Matching
  • Image indexing and retrieval
  • Aligning images, e.g., for panorama stitching
  • Video stabilization
  • 3D reconstruction
  • Object recognition, including face recognition

12/6/2013 CS 534: Computation Photography 9

http://doi.ieeecomputersociety.org/cms/Computer.org/dl/trans/tp/2007/11/figures/i192714.gif

Local features: types

Types of features and feature descriptors

  • Image intensity or gradient patches
  • Shift Invariance Feature Transform (SIFT) – very

popular!

  • DAISY
  • SURF
  • Many more…

12/6/2013 CS 534: Computation Photography 12

Face detection: goal

Automatically detect the presence and location of faces in images.

12/6/2013 CS 534: Computation Photography 13

Shen et al., Detecting and Aligning Faces by Image Retrieval, CVPR 2013

Face detection: motivation

  • Automatic camera focus

12/6/2013 CS 534: Computation Photography 14

http://cdn.conversations.nokia.com.s3.amazonaws.com/wp-content/uploads/2013/09/Nokia-Pro-Camera-auto-focus_half-press.jpg http://cdn.conversations.nokia.com.s3.amazonaws.com/wp-content/uploads/2013/09/Nokia-Pro-Camera-auto-focus_half-press.jpg

slide-4
SLIDE 4

12/6/2013 4

Face detection: motivation

  • Automatic camera focus
  • Easier photo tagging

12/6/2013 CS 534: Computation Photography 15

http://sphotos-d.ak.fbcdn.net/hphotos-ak-ash3/163475_10150118904661729_7246884_n.jpg

Face detection: motivation

  • Automatic camera focus
  • Easier photo tagging
  • First step in any face recognition algorithm

12/6/2013 CS 534: Computation Photography 16

http://images.fastcompany.com/upload/camo1.jpg

Face detection: challenges

  • Large face shape and appearance variation
  • Scale and rotation (yaw, roll, pitch) variation
  • Background clutter
  • Occlusions
  • Image noise
  • Efficiency
  • False positives

12/6/2013 CS 534: Computation Photography 17

Face detection: Viola-Jones*

  • Paul Viola and Michael Jones, Robust Real-time

Face Detection, International Journal of Computer Vision (IJCV), 2004.

  • Feature type?
  • Which features are important?
  • Decide: face or not a face

12/6/2013 CS 534: Computation Photography 18 * Next few slides are based on a presentation by Kostantina Pall & Alfredo Kalaitzis, available at http://www1.cs.columbia.edu/~belhumeur/courses/biometrics/2010/violajones.ppt

slide-5
SLIDE 5

12/6/2013 5

Face detection: Viola-Jones

Feature type?

  • Useful domain knowledge:
  • The eye region is darker than the forehead or the

upper cheeks

  • The nose bridge region is brighter than the eyes
  • The mouth is darker than the chin
  • Encoding
  • Location and size: eyes, nose bridge, mouth, etc.
  • Value: darker vs. brighter

12/6/2013 CS 534: Computation Photography 19

Face detection: Viola-Jones

Feature type?

  • Rectangle features
  • Value = ∑(pixels in black)
  • ∑(pixels in white)
  • Three types: 2,3,4 rectangles
  • Very fast: integral image

12/6/2013 CS 534: Computation Photography 20

Face detection: Viola-Jones

12/6/2013 CS 534: Computation Photography 21

* From http://www.cs.ubc.ca/~lowe/425/slides/13-ViolaJones.pdf

Which features are important?

  • Tens of thousands of features to choose from
  • AdaBoost (Singer and Schapire, 1997)
  • Given a set of weak classifiers: ℎ𝑢 𝑦 ∈ {−1,1}
  • Iteratively combine classifiers to form a strong

classifier: 𝐼 𝑦 = 1 𝑗𝑔 𝛽𝑢ℎ𝑢(𝑦)

𝑢

> 𝑢ℎ𝑠𝑓𝑡ℎ𝑝𝑚𝑒 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓

Face detection: Viola-Jones

Final decision: face or not a face

  • Cascade of classifiers
  • 1. Two-feature classifier: >99% recall, >60% precision
  • 2. Five-feature classifier
  • 3. 10-feature classifier

… 10. 200-feature classifier

12/6/2013 CS 534: Computation Photography 22

slide-6
SLIDE 6

12/6/2013 6

Face detection: Viola-Jones

12/6/2013 CS 534: Computation Photography 23

http://vimeo.com/12774628#

Face detection: recent approaches

Xiangxin Zhu and Deva Ramanan, Face Detection, Pose Estimation, and Landmark Localization in the Wild, CVPR 2012.

12/6/2013 CS 534: Computation Photography 24

Face detection: recent approaches

Shen et al., Detecting and Aligning Faces by Image Retrieval, CVPR 2013.

12/6/2013 CS 534: Computation Photography 25

Face detection: recent approaches

Shen et al., Detecting and Aligning Faces by Image Retrieval, CVPR 2013.

12/6/2013 CS 534: Computation Photography 26

slide-7
SLIDE 7

12/6/2013 7

Face alignment and landmark localization: goal

Goal of face alignment: automatically align a face (usually non-rigidly) to a canonical reference Goal of face landmark localization: automatically locate face landmarks of interests

12/6/2013 CS 534: Computation Photography 27

http://www.mathworks.com/matlabcentral/fx_files/32704/4/icaam.jpg http://homes.cs.washington.edu/~neeraj/projects/face-parts/images/teaser.png

Face alignment and landmark localization: motivation

  • Preprocess for:
  • Face recognition
  • Portrait editing wizards
  • Face image retrieval
  • …
  • Face tracking
  • Expression recognition
  • Facial pose recognition

12/6/2013 CS 534: Computation Photography 28

http://static3.businessinsider.com/image/52127e2 169bedd4d60000012-752-564/realeyes-facial- recognition.png http://mission0ps.com/wp-content/uploads/2013/04/10-special-effects.jpg

Face alignment and landmark localization: challenges

  • Pose
  • Expression
  • Identity variation
  • Occlusions
  • Image noise

12/6/2013 CS 534: Computation Photography 29

Face alignment and landmark localization: approaches

Parametric appearance models

  • Cootes, Edwards, and Taylor, Active Appearance Models, ECCV 1998

12/6/2013 CS 534: Computation Photography 30

slide-8
SLIDE 8

12/6/2013 8

Face alignment and landmark localization: approaches

Parametric appearance models

  • Cootes, Edwards, and Taylor, Active Appearance Models, ECCV 1998

12/6/2013 CS 534: Computation Photography 31

Face alignment and landmark localization: approaches

Part-based deformable models

  • Saragih et al., Face Alignment through Subspace Constrained Mean-

Shifts, ICCV 2009

12/6/2013 CS 534: Computation Photography 32

Face alignment and landmark localization: approaches

Part-based deformable models

  • Saragih et al., Face Alignment through Subspace Constrained Mean-

Shifts, ICCV 2009

12/6/2013 CS 534: Computation Photography 33

Face alignment and landmark localization: approaches

Supervised descent

  • Xiong and De la Torre, Supervised Descent Method and its Applications to

Face Alignment, CVPR 2013

12/6/2013 CS 534: Computation Photography 34

slide-9
SLIDE 9

12/6/2013 9

Face alignment and landmark localization: approaches

Exemplar-based/non-parametric methods

  • Shen et al., Detecting and Aligning Faces by Image Retrieval, CVPR 2013.

12/6/2013 CS 534: Computation Photography 35

Face image parsing

Smith, Zhang, Brandt, Lin, and Yang, Exemplar-Based Face Parsing, CVPR 2013.

12/6/2013 CS 534: Computation Photography 36

Face image parsing: goal

Given an input face image, automatically segment the face into its constituent parts.

12/6/2013 CS 534: Computation Photography 37

Face image parsing: motivation

  • Like face alignment, can be used as a preprocess

for face recognition, automated portrait editing, etc.

  • Encodes ambiguity
  • Generalizes to hair, teeth, ears etc. across datasets

12/6/2013 CS 534: Computation Photography 38

slide-10
SLIDE 10

12/6/2013 10

Face image parsing: our approach

12/6/2013 CS 534: Computation Photography 39

2K exemplar images 11 landmarks ~150 SIFT features Exemplar labels

Database

Face image parsing: our approach

12/6/2013 CS 534: Computation Photography 40

2K exemplar images 11 landmarks ~150 SIFT features Exemplar labels

Database Step 0: Rough alignment & Top exemplar selection

100 top exemplars

Input

Face image parsing: our approach

12/6/2013 CS 534: Computation Photography 41

2K exemplar images 11 landmarks ~150 SIFT features Exemplar labels

Database Step 0: Rough alignment & Top exemplar selection Step 1: Nonrigid alignment Input

Face image parsing: our approach

12/6/2013 CS 534: Computation Photography 42

2K exemplar images 11 landmarks ~150 SIFT features Exemplar labels

Database Step 0: Rough alignment & Top exemplar selection Step 1: Nonrigid alignment Input Step 2: Exemplar label aggregation

* + * + … =

slide-11
SLIDE 11

12/6/2013 11

Face image parsing: our approach

12/6/2013 CS 534: Computation Photography 43

2K exemplar images 11 landmarks ~150 SIFT features Exemplar labels

Database Step 0: Rough alignment & Top exemplar selection Step 1: Nonrigid alignment Input Step 2: Exemplar label aggregation

* + * + =

Step 3: Pixel-wise label selection

w1 * + w2 * + w9 *

Label 1 Label 2

…

Label 9

Output

∝

Face image parsing: quantitative results

12/6/2013 CS 534: Computation Photography 44

Face image parsing: qualitative results

12/6/2013 CS 534: Computation Photography 45

+

Input Soft segments Hard segments Ground truth

Face image parsing: qualitative results

12/6/2013 CS 534: Computation Photography 46

+

Input Soft segments Hard segments Ground truth