SLIDE 1
CS 395T: Class Specific FaceTracer: A Search Engine for Large - - PowerPoint PPT Presentation
CS 395T: Class Specific FaceTracer: A Search Engine for Large - - PowerPoint PPT Presentation
CS 395T: Class Specific FaceTracer: A Search Engine for Large Collections of Images with Faces Nona Sirakova October 19 2012 Database Fromat: Eye & mouth corners for a single person per image Google VS MugShot Top picks for angry man
SLIDE 2
SLIDE 3
Google VS MugShot
Top picks for angry man
In the database, but not retrieved as angry.
SLIDE 4
Does MugHunt work with natural language?
SLIDE 5
Demo: Mug Hunt: http://mughunt.securics.com/
SLIDE 6
Features and their values:
SLIDE 7
Features to use in Experiment 1:
SLIDE 8
Examples:
Face GIST Face SIFT Mouth SIFT Eyes SIFT
SLIDE 9
Experiment 1 set-up:
Attribute Face Gist Error Face Sift Error Eyes Sift Error Mouth Sift Error Gender (male, female) 6.0 % 14.8 %
22.2 % 18.0 %
Age (baby, child, youth, middle age, senior) 14.3 % 20.0 %
24.0 % 24.4 %
Race (Asian, Black, White) 6.0 % 35.2 %
17.2 % 21.8 %
Hair Color (Blonde, not Blonde) 10.3 % 11.7 %
24.4 % 30.0 %
Eye Wear (none, eyeglasses, sunglasses) 4.0 % 9.0 %
4.4 % 43.0 %
Mustache (true, false) 3.7 % 8.2 %
34.8 % 4.0 %
Facial expression (smiling, not smiling) 3.5 % 4.0 %
43.8 % 6.4 %
gender: male female
SLIDE 10
Experiment 1 set-up:
Attribute Face Gist Error Face Sift Error Eyes Sift Error Mouth Sift Error Gender (male, female) 6.0 % 14.8 %
22.2 % 18.0 %
Age (baby, child, youth, middle age, senior) 14.3 % 20.0 %
24.0 % 24.4 %
Race (Asian, Black, White) 6.0 % 35.2 %
17.2 % 21.8 %
Hair Color (Blonde, not Blonde) 10.3 % 11.7 %
24.4 % 30.0 %
Eye Wear (none, eyeglasses, sunglasses) 4.0 % 9.0 %
4.4 % 43.0 %
Mustache (true, false) 3.7 % 8.2 %
34.8 % 4.0 %
Facial expression (smiling, not smiling) 3.5 % 4.0 %
43.8 % 6.4 %
gender: male female
SLIDE 11
Experiment 1 set-up:
Attribute Face Gist Error Face Sift Error Eyes Sift Error Mouth Sift Error Gender (male, female) 6.0 % 14.8 %
22.2 % 18.0 %
Age (baby, child, youth, middle age, senior) 14.3 % 20.0 %
24.0 % 24.4 %
Race (Asian, Black, White) 6.0 % 35.2 %
17.2 % 21.8 %
Hair Color (Blonde, not Blonde) 10.3 % 11.7 %
24.4 % 30.0 %
Eye Wear (none, eyeglasses, sunglasses) 4.0 % 9.0 %
4.4 % 43.0 %
Mustache (true, false) 3.7 % 8.2 %
34.8 % 4.0 %
Facial expression (smiling, not smiling) 3.5 % 4.0 %
43.8 % 6.4 %
SLIDE 12
Experiment 1 set-up:
Attribute Face Gist Error Face Sift Error Eyes Sift Error Mouth Sift Error Gender (male, female) 6.0 % 14.8 %
22.2 % 18.0 %
Age (baby, child, youth, middle age, senior) 14.3 % 20.0 %
24.0 % 24.4 %
Race (Asian, Black, White) 6.0 % 35.2 %
17.2 % 21.8 %
Hair Color (Blonde, not Blonde) 10.3 % 11.7 %
24.4 % 30.0 %
Eye Wear (none, eyeglasses, sunglasses) 4.0 % 9.0 %
4.4 % 43.0 %
Mustache (true, false) 3.7 % 8.2 %
34.8 % 4.0 %
Facial expression (smiling, not smiling) 3.5 % 4.0 %
43.8 % 6.4 %
SLIDE 13
Experiment 1 set-up:
Attribute Face Gist Error Face Sift Error Eyes Sift Error Mouth Sift Error Gender (male, female) 6.0 % 14.8 %
22.2 % 18.0 %
Age (baby, child, youth, middle age, senior) 14.3 % 20.0 %
24.0 % 24.4 %
Race (Asian, Black, White) 6.0 % 35.2 %
17.2 % 21.8 %
Hair Color (Blonde, not Blonde) 10.3 % 11.7 %
24.4 % 30.0 %
Eye Wear (none, eyeglasses, sunglasses) 4.0 % 9.0 %
4.4 % 43.0 %
Mustache (true, false) 3.7 % 8.2 %
34.8 % 4.0 %
Facial expression (smiling, not smiling) 3.5 % 4.0 %
43.8 % 6.4 %
SLIDE 14
Experiment 1 set-up:
Attribute Face Gist Error Face Sift Error Eyes Sift Error Mouth Sift Error Gender (male, female) 6.0 % 14.8 %
22.2 % 18.0 %
Age (baby, child, youth, middle age, senior) 14.3 % 20.0 %
24.0 % 24.4 %
Race (Asian, Black, White) 6.0 % 35.2 %
17.2 % 21.8 %
Hair Color (Blonde, not Blonde) 10.3 % 11.7 %
24.4 % 30.0 %
Eye Wear (none, eyeglasses, sunglasses) 4.0 % 9.0 %
4.4 % 43.0 %
Mustache (true, false) 3.7 % 8.2 %
34.8 % 4.0 %
Facial expression (smiling, not smiling) 3.5 % 4.0 %
43.8 % 6.4 %
SLIDE 15
Experiment 2 set-up:
- Part 1:
○ Find the GIST descriptor for each face. ○ Plug in GIST space. ○ For a query, plug the query in GIST space. ○ Find query's 5 nearest neighbors.
- Part 2:
○ Find the GIST descriptor for each face. ○ Plug in GIST space & create descriptors. ○ Create an attribute space, and describe every image in terms of its attributes. ○ For a query, find the nearest 5 neighbors in the attribute space.
- Compare part 1 and part 2.
SLIDE 16
Experiment 2 set-up Part 1:
- Find the GIST descriptor for each face.
- Plug in GIST space.
SLIDE 17
Experiment 2 set-up Part 1:
- Find the GIST descriptor for query face.
Visual Query
SLIDE 18
Experiment 2 set-up Part 1:
- Plug query's GIST descriptor in GIST space.
Visual Query
SLIDE 19
Experiment 2 set-up Part 1:
- Find query's 5 nearest neighbors.
Visual Query
SLIDE 20
Experiment 2 set-up Part 2:
- Find the GIST descriptor for each face.
- Plug descriptor in GIST space.
- So far, just like part 1.
SLIDE 21
Experiment 2 set-up Part 2:
- Use SVM on for to train for each attribute.
Male VS Female Smiling VS Not Smiling Eye Wear VS No Eye Wear
SLIDE 22
Experiment 2 set-up Part 2:
- Each GIST point now has attribute-space
coordinates:
Male VS Female Smiling VS Not Smiling Eye Wear VS No Eye Wear
[ - 3.7, 0.4 , 3.5 ]
SLIDE 23
Experiment 2 set-up Part 2:
- Create an attribute space, and describe
every image in terms of its attributes.
Facial Expression Eye Wear Gender
[ 7.2, 11, -3 ]
SLIDE 24
Experiment 2 set-up Part 2:
- Create an attribute space, and describe
every image in terms of its attributes.
Facial Expression Eye Wear Gender
[ 7.2, 11, -3 ]
SLIDE 25
Experiment 2 set-up Part 2:
- For a query image: plug the attribute vector
into the attribute space and take the closest 5 neighbors:
Facial Expression Eye Wear Gender
[ 15.2, 6, 22 ]
SLIDE 26
Exp 2 Results Attribute VS Gist Space:
Attribute Space GIST Space
SLIDE 27
Exp 2 Results Attribute VS Gist Space:
SLIDE 28
Exp 2 Results Attribute VS Gist Space:
I drew in the beard to illustrate how much the man looks like the one in the closest image.
SLIDE 29
Exp 2 Results Attribute VS Gist Space:
SLIDE 30
Exp 2 Results Attribute VS Gist Space:
SLIDE 31