The Mug-Shot Search Problem Ellie Baker and Margo Seltzer Harvard - - PowerPoint PPT Presentation

the mug shot search problem
SMART_READER_LITE
LIVE PREVIEW

The Mug-Shot Search Problem Ellie Baker and Margo Seltzer Harvard - - PowerPoint PPT Presentation

The Mug-Shot Search Problem Ellie Baker and Margo Seltzer Harvard University, Cambridge, MA., USA ellie@eecs.harvard.edu Face Recognition plus Composite Creation Eigenfaces (Turk & Pentland 1991) Uses PCA to compress images to a


slide-1
SLIDE 1

The Mug-Shot Search Problem

Ellie Baker and Margo Seltzer

Harvard University, Cambridge, MA., USA ellie@eecs.harvard.edu

slide-2
SLIDE 2

2

Face Recognition plus Composite Creation

  • Eigenfaces (Turk & Pentland 1991)
  • Uses PCA to compress images to a low dimensional

space of small set of basis vectors called eigenfaces.

  • Location in eigenface-space determines the distance

between images.

  • Distance from a query image can be used to specify a

sort order on a database.

  • Composites
slide-3
SLIDE 3

3

User Study Goals

  • How well does the eigenface metric

correlate with users’ assessments of facial similarity?

  • Given whatever level of correlation there

is between eigenfaces and human users, what search strategies make the best use of it?

  • Are the composites helpful?
slide-4
SLIDE 4

4

Prototype System Overview

  • Uses eigenface engine and 4500 image

database from Photobook (Pentland, Picard,

Sclaroff - 1994).

  • Queries are either database faces or

composites.

  • Composites are constructed by

recombining parts from images in the database.

  • Interim composites may be used for

retrieval and interim retrieval results may likewise be used to update an evolving composite.

slide-5
SLIDE 5

5

Composite Creation

  • Random generation and feature editing
slide-6
SLIDE 6

6

Register Mental Image

slide-7
SLIDE 7

7

View 100 Random Database Faces

slide-8
SLIDE 8

8

System Generates 10 Random Composites From User’s Choices.

slide-9
SLIDE 9

9

User Produces a Composite Via Manual Editing

slide-10
SLIDE 10

10

Evaluation Post-mortem

  • image score = number of image

inspections required to find target if that image is used as a query.

  • strategy score = number of image

inspections required to find target using that strategy.

  • Determine image scores for each of

users’:

  • Top five database choices
  • Random composite choice
  • Final edited composite
  • Which strategies elicit the best average

scores over all subjects?

slide-11
SLIDE 11

11

Best and Worst Case Expected Strategy Scores

  • Worst Case : sequential search on 4500

images —expected strategy score is 2250.

  • Expected image score of closest of N

random selections is ~(DatabaseSize/N).

  • 4500/100 = 45, so expected score of

closest image in random set of 100 is 45.

  • Best Case: expected strategy score is

100+45 = 145.

slide-12
SLIDE 12

12

Eigenface Best vs. Users’ best

40 137 168 206 1230 40 942 137

slide-13
SLIDE 13

13

Results

  • Mean scores for optimal strategies

(within a defined class of “reasonable” strategies)

Target 1: Database images only 323 Target 1: Database + Composites 260 Target 2: Database images only 677 Target 2: Database + Composites 482

slide-14
SLIDE 14

14

Conclusions

  • Eigenface correlation with users’

similarity metric exists, but is far from perfect.

  • Composites definitely help.
  • Hybrid search strategies that use both

composites and database images as queries appear to be most successful.