What makes an image memorable? P. Isola, J. Xiao, A. Torralba, A. - - PowerPoint PPT Presentation

▶
what makes an image memorable
SMART_READER_LITE
LIVE PREVIEW

What makes an image memorable? P. Isola, J. Xiao, A. Torralba, A. - - PowerPoint PPT Presentation

Jan 15, 2023 334 likes •701 views

What makes an image memorable? P. Isola, J. Xiao, A. Torralba, A. Oliva. CVPR 2011 Islam Beltagy Experiment https://picasaweb.google. com/101392440470561716618/CS395TVi sualRecognition LabelMe Images database and annotation tool

slide-1
SLIDE 1

What makes an image memorable?

  • P. Isola, J. Xiao, A. Torralba, A. Oliva. CVPR 2011
Islam Beltagy
slide-2
SLIDE 2

Experiment

  • https://picasaweb.google.

com/101392440470561716618/CS395TVi sualRecognition

slide-3
SLIDE 3

LabelMe

  • Images database and annotation tool
  • A helpful matlab toolbox
slide-4
SLIDE 4

Agenda

  • Memorability vs Object class
  • Memorabiliy vs Saliency
slide-5
SLIDE 5

Memorability and object classes

  • Task:

– Find relation between each individual object class and memorability

  • Features:

– Object Presence – Object Counts – Object Area

slide-6
SLIDE 6

Memorability and object classes

  • Use Spearmen correlation to find

correlation between class features statistics (presence, count, area) and memorability (from ground truth).

slide-7
SLIDE 7
slide-8
SLIDE 8 Object type (all sorted by correlation with Presence)
slide-9
SLIDE 9
slide-10
SLIDE 10

Memorability and object classes

  • Most and Least memorable object classes

are 50/1360 class. (|Correlation| >=0.2)

  • Most of the object classes are not

correlated with memorability, why ?

slide-11
SLIDE 11 Windows (freq: 588) door (freq: 360) Ice field (freq: 4) |Correlation|>=0.2
slide-12
SLIDE 12

Memorability and object classes

  • Most of the object classes are not

correlated with memorability, why ? Because:

– Not frequent enough to find a correlation pattern (dataset limitation) – Very common on all memorability levels – Correlated with medium levels of memorability

slide-13
SLIDE 13

Memorability and object classes

  • What about high correlation with very low

frequency ? (like “ice field”)

slide-14
SLIDE 14
slide-15
SLIDE 15

Filter by frequency

50
slide-16
SLIDE 16

Side effect of filtration by frequency

slide-17
SLIDE 17 High +ve correlation High - ve correlation Low correlation

Correlation after filtration

slide-18
SLIDE 18

Memorability and object classes

  • Most of the object classes are not

correlated with memorability, why ? Because:

– Not frequent enough to find a correlation pattern (filter infrequent images) – Very common on all memorability levels – Correlated with medium levels of memorability

slide-19
SLIDE 19 wall floor person person sitting ceiling shelves chair fluorescent tube screen ceiling lamp bag stand bottle seats box table desk boxes curtain bulletin board sconce cabinet text basket painting picture column pole door flowers staircase bench pot sign steps railing fence window car sidewalk streetlight boat path sand sea road water ground field plant grass mountain building tree sky
slide-20
SLIDE 20 wall floor person person sitting ceiling shelves chair fluorescent tube screen ceiling lamp bag stand bottle seats box table desk boxes curtain bulletin board sconce cabinet text basket painting picture column pole door flowers staircase bench pot sign steps railing fence window car sidewalk streetlight boat path sand sea road water ground field plant grass mountain building tree sky
slide-21
SLIDE 21 wall floor person person sitting ceiling shelves chair fluorescent tube screen ceiling lamp bag stand bottle seats box table desk boxes curtain bulletin board sconce cabinet text basket painting picture column pole door flowers staircase bench pot sign steps railing fence window car sidewalk streetlight boat path sand sea road water ground field plant grass mountain building tree sky
slide-22
SLIDE 22

Agenda

  • Memorability vs Object class
  • Memorabiliy vs Saliency
slide-23
SLIDE 23

Saliency vs Memorability

  • Experiment Design

– Run Saliency algorithm (T. Liu et al. CVPR 2007) on the memorability dataset. – Find correlation between area of salient region and memorability score – Find correlation between location of Salient block and memorability score – Study object classes statistics on salient regions

slide-24
SLIDE 24

Saliency vs Memorability

  • Corr (memorability score, salient region

area) = -3.01%

  • Corr (memorability score, salient region

location) = 4.87%

  • Why ?
slide-25
SLIDE 25

Saliency vs Memorability

High memorability score

slide-26
SLIDE 26

Saliency vs Memorability

Low memorability score

slide-27
SLIDE 27

Saliency and Object classes

  • Compare object class statistics (Presence,

Area) for :

– Full picture – Salient region – Non-Salient region

slide-28
SLIDE 28

Filtered by frequency and sorted by correlation

slide-29
SLIDE 29

Filtered by frequency and sorted by correlation

slide-30
SLIDE 30
slide-31
SLIDE 31 Person, Person setting
slide-32
SLIDE 32 Window
slide-33
SLIDE 33

Conclusion

  • Found a list of object classes sorted by

their relation to memorability

  • Memorability is not directly related to

saliency

  • Saliency is related to specific list of object

classes

slide-34
SLIDE 34

References

  • What Makes an Image Memorable? P. Isola, J.

Xiao, A. Torralba, A. Oliva. CVPR 2011.

  • Learning to Detect a Salient Object. T. Liu et al.

CVPR 2007

  • Matlab functions (http://www.mathworks.

com/matlabcentral/fileexchange/) : lscatter, herrorbar, rotateticklabel

slide-35
SLIDE 35

Thank You