Relative Attributes by Devi Parikh, Kristen Grauman ICCV2011 - - PowerPoint PPT Presentation

Relative Attributes

by Devi Parikh, Kristen Grauman ICCV2011

Experiment presentation by Wei-Lin Hsiao

Question: How much improvement can we get from instance-level comparison?

long on the leg

>

pointy at the front

>

• pen

>

Experiment setup

• Compare results using instance-level with using

category-level on:

• Image pair order accuracy
• zero-shot learning accuracy

Image pair order accuracy

pointy

• pen

bright

• rnaments

shiny high-heel long on leg formal sporty feminine

pointy Learned ranking using instance comparison

pointy Learned ranking using category comparison

pointy Instance Category

Improvements made by instance comparison

pointy

• pen

bright

• rnaments

shiny high-heel long on leg formal sporty feminine

Instance-level information helps most when there isn’t a clear category ranking within an attribute

Sporty Bright Ornament High-heels

categories the proportion each category takes in the attribute

Zero-shot learning accuracy

Guidance/Dataset Shoes OSR Category 0.3423 0.5391 Instance 0.3826 0.5409

Zero-shot classification Classification on all seen categories

Guidance/Dataset Shoes OSR Category 0.3665 0.6205 Instance 0.3825 0.6846

predicted class

athletic shoes boots clogs flats high heels pumps rainboots sneakers stiletto wedding shoes

true class Category-level, prediction on all seen categories

athletic boots clogs flats high heels pumps rainboots sneakers stiletto wedding

category instance

athletic boots clogs flats high heels pumps rainboots sneakers stiletto wedding athletic boots clogs flats high heels pumps rainboots sneakers stiletto wedding

Classification results when all classes seen

Guidance/Dataset Shoes Category 0.3665 Instance 0.3825

Unable to distinguish athletic/sneaker, boots/rainboots because category comparison too similar

athletic boots clogs flats high heels pumps rainboots sneakers stiletto wedding athletic boots clogs flats high heels pumps rainboots sneakers stiletto wedding athletic boots clogs flats high heels pumps rainboots sneakers stiletto wedding

category instance Classification results when pumps, rain boots unseen

Guidance/Dataset

Shoes

Category

0.3423

Instance

0.3826

Classification results when all categories seen category instance

coast forest highway inside city mountain

• pen country

street tall building

Guidance/Dataset OSR Category 0.6205 Instance 0.6846

Attribute/ Class Coast Forest Highway Inside City Mountai n Open country Street Tall building Natural 1 1 5 6 1 1 6 8 Open 1 7 1 6 4 1 5 7 Perspective 7 5 4 3 6 8 2 1 Large

• bjects

2 8 2 4 6 6 4 1 Diagonal plane 5 8 2 3 6 6 3 1 Close depth 8 1 1 1 7 6 1 5

Classification results when all categories seen

coast forest highway inside city mountain

• pen country

street tall building coast forest highway inside city mountain

• pen country

street tall building

category instance When coast, street unseen

OSR Category 0.5391 Instance 0.5409

c

• a

s t f

• r

e s t h i g h w a y i n s i d e c i t y m

• u

n t a i n

• p

e n c

• u

n t r y s t r e e t t a l l b u i l d i n g c

• a

s t f

• r

e s t h i g h w a y i n s i d e c i t y m

• u

n t a i n

• p

e n c

• u

n t r y s t r e e t t a l l b u i l d i n g

coast forest highway inside city mountain

• pen country

street tall building coast forest highway inside city mountain

• pen country

street tall building coast forest highway inside city mountain

• pen country

street tall building

category instance

all seen coast, street unseen inside city, tall building unseen

Conclusion

• Instance-level information in general helps

classification between two very similar classes.

• Instance-level helps more when the classification

task is more fine-grained.

Demo

http://godel.ece.vt.edu/whittle/ https://filebox.ece.vt.edu/~parikh/demos.htm#whittle_demo