Relative Attributes by Devi Parikh, Kristen Grauman ICCV2011 Experiment presentation by Wei-Lin Hsiao
Question: How much improvement can we get from instance-level comparison? > pointy at the front > > long on the leg open
Experiment setup • Compare results using instance-level with using category-level on: • Image pair order accuracy • zero-shot learning accuracy
Image pair order accuracy pointy bright shiny long on leg sporty open ornaments high-heel formal feminine
Learned ranking using instance comparison pointy
Learned ranking using category comparison pointy
Instance pointy Category
Improvements made by instance comparison pointy bright shiny long on leg sporty open ornaments high-heel formal feminine
Instance-level information helps most when there isn’t a clear category ranking within an attribute the proportion each category takes in the attribute Ornament Bright High-heels Sporty categories
Zero-shot learning accuracy Zero-shot classification Guidance/Dataset Shoes OSR Category 0.3423 0.5391 Instance 0.3826 0.5409 Classification on all seen categories Guidance/Dataset Shoes OSR Category 0.3665 0.6205 Instance 0.3825 0.6846
Category-level, prediction on all seen categories true class athletic shoes boots clogs flats high heels pumps rainboots sneakers stiletto wedding shoes predicted class
Classification results when all classes seen category instance athletic boots clogs flats high heels pumps rainboots sneakers stiletto wedding high heels athletic boots clogs pumps high heels flats athletic clogs boots pumps rainboots flats sneakers wedding stiletto rainboots sneakers wedding stiletto Guidance/Dataset Shoes Category 0.3665 Instance 0.3825
Unable to distinguish athletic/sneaker, boots/rainboots because category comparison too similar
Classification results when pumps, rain boots unseen category instance athletic boots clogs flats high heels pumps rainboots sneakers stiletto wedding high heels athletic boots clogs pumps high heels flats athletic clogs boots pumps rainboots flats sneakers wedding stiletto rainboots sneakers wedding stiletto Guidance/Dataset Shoes Category 0.3423 Instance 0.3826
Classification results when all categories seen category instance coast forest highway inside city mountain open country street tall building Guidance/Dataset OSR Category 0.6205 Instance 0.6846
Attribute/ Coast Forest Highway Inside Mountai Open Street Tall Class City n country building Natural 1 1 5 6 1 1 6 8 Open 1 7 1 6 4 1 5 7 Perspective 7 5 4 3 6 8 2 1 Large 2 8 2 4 6 6 4 1 objects Diagonal 5 8 2 3 6 6 3 1 plane Close depth 8 1 1 1 7 6 1 5
Classification results when all categories seen category instance coast forest highway inside city mountain open country street tall building n t y i s t a y t a s i t g y c t r e e o n t n a n e r e c u i o w u r d d o t When coast, street unseen f o h s l i m i s c u g n b i n h i e l l a p t o coast forest highway inside city mountain open country street OSR tall building n t y y s i t t a r s a i t y c t t g n e n e o a n u c r e u e Category 0.5391 w o i o d o r d f t h c i m s l s i g u n n i b h e i p l l o a t Instance 0.5409
category instance all seen coast forest highway inside city mountain open country street tall building coast, street coast unseen forest highway inside city mountain open country street tall building inside city, coast forest tall highway building inside city unseen mountain open country street tall building
Conclusion • Instance-level information in general helps classification between two very similar classes. • Instance-level helps more when the classification task is more fine-grained.
Demo http://godel.ece.vt.edu/whittle/ https://filebox.ece.vt.edu/~parikh/demos.htm#whittle_demo
Recommend
More recommend
