Current View of Recognition Training Appearance Object Appearance Examples Representation Representation Model LAB Histogram x x x x x oo Textons oo x x o o HOG x x x Bag of SIFT A. Farhadi, I. Endres, and D. Hoiem 2010
Current View of Recognition g Training Appearance Object Appearance Examples Representation Representation Model LAB Histogram x x x x x oo Textons oo x x o o HOG x x x Bag of SIFT Lots of effort – fancy stuff A. Farhadi, I. Endres, and D. Hoiem 2010
Current View of Recognition Training Appearance Object Appearance Examples Representation Representation Model LAB Histogram x x x x x oo Textons oo x x o o HOG x x x Bag of SIFT Not much changed A. Farhadi, I. Endres, and D. Hoiem 2010
Value of basic categories Has head Is animal Is furry DOG Is small Can be pet Eats meat A. Farhadi, I. Endres, and D. Hoiem 2010
Limitations of basic categories They provide limited prediction and description DOG DOG A. Farhadi, I. Endres, and D. Hoiem 2010
Limitations of basic categories g They do not apply to objects from novel categories y pp y j g Familiar Objects New Object ??? Horse Dog Cat A. Farhadi, I. Endres, and D. Hoiem 2010
Limitations of basic categories g They do not make it easier to learn new categories y g Dog Appearance Classifier Features Appearance Zebra Features Classifier
Category-based representation • Limited description and prediction • No generalization to objects outside of learned categories g • Provides little guidance for learning So what would make a better So what would make a better representation? A. Farhadi, I. Endres, and D. Hoiem 2010
Attribute-based Representation Learn intermediate structure with object categories Multiple Categories ears fur animal, land animal, …, cat Viewpoint/pose eyes lying down, left side, facing camera mouth th F Function ti fast runner, climb trees, eat small tail animals, jump high, household pet scratch pet, scratch feet A. Farhadi, I. Endres, and D. Hoiem 2010
What we mean by attributes • Properties that we want to describe or predict • Shared across basic categories • Made explicit through supervision Multiple Categories ears fur animal, land animal, …, cat Viewpoint/pose eyes lying down, left side, facing camera mouth th F Function ti fast runner, climb trees, eat small tail animals, jump high, household pet scratch pet, scratch feet A. Farhadi, I. Endres, and D. Hoiem 2010
What do these attributes get us? Image Level Contains donkey Detailed Attributes Level Categories Animal Land animal d l Mammal Four legged animal Elk Pose Lying down = 1 Back = 1 … Object Level Object Level Functional Can see Horse Horse Can walk Herbivorous … Material Pixel segmentations A. Farhadi, I. Endres, and D. Hoiem 2010
Advantages of supervised attributes • Enables verbal description of objects and images p j g Large angry dog with pointy teeth A. Farhadi, I. Endres, and D. Hoiem 2010
Advantages of supervised attributes • Provides correspondence for objects from different categories categories STANDING HEAD HEAD SITTING LEG LEG LEG HEAD STANDING LEG A. Farhadi, I. Endres, and D. Hoiem 2010
Domain-based Recognition Basic-Level Superordinate Parts Parts Categories Categories … Cat Dog Detector Detector Head 4-Legged Animal Detector D t Detector t A. Farhadi, I. Endres, and D. Hoiem 2010
Domain-based Recognition Cat Detector 4-Legged Animal gg D Dog Detector Head 4-Legged Animal Detector D t t Head Walking Left Detector A. Farhadi, I. Endres, and D. Hoiem 2010
Domain-based recognition: overview Voting using Voting using Trained Detectors Shared Spatial Models Animal Vehicle Basic Level Categories Elephant, Dog, Eagle, Object Camel, Lizard, Bat, Localization Dog, Penguin, Monkey, … Broad Categories Four-legged Animal, Attribute ib Mammal, Water Animal, Animal Four-legged Predictors Animal Mammal Head Can run Can run Parts Object Can Jump Leg Leg, Horn, Wing, Head, Eye, Description Is Herbivorous Facing right Ear, Foot, Mouth, Nose, Tail A. Farhadi, I. Endres, and D. Hoiem 2010
CORE Dataset C ross-category O bject RE cognition • 2780 Images – from ImageNet 2780 I f I N • 3192 Objects – 28 Categories • 26695 Parts – 71 types • 30046 Attributes – 34 types • 1052 Material Images – 10 types Download or browse online: http://vision.cs.uiuc.edu/CORE http://vision.cs.uiuc.edu/CORE A. Farhadi, I. Endres, and D. Hoiem 2010
CORE Dataset Annotation Example Mirrors Vehicle Gas tank Two-wheeled Motorcycle Seat Headlight Lic. Plate Motorcycle Facing right Tail light On the street Metal Exhaust Has a rider Has a rider Rubber Rubber Engine Wheel Wheel A. Farhadi, I. Endres, and D. Hoiem 2010
Dataset examples: animals Categories Seen During Training and Testing Categories Seen Only During Testing A. Farhadi, I. Endres, and D. Hoiem 2010
Dataset examples: vehicles Categories Seen Only Categories Seen Only Categories Seen During Training and Testing During Testing A. Farhadi, I. Endres, and D. Hoiem 2010
Result: Part detectors can generalize across categories Part Detections for Novel Object Hump Head Leg Detectors trained using (Felzenszwalb Girshik McAllester Ramanan 2009) method
Result: Broad category detectors can generalize across basic categories Category Detections for Novel Object Four-legged Animal Mammal Animal Mammal Detectors trained using (Felzenszwalb Girshik McAllester Ramanan 2009) method
describe objects from familiar categories i Trunk u Trunk Leg Leg Foot Foot Foot
describe objects from familiar categories i ROC for Localization of Familiar Objects A. Farhadi, I. Endres, and D. Hoiem 2010
describe objects from familiar categories i AUC for Attribute Prediction for Familiar Objects Baseline: Infer from Basic Categories Our Method: Infer from All Animals Vehicles 1 1 0,9 0 9 0 9 0,9 0,8 0,8 0,7 0,7 0,6 0,6 0,5 0,5 Has Part Has Part Basic Basic Broad Function Broad Function Pose Pose Has Part Basic Cat Broad Function Has Part Basic Cat Broad Function Pose Pose Cat Cat Cat A. Farhadi, I. Endres, and D. Hoiem 2010
Result using only basic categories Elk Semi Truck Eagle Camel Snowmobile Dog A. Farhadi, I. Endres, and D. Hoiem 2010
Result 3: We can find and describe objects from novel categories Four-legged Animal Animal Mammal a a Vehicle V hi l Head Wheel Leg Can run Can Jump Is Herbivorous b Moves on road Facing right Facing right A. Farhadi, I. Endres, and D. Hoiem 2010
Result 3: We can find and describe objects from novel categories ROC for Localization of Unfamiliar Objects A. Farhadi, I. Endres, and D. Hoiem 2010
Result 3: We can find and describe objects from novel categories AUC for Attribute Prediction for Unfamiliar Objects Baseline: Infer from Basic Categories Our Method: Infer from All Animals Vehicles 0,8 0,8 0,7 0,7 0 6 0,6 0,6 0 6 0,5 0,5 Has Part Broad d Function Pose Has Part Broad d Function Pose Cat Cat
Recommend
More recommend
