Announcements Reminder: Assignment 1 due Feb 19 on Canvas Reminder: - PDF document

2/10/2016 Announcements • Reminder: Assignment 1 due Feb 19 on Canvas • Reminder: Optional CNN/Caffe tutorial on Monday Recognizing object categories Feb 15, 5-7 pm Kristen Grauman • Presentations: UT -Austin • Choose paper, coordinate • Experiment and paper can overlap • Be very mindful of time limit Last time: Recognizing instances Last time: Recognizing instances • 1. Basics in feature extraction: filtering • 2. Invariant local features • 3. Recognizing object instances Recognition via feature Today matching+spatial verification • Intro to categorization problem Pros : • Ef f ective when we are able to f ind reliable f eatures • Object categorization as discriminative classification within clutter • Boosting + fast face detection example • Great results f or matching specif ic instances • Nearest neighbors + scene recognition example Cons : • Support vector machines + pedestrian detection example • Pyramid match kernels, spatial pyramid match • Scaling with number of models • Convolutional neural networks + ImageNet example • Spatial v erif ication as post-processing – not • Some new representations along the way seamless, expensiv e f or large-scale problems • Rectangular filters • Not suited f or category recognition. • GIST • HOG Kristen Grauman 1

2/10/2016 What does recognition involve? Detection: are there people? Fei-Fei Li Activity: What are they doing? Object categorization mountain tree building banner street lamp vendor people Instance recognition Scene and context categorization • outdoor • city Potala • … Palace A particular sign 2

2/10/2016 Attribute recognition Object Categorization • Task Description  “Given a small number of training images of a category, recognize a-priori unknown instances of that category and assi gn Perceptual and Sensory Augmented Computing the correct category label.” • Which categories are feasible visually? Visual Object Recognition Tutorial gray made of fabric “Fido” German dog animal living shepherd being crowded flat K. Grauman, K. Grauman, B B . Leibe . Leibe Visual Object Categories Visual Object Categories • Basic-level categories in humans seem to be defined • Basic Level Categories in human categorization predominantly visually. [Rosch 76, Lakoff 87] • There is evidence that humans (usually) Perceptual and Sensory Augmented Computing Perceptual and Sensory Augmented Computing  The highest level at which category members have similar … start with basic-level categorization perceived shape before doing identification.  The highest level at which a single mental image reflects the animal Visual Object Recognition Tutorial Visual Object Recognition Tutorial entire category  Basic-level categorization is easier Abstract and faster for humans than object  The level at which human subjects are usually fastest at … levels … identification! identifying category members quadruped  How does this transfer to automatic  The first level named and understood by children … classification algorithms?  The highest level at which a person uses similar motor actions Basic level dog cat cow for interaction with category members German Doberman shepherd Individual … … “Fido” level K. Grauman, K. Grauman, B B . Leibe . Leibe K. Grauman, K. Grauman, B B . Leibe . Leibe Challenges: robustness Other Types of Categories • Functional Categories  e.g. chairs = “something you can sit on” Perceptual and Sensory Augmented Computing Visual Object Recognition Tutorial Illumination Object pose Clutter Occlusions Intra-class Viewpoint appearance K. Grauman, K. Grauman, B B . Leibe . Leibe 3

2/10/2016 Challenges: Challenges: complexity context and human experience • Millions of pixels in an image • 30,000 human recognizable object categories • 30+ degrees of freedom in the pose of articulated objects (humans) • Billions of images online • 144K hours of new video on YouTube daily • … • About half of the cerebral cortex in primates is devoted to processing visual information [Felleman and van Essen 1991] Function Dy namics Context cues Video credit: J. Davis Challenges: learning with Evolution of methods minimal supervision More Less • • • Hand-crafted models Hand-crafted features “End -to- end” • • learning of 3D geometry Learned models features and • • Hypothesize and align Data-driven models*,** Generic category recognition: Window-based object detection: recap basic framework Training: 1. Obtain training data 2. Define features • Build/train object model 3. Define classifier – (Choose a representation) Given new image: 1. Slide window Training examples – Learn or f it parameters of model / classif ier 2. Score by classifier • Generate candidates in new image • Score the candidates Car/non-car Classifier Feature extraction Kristen Grauman 4

2/10/2016 Discriminative classifier construction Issues Nearest neighbor Neural networks • What classifier? – Factors in choosing: 10 6 examples LeCun, Bottou, Bengio, Haffner 1998 • Generativ e or discriminativ e model? Shakhnarovich, Viola, Darrell 2003 Rowley , Baluja, Kanade 1998 Berg, Berg, Malik 2005... … • Data resources – how much training data? • How is the labeled data prepared? Conditional Random Fields Support Vector Machines Boosting • Training time allowance • T est time requirements – real-time? • Fit with the representation Guyon, Vapnik Viola, Jones 2001, McCallum, Freitag, Pereira Heisele, Serre, Poggio, Torralba et al. 2004, 2000; Kumar, Hebert 2003 2001,… Opelt et al. 2006,… … Kristen Grauman Kristen Grauman Slide adapted from Antonio Torralba Window-based models: Issues Three landmark case studies • What categories are amenable? – Similar to specific object matching, we expect spatial lay out to be f airly rigidly preserv ed. – Unlike specific object matching , by training classif iers we attempt to capture intra-class v ariation Boosting + f ace SVM + person NN + scene Gist or determine required discriminativ e f eatures. detection detection classif ication e.g., Hays & Efros e.g., Dalal & Triggs Viola & Jones Kristen Grauman Boosting intuition Viola-Jones face detector Main idea: – Represent local texture with ef f iciently computable “rectangular” f eatures within window of interest Weak Classifier 1 – Select discriminativ e f eatures to be weak classif iers – Use boosted combination of them as f inal classif ier – Form a cascade of such classif iers, rejecting clear negativ es quickly Kristen Grauman Slide credit: Paul Viola 5

2/10/2016 Boosting illustration Boosting illustration Weights Increased Weak Classifier 2 Boosting: pros and cons Boosting: training • Advantages of boosting • Integrates classification with feature selection • Initially, weight each training example equally • Complexity of training is linear in the number of training • examples In each boosting round: • Flexibility in the choice of weak learners, boosting scheme – Find the weak learner that achieves the lowest weighted training error • Testing is fast – Raise weights of training examples misclassified by current weak learner • Easy to implement • Compute f inal classif ier as linear combination of all weak learners (weight of each learner is directly proportional to • Disadvantages its accuracy ) • Needs many training examples • Often found not to work as well as an alternative discriminative classifier, support vector machine (SVM) • Exact f ormulas f or re-weighting and combining weak – especially for many-class problems learners depend on the particular boosting scheme (e.g., AdaBoost) Slide credit: Lana Lazebnik Slide credit: Lana La ze bn ik Computing sum within a rectangle Viola-Jones detector: features • Let A,B,C,D be the values of the integral “ Rectangular” filters image at the corners of a Feature output is dif f erence between D B rectangle adjacent regions • Then the sum of original image values w ithin the Value at (x,y) is rectangle can be A C Ef f iciently computable sum of pixels above and to the computed as: with integral image: any left of (x,y) sum = A – B – C + D sum can be computed in • Only 3 additions are constant time. required for any size of rectangle! Integral image Kristen Grauman Lana Lazebnik 6