SLIDE 1 Introduction to Recognition
Computer Vision CS 543 / ECE 549 University of Illinois
Many Slides from D. Hoiem, L. Lazebnik.
SLIDE 2 Outline
- Overview of image and region categorization
– Task description – What is a category
- Example of spatial pyramids bag-of-words scene
categorizer
- Key concepts: features and classification
- Deep convolutional neural networks (CNNs)
SLIDE 3 Recognition as 3D Matching
Recognizing solid objects by alignment with an image. Huttenlocher and Ullman IJCV 1990.
“Instance” Recognition “Category-level” Recognition
SLIDE 4 Detection, semantic segmentation, instance segmentation
semantic segmentation instance segmentation image classification
Image source
SLIDE 5
“Classic” recognition pipeline
Feature representation Trainable classifier Image Pixels Class label
SLIDE 6 Overview
Training Labels Training Images Classifier Training
Training
Image Features Image Features
Testing
Test Image
Trained Classifier
Trained Classifier Outdoor Prediction
SLIDE 7 Classifiers: Nearest neighbor
f(x) = label of the training example nearest to x
- All we need is a distance or similarity function for our inputs
- No training required!
Test example Training examples from class 1 Training examples from class 2
SLIDE 8 K-nearest neighbor classifier
- Which classifier is more robust to outliers?
Credit: Andrej Karpathy, http://cs231n.github.io/classification/
SLIDE 9 Linear classifiers
- Find a linear function to separate the classes:
f(x) = sgn(w × x + b)
SLIDE 10
- Linearly separable dataset in 1D:
- Non-separable dataset in 1D:
- We can map the data to a higher-dimensional space:
x x x x2
Nonlinear SVMs
Slide credit: Andrew Moore
SLIDE 11
Bag of features
1. Extract local features 2. Learn “visual vocabulary” 3. Quantize local features using visual vocabulary 4. Represent images by frequencies of “visual words”
SLIDE 12
Digit Classification Case Study
SLIDE 13 The MNIST DATABASE of handwritten digits
Yann LeCun & Corinna Cortes
- Has a training set of 60 K
examples (6K examples for each digit), and a test set of 10K examples.
- Each digit is a 28 x 28 pixel grey
level image. The digit itself
- ccupies the central 20 x 20
pixels, and the center of mass lies at the center of the box.
SLIDE 14 Bias-Variance Trade-off
10
2
10
3
10
4
10
5
5 10 15 20 25 30 35 Number of Training Examples Error Rate Performance on MNIST Dataset Gradient, Int Gradient, Linear Raw, Poly Raw, Rbf
SLIDE 15
Bias and Variance
SLIDE 16
Bias-Variance Trade-off
Performance as a function of model complexity (SVM)
SLIDE 17
Model Selection
SLIDE 18
Bias-Variance Trade-off
As a function of dataset size
SLIDE 19 Generalization Error
Testing Training Number of Training Examples Error Generalization Error Fixed classifier
SLIDE 20 Features vs Classifiers
10
2
10
3
10
4
10
5
5 10 15 20 25 30 35 Number of Training Examples Error Rate Performance on MNIST Dataset Gradient, Int Gradient, Linear Raw, Poly Raw, Rbf
SLIDE 21 What are the right features?
Depend on what you want to know!
– Local shape info, shading, shadows, texture
– linear perspective, gradients, line segments
- Material properties: albedo, feel, hardness
– Color, texture
– Optical flow, tracked points
SLIDE 22 Stuff vs Objects
- recognizing cloth fabric vs recognizing cups
SLIDE 23 Feature Design Process
- 1. Start with a model
- 2. Look at errors on development set
- 3. Think of features that can improve
performance
- 4. Develop new model, test whether new
features help.
- 5. If not happy, go to step 1.
- 6. “Ablations”: Simplify system, prune out
features that don’t help anymore in presence
SLIDE 24 Features vs Classifiers
10
2
10
3
10
4
10
5
5 10 15 20 25 30 35 Number of Training Examples Error Rate Performance on MNIST Dataset Gradient, Int Gradient, Linear Raw, Poly Raw, Rbf
SLIDE 25
“Classic” recognition pipeline
Feature representation Trainable classifier Image Pixels Class label
SLIDE 26
Categorization involves features and a classifier
Training Labels Training Images Classifier Training
Training
Image Features Image Features
Testing
Test Image Trained Classifier Outdoor Prediction Trained Classifier
SLIDE 27 New training setup with moderate sized datasets
Training Labels Training Images Tune CNN features and Neural Network classifier
Trained Classifier
Dataset similar to task with millions of labeled examples
Initialize CNN Features
SLIDE 28
Categorization involves features and a classifier
Training Labels Training Images Classifier Training
Training
Image Features Image Features
Testing
Test Image Trained Classifier Outdoor Prediction Trained Classifier
SLIDE 29 New training setup with moderate sized datasets
Training Labels Training Images Tune CNN features and Neural Network classifier
Trained Classifier
Dataset similar to task with millions of labeled examples
Initialize CNN Features
