SLIDE 1 6.S093 Visual Recognition through Machine Learning Competition
Image by kirkh.deviantart.com
Aditya Khosla
SLIDE 2 Today’s class
- Part 1: Competition details
- Part 2: Image representation lecture
– Bag-of-words – Spatial pyramid
- Part 3: Feature extraction tutorial
SLIDE 3
Competition details: dataset
person
10 object categories
airplane bicycle car cup/mug dog(s) guitar hamburger sofa traffic light
SLIDE 4
Competition details: dataset Training set
8,000 images
Validation set
2,000 images
Testing set
5,000 images
labels provided NO labels provided
Leaderboard set
SLIDE 5 Competition details: submission
- For each image, you provide the probability of every
class belonging in it (as returned by your algorithm) airplane bicycle car cup dog guitar hamburger sofa traffic light person 1
SLIDE 6 Competition details: evaluation
SLIDE 7
Competition details: prizes Cash
first
+ cash
second third
+ cash
SLIDE 8
Competition details: thank you!
SLIDE 9
Image representation: bag-of-words
SLIDE 10 Document representation: bag-of-words
- Order-less document representation: frequencies
- f words from a dictionary Salton & McGill (1983)
SLIDE 11 Document representation: bag-of-words
- Order-less document representation: frequencies
- f words from a dictionary Salton & McGill (1983)
US Presidential Speeches Tag Cloud
SLIDE 12 Document representation: bag-of-words
- Order-less document representation: frequencies
- f words from a dictionary Salton & McGill (1983)
US Presidential Speeches Tag Cloud
SLIDE 13 Document representation: bag-of-words
- Order-less document representation: frequencies
- f words from a dictionary Salton & McGill (1983)
US Presidential Speeches Tag Cloud
SLIDE 14 Image representation: bag-of-words
document bag-of-words
SLIDE 15 Image representation: bag-of-words
document bag-of-words image bag-of-visual words
SLIDE 16
Object Bag of ‘words’
SLIDE 17
Object Ugly bag of ‘words’
SLIDE 18
Object Stylish bag of ‘words’
SLIDE 19
Object Stylish bag of ‘words’
SLIDE 20
visual dictionary
SLIDE 21 Image representation: bag-of-words
SLIDE 22 Image representation: bag-of-words
- 1. Extract descriptors
- 2. Learn “visual dictionary”
SLIDE 23 Image representation: bag-of-words
- 1. Extract descriptors
- 2. Learn “visual dictionary”
- 3. Quantize features using visual vocabulary
SLIDE 24 Image representation: bag-of-words
- 1. Extract descriptors
- 2. Learn “visual dictionary”
- 3. Quantize features using visual vocabulary
SLIDE 25 Image representation: bag-of-words
- 1. Extract descriptors
- 2. Learn “visual dictionary”
- 3. Quantize features using visual vocabulary
- 4. Represent images by frequencies of “visual words”
SLIDE 26
- 1. Extracting descriptors
regular grid interest points
SLIDE 27 Image representation: yesterday
gradient magnitude gradient orientation feature vector
SLIDE 28 Image representation: yesterday
gradient magnitude gradient orientation descriptor
SLIDE 29
- 2. Learning “visual dictionary”
Compute descriptor
SLIDE 30
- 2. Learning “visual dictionary”
descriptors
…
SLIDE 31
- 2. Learning visual dictionary
descriptors
…
SLIDE 32
- 2. Learning visual dictionary
descriptors
…
Clustering
SLIDE 33
- 2. Learning visual dictionary
descriptors
…
Clustering visual vocabulary
SLIDE 34 Example visual vocabulary
Fei-Fei et al. 2005
SLIDE 35 Image patch examples
Sivic et al. 2005
SLIDE 36 Image patch examples
Sivic et al. 2005
How to choose the vocabulary size?
SLIDE 37 Bag-of-words: limitations
- What about the structure of the image?
=?
SLIDE 38 Image representation: spatial pyramids
level 0
SLIDE 39 Image representation: spatial pyramids
level 0 level 1
SLIDE 40 Image representation: spatial pyramids
level 0 level 1 level 2
SLIDE 41
Tutorial
