SLIDE 1
System Overview
Video Category correlation Sequential context
Audio Images ASR TFIDF MFCC GB Feature extraction SVM GMM SVM Primary systems Source combination 1-best selection Higher-level systems
Detecting Categories in News Video Using Image Features Slav - - PowerPoint PPT Presentation
Detecting Categories in News Video Using Image Features Slav - - PowerPoint PPT Presentation
Detecting Categories in News Video Using Image Features Slav Petrov, Arlo Faria, Pascal Michaillat, Alex Berg, Andreas Stolcke, Dan Klein, Jitendra Malik System Overview Images GB SVM Category correlation Sequential context Source Video
SLIDE 2
SLIDE 3
Image Features in TrecVid ’05
- IBM:
- Color Histogram
- Color Correlogram
- Color Moments
- CMU (local):
- Color Histograms (in different color spaces)
- Texture Histograms
- Columbia (part based model):
- Color
- Texture
- Tsinghua (local and global):
- Color Auto-Correlograms
- Color Coherence Vectors
- Color Histograms
- Co-occurence Texture
- Wavelet Texture Grid
- Edge Histogram Layout
- Edge Histograms
- Size
- Spatial Relation
- Color Moments
- Edge Histograms
- Wavelet Texture
SLIDE 4
Image Features in TrecVid ’05
✓
Shape
✓ ✓ ✓
Edge Histograms
✓
Wavelets
✓ ✓ ✓
Histograms
Texture ✓ ✓
Correlograms
✓ ✓
Moments
✓ ✓ ✓ ✓
Histograms
Color
Berkeley Tsinghua Columbia CMU IBM
SLIDE 5
Exemplars for Recognition
- Use exemplars for recognition
- Compare query image and each exemplar
using shape cues
Query Image Database of Exemplars
SLIDE 6
Finding similar patches
Exemplar Query
SLIDE 7
Compute sparse channels from image
Geometric Blur (Local Appearance Descriptor)
Geometric Blur Descriptor
~
Apply spatially varying blur and sub-sample Extract a patch in each channel
Idealized signal Descriptor is robust to small affine distortions
[Berg & Malik, CVPR’01]
SLIDE 8
Horizontal Channel Vertical Channel Increasing Blur
GB in Practice
- In practice compute discrete blur levels for whole
image and sample as needed for each feature location.
SLIDE 9
Comparing Images
- Sample 200 GB features from edge points
- Dissimilarity from A to B is
where the Fx are the GB features.
[Berg, Berg & Malik, CVPR’05]
SLIDE 10
Caltech 101 Dataset
- Object Recognition Benchmark
- 101 Categories:
- Stereotypical pose
- Little clutter
- Objects centered
- One object per image
SLIDE 11
Caltech 101 Results
[Zhang, Berg, Maire & Malik, CVPR’06]
uses GB features
SLIDE 12
Primal features for SVM
- Compare to 50 prototypes from each class
- Use distances as feature vector for an SVM
…… .. .. ..
Query Prototype s Featur e Vector
0.7 0.9
…
0.1
…
0.8
… ……….
0.7
SLIDE 13
SVM features interpretation
- Slices of the Kernel
Matrix:
- Fixed-points in a
higher dimensional vector space:
q q ti ti tj tk tk tj
SLIDE 14
SVM Specifics
- SVMlight package
- Same parameters for all categories:
- Linear kernel
- Default regularization parameter
- Asymmetric cost doubling the weight of
positive examples
SLIDE 15
Results ’06
Sports Meeting Computer- TV-Screen Car
mAP = 0.11
Results ’05 Berkeley-Shape mAP = 0.38 Best ’05 (IBM) mAP = 0.34
Best Berkeley-Shape Median
SLIDE 16
Limitations
- Several objects per image:
- Features do not capture:
- Different Scales
- Color
SLIDE 17
Conclusions
- Shape is an important cue for object
recognition.
- System that uses shape features only
can have competitive performance.
- Shape features are orthogonal to
features used in the past.
SLIDE 18