CS381V Paper Presentation Chun-Chen Kuo Selective Search for - - PowerPoint PPT Presentation

▶

Jan 15, 2024 764 likes •1.16k views

CS381V Paper Presentation Chun-Chen Kuo Selective Search for Object Recognition Outline Problem statement Technical details Evaluation Extensions Problem Statement Image Classification Input: training set and

SLIDE 1

CS381V Paper Presentation

Chun-Chen Kuo

SLIDE 2

Selective Search for Object Recognition

SLIDE 3

Outline

Problem statement
Technical details
Evaluation
Extensions

SLIDE 4

Problem Statement

SLIDE 5

Image Classification

Input: training set and test set
Output: the class of the test images and the

confidence scores

ML model

Car (0.8)

image from ImageNet

SLIDE 6

Object Detection

Input: training set

and a test set

Output: all objects in the test images and

their bounding boxes

ML model

Car (0.9)

image from ImageNet

SLIDE 7

How to turn object detection problem to image

classification problem?

ML model

Car (0.9)

image from ImageNet

SLIDE 8

How many sub-regions should we test and how do

we generate them?

ML model

Car (0.9)

image from ImageNet

SLIDE 9

Exhaustive Search

Generate all possible windows
Complexity:

all size all location

image from ImageNet

SLIDE 10

Selective Search

Reduce the number of hypotheses while keep

recall high

Select some high quality hypotheses, which are

subset of all possible hypotheses

SLIDE 11

Technical Details

SLIDE 12

Intuition

Explore image structure and group regions from

small scale to high scale (hierarchical grouping)

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 13

Algorithm

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 14

Similarity Function

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 15

Similarity Function

Color Texture Part

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 16

Similarity Function

Color? Texture? Part?
No single strategy to group regions
Need to diversify by using complementary

similarity measures

SLIDE 17

Similarity Function

Color similarity:
Normalized color histogram with 25 bins:
Propagate through the hierarchy:

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 18

Similarity Function

Texture similarity:
Take Gaussian derivatives in 8 orientations, and

extract histogram with bin size=10:

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 19

Similarity Function

Size similarity:
Merge small regions first

Prevent a big region eating small regions

SLIDE 20

Similarity Function

Fill similarity:

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 21

Similarity Function

Combine them:

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

a=[1,1,1,1] => C+T+S+F a=[0,1,1,1] => T+S+F

SLIDE 22

Complementary Color Space

Also diversify in color space

invariance

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 23

Evaluation

SLIDE 24

Metrics

Average Best Overlap (ABO)

Mean Average Best Overlap (MABO)

mean of ABO over all classes

SLIDE 25

Some Examples

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 26

Flat v.s Hierarchy

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

Hierarchy is good!

SLIDE 27

Diversification Strategies

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

Diversification is good!

SLIDE 28

Compare to Other Methods

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

State of the art!

SLIDE 29

Contribution and Strength

Hierarchical grouping and diversification strategies
Nice trade-off between quality(MABO) and

quantity(# window)

SLIDE 30

Weakness

The algorithm for sorting the object hypotheses s.t.

the most likely hypothesis comes first

No evaluation on it?
Favor large scale but times rand() to prevent over-

favor?

SLIDE 31

Extension

SLIDE 32

R-CNN

Regions with Convolutional Neural Network

features

Rich feature hierarchies for accurate object detection and semantic segmentation. R. Girshick et al. CVPR 2013

SLIDE 33

Visual-Semantic Alignment

A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE Conference on Computer Vision and

Pattern Recognition, pages 3128–3137, 2015.

SLIDE 34

Reference

J. R. Uijlings, K. E. van de Sande, T. Gevers, and A. W.
Smeulders. Selective search for object recognition.

International journal of computer vision, 104(2):154–171, 2013

R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature

hierarchies for accurate object detection and semantic

segmentation. In CVPR, 2014.
A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments

for generating image descriptions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3128–3137, 2015.  

SLIDE 35

Appendix

SLIDE 36

Application on Object Detection

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 37

Diversification Strategies

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013

SLIDE 38

Trade-off between Quality and Quantity

Selective Search for Object Recognition. J. Uijilings, K. van de Sande, T. Gevers, A. Smeulders. IJCV 2013