Large-Scale R-CNN with Classifier Adaptive Quantization Redmon et - - PowerPoint PPT Presentation

Mincheul Kang

Large-Scale R-CNN with Classifier Adaptive Quantization

1

Redmon et al., ECCV 2016

Review

2

• Deepfashion: Powering robust clothes recognition

and retrieval with rich annotations – CVPR 16

• Try to solve this problem with rich annotations
• Category and Attribute Annotation
• Landmarks Annotation
• Pair annotation

From Joongun’s slide

Contents

3

• 1. Introduction
• 2. Background
• 3. Related work
• 4. Approach
• 5. Result
• 6. Conclusion

Introduction

4

• Object detection
• Image retrieval, Robotics, Self-driving car

Fast and Accurate

Fast R-CNN slides : Ross Girshick http://www.nvidia.com/object/drive-px.html http://kitschthingoftheday.blogspot.com/2011/06/breakfast-making-robots-at-tum.html

Large-scale

Introduction

5

• Real-time, Large-scale object detection
• Retrieve objects for a certain category from large image

collections immediately and accurately

• Problem : huge costs

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Introduction

6

• Extend R-CNN for large scale
• Classify millions/billions of features in real-

time

• Use the techniques of nearest neighbor search
• Collaboration of two fields

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Background

7

• Inverted index
• Generate a codebook by quantization
• e.g. k-means clustering
• Given a query,
• Find its K closest words
• Retrieve all the data in the K

lists corresponding to the words

From lecture notes

Background

8

• K-means Clustering
• A method of vector quantization
• Minimize the within-cluster sum of squares

http://sanghyukchun.github.io/69/

Background

9

• Product Quantization (PQ)
• Use separate small codebook for each chunk
• Low memory/time cost
• Vector split into m subvectors
• Subvectors are quantized separately by quantizers

http://nick0702.blogspot.kr/2013/03/aggregating-local-descriptors-into.html

=> 64-bit quantization index

Related work

10

• YOLO : Real-time object detection
• Predict all bounding boxes across all classes for an

image simultaneously

• Pascal 2007 (5k) -> 20 classes
• R-CNN, Fast R-CNN, Faster R-CNN

You only look once: Unified, real-time object detection, J Redmon et al., CVPR 2016

69.0

Related work

11

• R-CNN (Region proposals + CNN)
• Selective search
• CNN that extracts a fixed-length feature vector from

each region

• Binary linear SVMs

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, Ross Girshick et al., CVPR 2014

Approach

12

• Framework of Large-scale R-CNN

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Approach

13

• Offline procedure

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

• Detect proposal objects and extract

features

• Similar to R-CNN

Approach

14

• Construct an inverted index

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

• Avoid exhaustive search by an inverted index
• Use the classifier adaptive quantization(CAQ) to learn

codebook instead of k-means

Approach

15

• Limitation of k-means
• It is suitable for nearest neighbor search
• But, It is not optimal for linear classification
• Instead of K-means -> CAQ

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Approach

16

• Classifier Adaptive Quantization (CAQ)
• CAQ is based on the k-means algorithm
• Linear SVM classifier

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Approach

17

• Compress data with residual vector quantization

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Approach

18

• Residual vector quantization (RVQ)
• PQ-based quantization
• Reduce the quantization error
• Learn multiple sub-codebooks one by one by

minimizing the error greedily.

Approximate nearest neighbor search by residual vector quantization, Chen et al., Sensors 10.12 2010

Learning codebooks Quantizing a vector

Residual vectors

Approach

19

• Comparison of PQ, OPQ and RVQ
• Data compression methods on PASCAL VOC 2007

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Approach

20

• Online procedure

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Results

21

• Efficiency over R-CNN
• PASCAL dataset(5K images, 10M features)
• 250x faster, 106x memory reduction with comparable

accuracy

You only look once: Unified, real-time object detection, J Redmon et al., CVPR 2016 Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

GoogLeNet VGGNet-16 69.0

Results

22

• Large-scale dataset PASCAL+Imagenet (~105K

images)

• 105K search in 130ms

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Results

23

• CAQ vs K-means
• CAQ improves by ~10% mAP over K-means

Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Inverted multi index The number of category

Conclusion

24

• Pros
• Compared to R-CNN, 250x speed-up and 106x memory

reduction

• Quickly detect the object on large-scale data.
• Present classifier adaptive quantization instead of k-

means

• Cons
• The accuracy is not high yet.

Q & A

25