Deep Learning of Binary Hash Codes for Fast Image Retrieval Kevin - - PowerPoint PPT Presentation

Deep Learning of Binary Hash Codes for Fast Image Retrieval

Kevin Lin, Huei-Fang Yang, Jen-Hao Hsiao, Chu-song chen Yahoo! Taiwan CVPR 2015

• 2016. 11. 6.

박중언

1

Index

• Review
• Background & Motivation
• Method
• Experiment & Result
• Q & A
• Quiz

2

Review

3

Review - Video Object Segmentation

4 http://sglab.kaist.ac.kr/~sungeui/IR/Presentation/first_2016/%EC%A3%BC%EC%84%B8%ED%98%84.pdf

Review - Video Object Segmentation

5 http://sglab.kaist.ac.kr/~sungeui/IR/Presentation/first_2016/%EC%A3%BC%EC%84%B8%ED%98%84.pdf

Background & Motivation

6

Background - Inverted Index

7

• Reduce search space effectively with agreeable loss of

accuracy

• Use those ANN techniques for efficiently finding near clusters

http://sglab.kaist.ac.kr/~sungeui/IR/Slides2016/Lec4b-bow.pdf

Motivation

• Need fast retrieval within huge amount of image data sets.
• Need to generate the binary compact codes directly from the

deep CNN.

8

Motivation

• Consider characteristic of CNN layer depth
• Feature from deep layer
• similar appearance
• Feature from shallow layer
• similar High-level semantics

9

Method

10

Method

• The method includes

three main component consist of 3 steps.

• Pre-Training
• Fine-Tuning
• Hierarchical Search

11

Method – pre-training

• Supervised pre-training on the large-scale ImageNet dataset
• > 1M images, > 1000 categories
• Trained with Alexnet

12

Method – fine-tuning

• Fine-tuning the network with the latent layer to

simultaneously learn domain specific feature representation and a set of hash-like function

13

Method – fine-tuning

• The weights of the latent layer H and the final classification

layer F8 are randomly initialized.

• The initial random weights of latent layer H acts like LSH [6] which uses random

projections for constructing the hashing bits

• A. Gionis, P

. Indyk, R. Motwani, et al. Similarity search in high dimensions via hashing. In VLDB, volume 99, pages 518–529, 1999. 1, 2, 4, 6

14

Method – fine-tuning

• latent layer H are activated by sigmoid functions so the

activations are approximated to {0,1}.

• Sigmoid function
• To achieve domain adaptation, fine-tune the proposed

network on the target-domain dataset via back propagation.

15

Method – image retrieval

• Retrieves images similar to the query one via the hierarchical

deep search

• Hierarchical search has two steps.
• First, it finds nearest n coarse feature with Coarse-level search
• Second, fine Fine-level search for candidates belong to the coarse

feature.

16

Method – image retrieval

• Similarity level in coarse-level search is as the Hamming distance

17

Method – image retrieval

18

Experiment & Result

19

Experiment

• Supervised pre-learning on ImageNet
• Fine-tuning on target domain
• MNIST, CIFAR-10
• Image Retrieval via Hierarchy deep search

20

Experiment

• Experiment has done in MNIST, CIFAR-10, and Yahoo-1M

dataset

• Define precision@k to measure performance
• (number of ground truth images in top k) / k

21

Experiment - MNIST

• F8 to 10 way, 10 object categories, and h is also set as 48

and 128.

• 50,000 training iterations

22

Experiment - MNIST

• Classification performed for 1000 images on training set(left) ,

test set(right)

23

Experiment – CIFAR-10

• CIFAR-10, F8 to10 way, 10 object categories, and h is also set

as 48 and 128.

24

Experiment – CIFAR-10

• Classification performed for 1000 images on training set(left) ,

test set(right)

25

Experiment – Yahoo! 1M dataset

• 116 object categories, and h in the latent layer to 128
• randomly select 1000 images

26

Experiment – Yahoo! 1M dataset

• (1) AlexNet: F7 feature from the pre-trained CNN [14];
• (2) Ours-ES: F7 features from our network;
• (3) Ours-BCS: Latent binary codes from our network;
• (4) Ours-HDS: F7 features and latent binary codes from our

network.

27

Result – Yahoo! 1M dataset

28

• Classification performed for 1000 images on Training

set(left) , test set(right)

Result - Speed

• 971.3x faster than traditional exhaustive search with 4096-

dimensional features.

29

Conclusion

• Introducing a simple, yet effective supervised learning

framework for rapid image retrieval.

• Suggested CNN techniques that learns domain specific

image representations and a set of hashing-like functions for rapid image retrieval.

• The proposed method outperforms all of the state of-the-art

works on the public dataset

• Our approach learns binary hashing codes in a pointwised

manner and is easily scalable to the data size in comparison

• f conventional pair-wised approaches

30

Q & A

31

Thank you!

32