Binarized Mode Seeking for Scalable Visual Pattern Discovery Wei - - PowerPoint PPT Presentation

▶

Dec 09, 2022 444 likes •613 views

Binarized Mode Seeking for Scalable Visual Pattern Discovery Wei Zhang, Xiaochun Cao, Rui Wang, Yuanfang Guo, Zhineng Chen http://vireo.cs.cityu.edu.hk/wzhang Background Big Vis isual Data 3,320

SLIDE 1

Wei Zhang, Xiaochun Cao, Rui Wang, Yuanfang Guo, Zhineng Chen

张炜

中国科学院信息工程研究所 http://vireo.cs.cityu.edu.hk/wzhang

Binarized Mode Seeking for Scalable Visual Pattern Discovery

SLIDE 2

Background – Big Vis isual Data

3,320 images uploaded / minute 100 hours videos uploaded / minute 350 million new photos / day 1000 hours of TV News / day

SLIDE 3

Background – Search & Mining

How to manage large volume of data?

Applications: browsing / recommendation / visualization / tagging …
Key techniques: searching and mining

Dataset

frequent items

search mining/discovery

… …

query relevant items 3

SLIDE 4

Visual Pattern Discovery

Given an unsorted data collection (e.g., > 108)

What kind of images are in the dataset?
What’s the difference with other ‘common’

datasets?

SLIDE 5

small patterns, large collection Previous works

Object-level
Optimization - Tan09; Yuan07; Liu10
FIM - Quack08
clustering - Sivic04
hashing: Chum09
ToF: Zhang14/16 – small pattern
Frame level
graph clustering: Philbin08
hashing: Chum10

Zhang17 – large collection

108 107 106 105 104 103 102 101 pattern-scale Tan09 Yuan07 Liu10 Quack08 Sivic04 Chum10 Chum09 Philbin08 data-scale easy hard difficulty

small dataset large dataset

Main Challenge

Zhang14/16 Zhang17

bject-level

frame-level

SLIDE 6

10110100001011 10100011101010 00101100110111 00110110101010 00111111010101 10111000110010 00111011101000 ... ... 10101101001010 10101101001010 00101010101000 ... ...

(a) large image sets (b) binary codes (c) modes (d) patterns

bMS cbMS

10110100001011 10100011101010 00101100110111 00110110101010 00111111010101 00111011101000 ... ... 10101101011010 00101010101000 00101011101000 ... ...

target set contrastive set frequent patterns informative patterns

Zhang17 – large collection

Scalability Issue
Memory: 10^6, FC7 feature  30G
CPU bottleneck
Informative patterns
Frequent
Discriminative

Binarized Mode Seeking for Scalable Visual Pattern Discovery

SLIDE 7

bMS: Binarized Mode Seeking

Binary constraint Binomial-based kernel

SLIDE 8

Results on ILSVRC and Flickr

SLIDE 9

cbMS: Contrastive Binarized Mode Seeking

Contrastive density

Positive set: the target set
Negative set: adopted as reference

SLIDE 10

cbMS: Contrastive Binarized Mode Seeking

Contrastive density

Positive set: the target set
Negative set: a reference set

bMS

SLIDE 11

Evaluation

ILSVRC: 1.3 million images 1000 objects

No significant performance drop
Runs much faster

SLIDE 12

Patterns discovered on ILSVRC

SLIDE 13

Example patterns on Flickr

SLIDE 14

Given a dataset with TOO MANY

images (can’t fit into memory)

What kind of images are in the

dataset?

What’s the difference with other

“common” datasets?

Binarized data / algorithm
50X CPU speedup
30X memory saving

Summary

Binarized Mode Seeking for Scalable Visual Pattern Discovery

Patterns auto-discovered by bMS (left) and cbMS (right) from the unlabeled Flickr10M dataset. Comparison of bMS and cbMS in ILSVRC. Middle: sample images from

ther categories that are visually similar to the left column.

SLIDE 15

Google将在北京招聘更多的人工智能、机器学习技术专家，该团队将与Google Cloud首席科学家李飞飞紧密合作，致力于人工智能、机器学习领域的研究和技术开发。如果您有以下领域相关经验：计算机视觉、视频处理、图像处理、语音识别、自然语言处理、机器学习、深度学习、HCI、数据挖掘等，欢迎申请Research Scientist职位！这是Google首次在华招聘科学家, 机会难得！有任何问题，或者需要内推，可以直接小窗口联系Lu Jiang, 或者将简历发送到 lujiang@cs.cmu.edu. JD链接在此：

SLIDE 16