Tour the World: building a webscale landmark recognition engine 1 - - PowerPoint PPT Presentation

▶
tour the world building a webscale landmark recognition
SMART_READER_LITE
LIVE PREVIEW

Tour the World: building a webscale landmark recognition engine 1 - - PowerPoint PPT Presentation

Oct 20, 2022 321 likes •566 views

Tour the World: building a webscale landmark recognition engine 1 Y a n - T a o Z h e n g , M i n g Z h a o , Y a n g S o n g , H a r t w i g A d a m , U l r i c h B u d d e m e i e r , A l e s s a n d r o B i s s a c c o , F e r n a

slide-1
SLIDE 1

Y a n - T a o Z h e n g , M i n g Z h a o , Y a n g S o n g , H a r t w i g A d a m , U l r i c h B u d d e m e i e r , A l e s s a n d r o B i s s a c c o , F e r n a n d o B r u c h e r , T a t - S e n g C h u a , a n d H a r t m u t N e v e n C V P R 2 0 0 9

Tour the World: building a webscale landmark recognition engine

P r e s e n t e r : C a n s ı n Y ı l d ı z

03.12.2009

1

Cansın Yıldız

slide-2
SLIDE 2

Introduction

03.12.2009

Cansın Yıldız

2

slide-3
SLIDE 3

Problems

03.12.2009

Cansın Yıldız

3

 Discovering Landmarks in the World  Mining True Landmark Images  Efficiency

slide-4
SLIDE 4

Problems

03.12.2009

Cansın Yıldız

4

 Discovering Landmarks in the World

 Two complementary sources:  GEO-tagged photos from picasa.google.com  Travel guide articles from wikitravel.com

 Mining True Landmark Images  Efficiency

slide-5
SLIDE 5

Problems

03.12.2009

Cansın Yıldız

5

 Discovering Landmarks in the World  Mining True Landmark Images

 Visual clustering on the noisy image set  Further cleaning of clusters

 Efficiency

slide-6
SLIDE 6

Problems

03.12.2009

Cansın Yıldız

6

 Discovering Landmarks in the World  Mining True Landmark Images  Efficiency

 Parallel computing of landmark models  Efficient clustering algorithm  Efficient image matching

slide-7
SLIDE 7

Overview

03.12.2009

Cansın Yıldız

7

slide-8
SLIDE 8

GPS Tagged Photos

03.12.2009

Cansın Yıldız

8

  • 1. Learning landmarks from GPS-tagged photos

1.

slide-9
SLIDE 9

GPS Tagged Photos

03.12.2009

Cansın Yıldız

9

1.Geo-cluster

 Create clusters (I1) based on GPS coordinates  Delete clusters with not enough unique authors

 ~140k geo-clusters, ~14k visual clusters, 2240 landmarks

slide-10
SLIDE 10

Travel Guide Articles

03.12.2009

Cansın Yıldız

10

  • 2. Learning landmarks from travel guide articles

2.

slide-11
SLIDE 11

Travel Guide Articles

03.12.2009

Cansın Yıldız

11

1.Download wikitravel articles of every city on earth 2.Mine landmark names from articles if,

 Text is within Section “See” or “To See”  Text is within a bullet list.  Text is written in bold.

3.Retrieve landmark images (I2) from google image search

 7315 landmark candidates, 3246 landmarks

slide-12
SLIDE 12

Noisy Landmark Image Set

03.12.2009

Cansın Yıldız

12

geo cluster1 geo clusterm

… I1 …

visual cluster1 visual clusterk

… …

landmark images1

… I2 …

visual cluster1 visual clusterz

… …

landmark imagesn

Visual Model Noisy Image Set

visual clustering

slide-13
SLIDE 13

Visual Clustering and Cleaning

03.12.2009

Cansın Yıldız

13

  • 3. Visual Clustering and Cleaning

3.

slide-14
SLIDE 14

Visual Clustering and Cleaning

03.12.2009

Cansın Yıldız

14

Perform visual clustering for each set in set I= I1 + I2

  • 1. Object matching based on local features

2.Constructing match region graph 3.Graph clustering on match regions 4.Cleaning visual model

slide-15
SLIDE 15

Visual Clustering and Cleaning

03.12.2009

Cansın Yıldız

15

1.Object matching based on local features

 Use LOG filters to detect interest points  Use SIFT for local descriptors  Get match score and match region

slide-16
SLIDE 16

Visual Clustering and Cleaning

03.12.2009

Cansın Yıldız

16

2.Constructing match region graph

slide-17
SLIDE 17

Visual Clustering and Cleaning

03.12.2009

Cansın Yıldız

17

3.Graph clustering on match regions

 Lack of priori knowledge, can’t use k-means  Use hierarchical agglomerative clustering instead.

slide-18
SLIDE 18

Visual Clustering and Cleaning

03.12.2009

Cansın Yıldız

18

4.Cleaning visual model

 Clean clusters having map images using photographic vs. non-

photographic image classifier.

 Clean clusters having not enough number of authors.  Clean clusters having images dominated by people using

multi-view face detector.

slide-19
SLIDE 19

Visual Clustering and Cleaning

03.12.2009

Cansın Yıldız

19

Sample Visual Cluster

slide-20
SLIDE 20

Results - Distribution

03.12.2009

Cansın Yıldız

20

Distribution of Recognized Landmarks

slide-21
SLIDE 21

Results – True Positives

03.12.2009

Cansın Yıldız

21

Examples of True Positives

slide-22
SLIDE 22

Results – False Positives

03.12.2009

Cansın Yıldız

22

Landmarks can be locally visually similar Regions in landmark model can be non-representative Negative images and landmark model images can be similar

slide-23
SLIDE 23

Related Work

03.12.2009

Cansın Yıldız

23

 I know what you did last summer: object-level auto-

annotation of holiday snaps, T. Quack et al., ICCV 2009

slide-24
SLIDE 24

Q U E S T I O N S ?

Cansın Yıldız

03.12.2009

24

Thank You