CERTH at MediaEval Placing Task 2013 Giorgos Kordopatis 1 , Symeon - - PowerPoint PPT Presentation

certh at mediaeval placing task 2013
SMART_READER_LITE
LIVE PREVIEW

CERTH at MediaEval Placing Task 2013 Giorgos Kordopatis 1 , Symeon - - PowerPoint PPT Presentation

CERTH at MediaEval Placing Task 2013 Giorgos Kordopatis 1 , Symeon Papadopoulos 2 , Eleftherios Spyromitros-Xioufis 2 , Andreas L. Symeonidis 2 and Yiannis Kompatsiaris 2 1 Electrical Engineering Dept., Aristotle University of Thessaloniki, Greece 2


slide-1
SLIDE 1

CERTH at MediaEval Placing Task 2013

Giorgos Kordopatis1, Symeon Papadopoulos 2, Eleftherios Spyromitros-Xioufis 2, Andreas L. Symeonidis 2 and Yiannis Kompatsiaris 2

1 Electrical Engineering Dept., Aristotle University of Thessaloniki, Greece 2 Information Technologies Institute, CERTH, Thessaloniki, Greece

slide-2
SLIDE 2

Summary of our participation

  • 2 tag-based runs

– Build upon the method of (Van Laere et al., ICMR ‘11) and make use

  • f a two-level LDA scheme to filter non-(geo)-discriminative tags
  • 2 visual runs

– A nearest neighbor scheme using VLAD+SURF features

(Spyromitros-Xioufis et al., WIAMIS ‘12) and an efficient indexing

method for very fast retrieval

  • 1 hybrid run

– Uses a simple fallback scheme to combine a tag-based and a visual run

  • No external gazetteers or Internet data!

2

slide-3
SLIDE 3

Placing using tags – Training (1/3)

  • Removal of noisy and irrelevant tags, e.g. “geolat=… ”

3

Filtering Spatial Clustering & local LDAs Global LDA & BoEW Assignment in Areas Similarity Search

Training Prediction

slide-4
SLIDE 4

Placing using tags – Training (2/3)

  • 𝑙-means clustering is applied on the lat-long values of the training

images to create 𝑙=5000 cluster-areas

  • A local topic model is created for the images of each area using

LDA (100/20)

4

Filtering Spatial Clustering & local LDAs Global LDA & BoEW Assignment in Areas Similarity Search

Training Prediction

slide-5
SLIDE 5

Placing using tags – Training (3/3)

  • A global topic model is created for all training

images using LDA (500/50)

  • For each global topic, a topic-area histogram is

computed by counting its frequency per area

  • Top terms of the highest entropy topics are

added in the bag-of-excluded words (BoEW)

5

Filtering Spatial Clustering & local LDAs Global LDA & BoEW Assignment in Areas Similarity Search

Training Prediction

slide-6
SLIDE 6
  • Noisy or BoEW tags are filtered and Jaccard similarity between the

image’s tags and each local topic is computed

  • Each test image 𝑈𝑗 is assigned to the most similar area:

– Area containing the highest similarity topic (tmax) – Area with the highest average (across topics) similarity (tmean) 6

Filtering Spatial Clustering & local LDAs Global LDA & BoEW Assignment in Areas Similarity Search

Training Prediction

Placing using tags – Prediction (1/2)

slide-7
SLIDE 7
  • Having assigned the image to an area:

– 𝑙 most similar images within the area are identified – their center of gravity is used as location estimate 7

Filtering Spatial Clustering & local LDAs Global LDA & BoEW Assignment in Areas Similarity Search

Training Prediction

Placing using tags – Prediction (2/2)

slide-8
SLIDE 8

Placing using visual features

  • We extract optimized VLAD+SURF vectors (Jegou and Chum, ECCV

2012) for all training images

  • Search efficiently with Product Quantization (Jegou et al., TPAMI

2011)

– 14 bytes per image! – 13 hours for querying the index with the full test set!

  • 1st variant – vnn: Assign the location of the most similar image
  • 2nd variant – vclust: Top 𝑙 =20 nearest neighbors are

considered and an incremental spatial clustering algorithm is applied

8

slide-9
SLIDE 9

Runs and Results

Measure tmax vnn tmean vclust hyb

acc(1km) 10.26 0.60 7.82 0.76 10.37 acc(10km) 23.53 0.99 19.86 1.16 23.70 Median error 651 6715 1028 6691 681 9

  • Tag-based runs: tmax and tmean
  • Visual runs: vnn and vclust
  • Hybrid run: hyb, if there is at least one clean tag, tmax is

used, else vnn is used

slide-10
SLIDE 10

Future work

  • A more thorough analysis on the different sources of

error for the proposed scheme

– # of geographical areas – # of topics and terms per topic used in the local and the global LDAs – type and quality of visual features

  • Extension to include additional metadata and external

resources, e.g. the author of an image as an indicator of the image location

10

slide-11
SLIDE 11

The end

This work was supported by the SocialSensor FP7 project

11