Fixing WTFs - Detecting Image Matches caused by Watermarks, - - PowerPoint PPT Presentation

▶
fixing wtfs detecting image matches caused by watermarks
SMART_READER_LITE
LIVE PREVIEW

Fixing WTFs - Detecting Image Matches caused by Watermarks, - - PowerPoint PPT Presentation

Jan 16, 2024 431 likes •821 views

Fixing WTFs - Detecting Image Matches caused by Watermarks, Timestamps, and Frames in Internet Photos Tobias Weyand, Chih-Yun Tsai, Bastian Leibe Fixing WTFs Poster No. 26 Overview Many Computer Vision Applications use Internet photos,

slide-1
SLIDE 1
slide-2
SLIDE 2

Fixing WTFs - Detecting Image Matches caused by Watermarks, Timestamps, and Frames in Internet Photos

Tobias Weyand, Chih-Yun Tsai, Bastian Leibe
slide-3
SLIDE 3 Poster No. 26 Fixing WTFs
  • Many Computer Vision Applications use Internet photos, e.g.
  • Image Retrieval, Image Clustering and Structure from Motion
  • Internet photos increasingly contain Watermarks, Timestamps, or Frames (WTFs) that
harm these applications.
  • We propose a simple, effective and fast method to detect WTFs during matching.
  • Code and dataset are publicly available at: tiny.cc/wtf
Overview
slide-4
SLIDE 4 Poster No. 26 Fixing WTFs WTFs in Image Matching
slide-5
SLIDE 5 Poster No. 26 Fixing WTFs WTFs in Image Matching Invalid matches
slide-6
SLIDE 6 Poster No. 26 Fixing WTFs WTFs in Image Matching Invalid matches
slide-7
SLIDE 7 Poster No. 26 Fixing WTFs WTFs in Image Matching Invalid matches

√

slide-8
SLIDE 8 Poster No. 26 Fixing WTFs WTFs in Image Matching
slide-9
SLIDE 9 Poster No. 26 Fixing WTFs WTFs in Image Matching Invalid matches
slide-10
SLIDE 10 Poster No. 26 Fixing WTFs WTFs in Image Matching Invalid matches
slide-11
SLIDE 11 Poster No. 26 Fixing WTFs WTFs in Image Matching Invalid matches

X

slide-12
SLIDE 12 Poster No. 26 Fixing WTFs WTFs in Image Retrieval Query Image
slide-13
SLIDE 13 Poster No. 26 Fixing WTFs WTFs in Image Retrieval Query Image
slide-14
SLIDE 14 Poster No. 26 Fixing WTFs WTFs in Image Retrieval Query Image
slide-15
SLIDE 15 Poster No. 26 Fixing WTFs WTFs in Image Retrieval Query Image Results
slide-16
SLIDE 16 Poster No. 26 Fixing WTFs WTFs in Image Retrieval Query Image Results

X X X X

slide-17
SLIDE 17 Poster No. 26 Fixing WTFs WTFs in Image Clustering
slide-18
SLIDE 18 Poster No. 26 Fixing WTFs WTFs in Image Clustering
slide-19
SLIDE 19 Poster No. 26 Fixing WTFs WTFs in Image Clustering
slide-20
SLIDE 20 Poster No. 26 Fixing WTFs WTFs in Image Clustering
slide-21
SLIDE 21 Poster No. 26 Fixing WTFs WTFs in Image Clustering Pseudo-Clusters
slide-22
SLIDE 22 Poster No. 26 Fixing WTFs Method
slide-23
SLIDE 23 Poster No. 26 Fixing WTFs Method Key assumptions:
 WTFs have similar appearance and occur in certain image positions.
slide-24
SLIDE 24 Poster No. 26 Fixing WTFs Method Key assumptions:
 WTFs have similar appearance and occur in certain image positions. Input Image Pair with feature matches
slide-25
SLIDE 25 Poster No. 26 Fixing WTFs Method Input Image Pair with feature matches Similarity Map Key assumptions:
 WTFs have similar appearance and occur in certain image positions.
slide-26
SLIDE 26 Poster No. 26 Fixing WTFs Method Input Image Pair with feature matches Similarity Map Spatial Histogram Key assumptions:
 WTFs have similar appearance and occur in certain image positions.
slide-27
SLIDE 27 Poster No. 26 Fixing WTFs Method “WTF” Input Image Pair with feature matches Similarity Map Spatial Histogram Classifier Decision Key assumptions:
 WTFs have similar appearance and occur in certain image positions.
slide-28
SLIDE 28 Poster No. 26 Fixing WTFs Method “WTF” Input Image Pair with feature matches Similarity Map Spatial Histogram Classifier Decision Key assumptions:
 WTFs have similar appearance and occur in certain image positions.
slide-29
SLIDE 29 Poster No. 26 Fixing WTFs Method “WTF” Input Image Pair with feature matches Similarity Map Spatial Histogram Classifier Decision Key assumptions:
 WTFs have similar appearance and occur in certain image positions.
slide-30
SLIDE 30 Poster No. 26 Fixing WTFs Method “WTF” Input Image Pair with feature matches Similarity Map Spatial Histogram Classifier Decision Key assumptions:
 WTFs have similar appearance and occur in certain image positions.
slide-31
SLIDE 31 Poster No. 26 Fixing WTFs Method “WTF” Input Image Pair with feature matches Similarity Map Spatial Histogram “non- WTF” Classifier Decision Key assumptions:
 WTFs have similar appearance and occur in certain image positions.
slide-32
SLIDE 32 Poster No. 26 Fixing WTFs Dataset
  • 36,240 image pairs from Flickr and Panoramio
  • 10% WTFs, 90% non-WTFs
  • Publicly available at: tiny.cc/wtf

… …

Non-WTFs WTFs
slide-33
SLIDE 33 Poster No. 26 Fixing WTFs Results False Positive Rate 0.2 0.4 0.6 0.8 1 True Positive Rate 0.2 0.4 0.6 0.8 1 Our Method (0.03 f99, 0.998 AUC) GPS (0.96 f99, 0.499 AUC) GPS+Heuristic (0.96 f99, 0.865 AUC) 3% False-positive rate @ 99% True positive rate
slide-34
SLIDE 34 Poster No. 26 Fixing WTFs Clustering results Clusters with multiple objects were split.

X X X X X X X X X X

slide-35
SLIDE 35 Poster No. 26 Fixing WTFs Clustering results Pseudo-clusters were removed.

X X X

slide-36
SLIDE 36 Poster No. 26 Fixing WTFs Clustering results Polluted clusters were cleaned.

X X X X X

slide-37
SLIDE 37 Poster No. 26 Fixing WTFs Conclusion
  • WTF matches harm many vision applications.
  • We propose a simple, fast and effective detector for them.
  • Our code is open source and easy to integrate: tiny.cc/wtf
slide-38
SLIDE 38

Come visit us at poster 26!

Get the code and dataset at tiny.cc/wtf