Overview Introduction to local features Harris interest points + - - PowerPoint PPT Presentation

▶
overview
SMART_READER_LITE
LIVE PREVIEW

Overview Introduction to local features Harris interest points + - - PowerPoint PPT Presentation

Nov 12, 2022 236 likes •626 views

Overview Introduction to local features Harris interest points + SSD, ZNCC, SIFT Scale & affine invariant interest point detectors Scale & affine invariant interest point detectors Evaluation and comparison of

slide-1
SLIDE 1

Overview

  • Introduction to local features
  • Harris interest points + SSD, ZNCC, SIFT
  • Scale & affine invariant interest point detectors
  • Scale & affine invariant interest point detectors
  • Evaluation and comparison of different detectors
  • Region descriptors and their performance
slide-2
SLIDE 2

Affine invariant regions - Motivation

  • Scale invariance is not sufficient for large baseline changes

A

detected scale invariant region

A

projected regions, viewpoint changes can locally be approximated by an affine transformation A

slide-3
SLIDE 3

Affine invariant regions - Motivation

slide-4
SLIDE 4

Affine invariant regions - Example

slide-5
SLIDE 5

Harris/Hessian/Laplacian-Affine

  • Initialize with scale-invariant Harris/Hessian/Laplacian

points

  • Estimation of the affine neighbourhood with the second

moment matrix [Lindeberg’94]

  • Apply affine neighbourhood estimation to the scale-

invariant interest points [Mikolajczyk & Schmid’02, Schaffalitzky & Zisserman’02]

  • Excellent results in a comparison [Mikolajczyk et al.’05]
slide-6
SLIDE 6

Affine invariant regions

  • Based on the second moment matrix (Lindeberg’94)

        ⊗ = = ) , ( ) , ( ) , ( ) , ( ) ( ) , , (

2 2 2 D y D y x D y x D x I D D I

L L L L L L G M σ σ σ σ σ σ σ σ µ x x x x x

x x

2 1

M = ′

  • Normalization with eigenvalues/eigenvectors
slide-7
SLIDE 7

Affine invariant regions

L R

x x A =

′ = ′

L R

Rx x

Isotropic neighborhoods related by image rotation

L 2 1 L

x x

L

M = ′

R 2 1 R

x x

R

M = ′

slide-8
SLIDE 8
  • Iterative estimation – initial points

Affine invariant regions - Estimation

slide-9
SLIDE 9
  • Iterative estimation – iteration #1

Affine invariant regions - Estimation

slide-10
SLIDE 10
  • Iterative estimation – iteration #2

Affine invariant regions - Estimation

slide-11
SLIDE 11
  • Iterative estimation – iteration #3, #4

Affine invariant regions - Estimation

slide-12
SLIDE 12

Harris-Affine versus Harris-Laplace

Harris-Laplace Harris-Affine

slide-13
SLIDE 13

Harris-Affine

Harris/Hessian-Affine

Hessian-Affine

slide-14
SLIDE 14

Harris-Affine

slide-15
SLIDE 15

Hessian-Affine

slide-16
SLIDE 16

Matches

22 correct matches

slide-17
SLIDE 17

Matches

33 correct matches

slide-18
SLIDE 18

Maximally stable extremal regions (MSER) [Matas’02]

  • Based on the idea of region

segmentation

  • State of the art results
slide-19
SLIDE 19

Maximally stable extremal regions (MSER) [Matas’02]

  • Extremal regions: connected components in a thresholded

image (all pixels above/below a threshold)

  • Maximally stable: minimal change of the component

(area) for a change of the threshold, i.e. region remains (area) for a change of the threshold, i.e. region remains stable for a change of threshold

slide-20
SLIDE 20

Maximally stable extremal regions (MSER) Examples of thresholded images

high threshold low threshold

slide-21
SLIDE 21

MSER

slide-22
SLIDE 22

Overview

  • Introduction to local features
  • Harris interest points + SSD, ZNCC, SIFT
  • Scale & affine invariant interest point detectors
  • Scale & affine invariant interest point detectors
  • Evaluation and comparison of different detectors
  • Region descriptors and their performance
slide-23
SLIDE 23

Evaluation of interest points

  • Quantitative evaluation of interest point/region detectors

– points / regions at the same relative location and area

  • Repeatability rate : percentage of corresponding points
  • Two points/regions are corresponding if

– location error small – area intersection large

  • [K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas,
  • F. Schaffalitzky, T. Kadir & L. Van Gool ’05]
slide-24
SLIDE 24

Evaluation criterion

H

% 100 # # ⋅ = regions detected regions ing correspond ity repeatabil

slide-25
SLIDE 25

Evaluation criterion

H

% 100 # # ⋅ = regions detected regions ing correspond ity repeatabil

% 100 ) 1 ( ⋅ − = union

  • n

intersecti error

  • verlap

2% 10% 20% 30% 40% 50% 60%

slide-26
SLIDE 26

Comparison of affine invariant detectors

60 70 80 90 100

repeatability %

Harris−Affine Hessian−Affine MSER IBR EBR Salient

Viewpoint change - structured scene

repeatability %

15 20 25 30 35 40 45 50 55 60 65 10 20 30 40 50 60

viewpoint angle repeatability %

reference image 20 60 40

slide-27
SLIDE 27

Scale change – textured scene

repeatability %

Comparison of affine invariant detectors

reference image 4

slide-28
SLIDE 28
  • Good performance for large viewpoint and scale changes
  • Results depend on transformation and scene type, no one best

detector

Conclusion - detectors

  • Detectors are complementary

– MSER adapted to structured scenes – Harris and Hessian adapted to textured scenes

  • Performance of the different scale invariant detectors is very similar

(Harris-Laplace, Hessian, LoG and DOG)

  • Scale-invariant detector sufficient up to 40 degrees of viewpoint

change

slide-29
SLIDE 29

Overview

  • Introduction to local features
  • Harris interest points + SSD, ZNCC, SIFT
  • Scale & affine invariant interest point detectors
  • Scale & affine invariant interest point detectors
  • Evaluation and comparison of different detectors
  • Region descriptors and their performance
slide-30
SLIDE 30

Region descriptors

  • Normalized regions are

– invariant to geometric transformations except rotation – not invariant to photometric transformations

slide-31
SLIDE 31

Descriptors

  • Regions invariant to geometric transformations except

rotation

– rotation invariant descriptors – normalization with dominant gradient direction – normalization with dominant gradient direction

  • Regions not invariant to photometric transformations

– invariance to affine photometric transformations – normalization with mean and standard deviation of the image patch

slide-32
SLIDE 32

Descriptors

Extract affine regions Normalize regions Eliminate rotational + illumination Compute appearance descriptors SIFT (Lowe ’04)

slide-33
SLIDE 33

Descriptors

  • Gaussian derivative-based descriptors

– Differential invariants (Koenderink and van Doorn’87) – Steerable filters (Freeman and Adelson’91)

  • SIFT (Lowe’99)
  • SIFT (Lowe’99)
  • Moment invariants [Van Gool et al.’96]
  • Shape context [Belongie et al.’02]
  • SIFT with PCA dimensionality reduction
  • Gradient PCA [Ke and Sukthankar’04]
  • SURF descriptor [Bay et al.’08]
  • DAISY descriptor [Tola et al.’08, Windler et al’09]
slide-34
SLIDE 34

Comparison criterion

  • Descriptors should be

– Distinctive – Robust to changes on viewing conditions as well as to errors of the detector

  • Detection rate (recall)

1

  • Detection rate (recall)

– #correct matches / #correspondences

  • False positive rate

– #false matches / #all matches

  • Variation of the distance threshold

– distance (d1, d2) < threshold

1 1

[K. Mikolajczyk & C. Schmid, PAMI’05]

slide-35
SLIDE 35

Viewpoint change (60 degrees)

0.6 0.7 0.8 0.9 1

#correct / 2101

esift

* *

shape context gradient pca cross correlation complex filters har−aff esift steerable filters gradient moments sift

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6

1−precision #correct / 2101

slide-36
SLIDE 36

esift

* *

Scale change (factor 2.8)

0.6 0.7 0.8 0.9 1

#correct / 2086

shape context gradient pca cross correlation complex filters har−aff esift steerable filters gradient moments sift

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6

1−precision #correct / 2086

slide-37
SLIDE 37

Conclusion - descriptors

  • SIFT based descriptors perform best
  • Significant difference between SIFT and low dimension

descriptors as well as cross-correlation

  • Robust region descriptors better than point-wise

descriptors

  • Performance of the descriptor is relatively independent of

the detector

slide-38
SLIDE 38

Available on the internet

  • Binaries for detectors and descriptors

– Building blocks for recognition systems

  • Carefully designed test setup
  • Carefully designed test setup

– Dataset with transformations – Evaluation code in matlab – Benchmark for new detectors and descriptors http://lear.inrialpes.fr/software