Feature Detection ] Logistics Write the use of free late days - - PowerPoint PPT Presentation

▶
feature detection
SMART_READER_LITE
LIVE PREVIEW

Feature Detection ] Logistics Write the use of free late days - - PowerPoint PPT Presentation

May 04, 2023 369 likes •1.04k views

Feature Detection ] Logistics Write the use of free late days right below the title. We only grade the latest submission. Regrading requests are allowed 2 weeks after the grade release. Wednesday lecture from 9:50am 10:40am

slide-1
SLIDE 1

Feature Detection

]

slide-2
SLIDE 2

Logistics

  • Write the use of free late days right below the title.
  • We only grade the latest submission.
  • Regrading requests are allowed 2 weeks after the grade

release.

  • Wednesday lecture from 9:50am – 10:40am this week.
  • Office hours
slide-3
SLIDE 3
  • Why you want to find features.
  • Harris corner detector.
  • Multi-scale detection.
  • Multi-scale blob detection.

Overview of today’s lecture

classical approach before ConvNets

slide-4
SLIDE 4

Planar object instance recognition

Database of planar objects Instance recognition

slide-5
SLIDE 5

3D object recognition

Database of 3D objects 3D objects recognition

slide-6
SLIDE 6

Recognition under occlusion

slide-7
SLIDE 7

Location Recognition

slide-8
SLIDE 8

Robot Localization

slide-9
SLIDE 9

Image matching

slide-10
SLIDE 10

NASA Mars Rover images

Where are the corresponding points?

slide-11
SLIDE 11
slide-12
SLIDE 12

Challenges: Invariance

Find features that are invariant to transformations

  • geometric invariance: translation, rotation, scale
  • photometric invariance: brightness, exposure, …
slide-13
SLIDE 13

Two Problems for Features

Feature detection

slide-14
SLIDE 14

Two Problems for Features

Feature detection Feature descriptor

slide-15
SLIDE 15

Two Problems for Features

Feature detection Feature descriptor

slide-16
SLIDE 16

How do you solve feature detection & matching in CNN

slide-17
SLIDE 17

How do you solve feature detection & matching in CNN

CNN CNN CNN CNN Yes No Yes No CNN No CNN Yes

slide-18
SLIDE 18

What makes a good feature?

Zoom-in demo

slide-19
SLIDE 19

Want uniqueness

Look for unusual image regions

  • Lead to unambiguous matches in other images

How to define “unusual”?

slide-20
SLIDE 20

Local measures of uniqueness

Consider a small window of pixels

  • Where are features good and bad?

Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.

slide-21
SLIDE 21

Local measures of uniqueness

Consider a small window of pixels

  • Where are features good and bad?

Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.

slide-22
SLIDE 22

Feature detection

“flat” region: no change in all directions “edge”: no change along the edge direction “corner”: significant change in all directions

Uniqueness = How does it change when shifted by a small amount?

Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.

slide-23
SLIDE 23

Feature detection

Define

E(u,v) = amount of change when you shift the window by (u,v) E(u,v) is small for all shifts E(u,v) is small for some shifts E(u,v) is small for no shifts

We want to be ______

slide-24
SLIDE 24

Consider shifting the window W by (u,v)

  • how do the pixels in W change?
  • compare each pixel before and after by

Sum of the Squared Differences (SSD)

  • this defines an SSD “error” E(u,v):

Feature detection: the math

W

slide-25
SLIDE 25

Taylor Series expansion of I: If the motion (u,v) is small, then first order approx is good Plugging this into the formula on the previous slide…

Small motion assumption

slide-26
SLIDE 26

Consider shifting the window W by (u,v)

  • how do the pixels in W change?
  • compare each pixel before and after by

summing up the squared differences

  • this defines an “error” of E(u,v):

Feature detection: the math

W

slide-27
SLIDE 27

Feature detection: the math

This can be rewritten:

slide-28
SLIDE 28

Feature detection: the math

This can be rewritten: Which [u v] maximizes E(u,v)? Which [u v] minimizes E(u,v)?

slide-29
SLIDE 29

Feature detection: the math

This can be rewritten: Which [u v] maximizes E(u,v)? Which [u v] minimizes E(u,v)?

slide-30
SLIDE 30

Feature detection: the math

This can be rewritten: x- x+ Eigenvector with the largest eigen value? Eigenvector with the smallest eigen value? x- x+

slide-31
SLIDE 31

Quick eigenvalue/eigenvector review

The eigenvectors of a matrix A are the vectors x that satisfy: The scalar λ is the eigenvalue corresponding to x

  • The eigenvalues are found by solving:
  • In our case, A = H is a 2x2 matrix, so we have
  • The solution:
slide-32
SLIDE 32

Feature detection

Local measure of feature uniqueness

  • E(u,v) = amount of change when you shift the window by (u,v)

E(u,v) is small for all shifts E(u,v) is small for some shifts E(u,v) is small for no shifts

We want to be large =

slide-33
SLIDE 33

Eigenvalues of H

? ?

slide-34
SLIDE 34

Eigenvalues of H

slide-35
SLIDE 35

Feature detection summary

Here’s what you do

  • Compute the gradient at each point in the image
  • Create the H matrix from the entries in the gradient
  • Compute the eigenvalues.
  • Find points with large response (λ- > threshold)
  • Choose those points where λ- is a local maximum as features
slide-36
SLIDE 36

Feature detection summary

Here’s what you do

  • Compute the gradient at each point in the image
  • Create the H matrix from the entries in the gradient
  • Compute the eigenvalues.
  • Find points with large response (λ- > threshold)
  • Choose those points where λ- is a local maximum as features

Called “non-local max suppression”

slide-37
SLIDE 37

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

0.03 0.02

f

Flat

slide-38
SLIDE 38

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

0.03 0.02 0.012

f

Flat

slide-39
SLIDE 39

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

0.03 0.02 0.012 3 0.02 0.02

f

Flat ?

slide-40
SLIDE 40

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

0.03 0.02 0.012 3 0.02 0.02

f

Flat Edge

slide-41
SLIDE 41

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

0.03 0.02 0.012 3 0.02 0.02 2.5 3 1.36

f

Flat Edge ?

slide-42
SLIDE 42

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

0.03 0.02 0.012 3 0.02 0.02 2.5 3 1.36

f

Flat Edge Corner

slide-43
SLIDE 43

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

0.03 0.02 0.012 3 0.02 0.02 2.5 3 1.36 5 6 2.73

f

Flat Edge Corner ?

slide-44
SLIDE 44

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

0.03 0.02 0.012 3 0.02 0.02 2.5 3 1.36 5 6 2.73

f

Flat Edge Corner Strong corner

slide-45
SLIDE 45

The Harris operator

λ- is a variant of the “Harris operator” for feature detection

  • The trace is the sum of the diagonals, i.e., trace(H) = h11 + h22
  • Very similar to λ- but less expensive (no square root)
  • Called the “Harris Corner Detector” or “Harris Operator”
  • Lots of other detectors, this is one of the most popular
slide-46
SLIDE 46

The Harris operator

Harris

  • perator
slide-47
SLIDE 47

Harris detector example

slide-48
SLIDE 48

f value (red high, blue low)

slide-49
SLIDE 49

Threshold (f > value)

slide-50
SLIDE 50

Find local maxima of f

slide-51
SLIDE 51

Harris features (in red)

slide-52
SLIDE 52

Harris corner response is invariant to rotation

Ellipse rotates but its shape (eigenvalues) remains the same Corner response R is invariant to image rotation

slide-53
SLIDE 53

Partial invariance to affine intensity change

  • Only derivatives are used => invariance to intensity

shift I → I + b

  • Intensity scale: I → a I

R

x (image coordinate) threshold

R

x (image coordinate)

Harris corner response is invariant to intensity changes

slide-54
SLIDE 54

The Harris detector is not invariant to changes in …

slide-55
SLIDE 55

The Harris corner detector is not invariant to scale

edge! corner!

slide-56
SLIDE 56

Multi-scale detection

slide-57
SLIDE 57

Scale invariant detection

Suppose you’re looking for corners Key idea: find scale that gives local maximum of f

  • f is a local maximum in both position and scale
slide-58
SLIDE 58

Slide from Tinne Tuytelaars

Lindeberg et al, 1996

Slide from Tinne Tuytelaars

Lindeberg et al., 1996

slide-59
SLIDE 59
slide-60
SLIDE 60
slide-61
SLIDE 61
slide-62
SLIDE 62
slide-63
SLIDE 63
slide-64
SLIDE 64
slide-65
SLIDE 65

Basic reading:

  • Szeliski textbook, Sections 4.1.

References