Corners and Interest Points Various slides from previous courses by: - - PowerPoint PPT Presentation

CS4501: Introduction to Computer Vision

Corners and Interest Points

Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays (Brown / Georgia Tech), A. Berg (Stony Brook / UNC), D. Samaras (Stony Brook) . J. M. Frahm (UNC), V. Ordonez (UVA).

• Frequency Domain
• Filtering in Frequency
• Edge Detection - Canny

Last Class

• Corner Detection - Harris
• Interest Points
• Local Feature Descriptors

Today’s Class

Edge Detection

Edge Detection

• Sobel + Thresholding

! ", $ = &1, ((", $) ≥ , 0, ( ", $ < ,

Digression

• Thresholding is often the most under-rated but effective Computer Vision
• technique. Sometimes this is all you need!

Example by Kristen Grauman, UT-Austin

Example: intensity-based detection. Warning when door is

• pened or closed.

Looking for dark pixels

fg_pix = find(im < 65);

Canny Edge Detector

Source: Juan C. Niebles and Ranjay Krishna.

• Obtains thin edges (1px wide)
• Tries to preserve edge connectivity.

Canny edge detector

• 1. Filter image with x, y derivatives of Gaussian
• 2. Find magnitude and orientation of gradient
• 3. Non-maximum suppression:
• Thin multi-pixel wide ridges down to single pixel width
• 4. Thresholding and linking (hysteresis):
• Define two thresholds: low and high
• Use the high threshold to start edge curves and the low threshold to

continue them

Source: Juan C. Niebles and Ranjay Krishna.

Canny Edge Detector

• Classic algorithm in Computer Vision / Image Analysis
• Commonly implemented in most libraries
• e.g. in Python you can find it in the skimage package.

OpenCV also has an implementation with python bindings.

Canny, J., A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8(6):679–698, 1986.

Corners (and Interest Points)

• How to find corners?
• What is a corner?

Corners: Why “Interest” Points?

Example from Silvio Savarese

If we can find them, then maybe we can “match” images belonging to the same object! Then maybe we can also “triangulate” the 3D coordinates of the image.

Corner Detection: Basic Idea

• We should easily recognize the point by looking through a small window
• Shifting a window in any direction should give a large change in intensity

“edge”: no change along the edge direction “corner”: significant change in all directions “flat” region: no change in all directions

Source: A. Efros

Harris Corner Detection

• Compute the following matrix of squared gradients for every pixel.

! = # $%

&

$%$' $'$% $'

& ()*+,

$% $' and are gradients computed using Sobel or some other approximation. ! = 0 ! = 0 . ! = / ! = / .

Harris Corner Detection

! = 0 ! = 0 $ ! = % ! = % $

• If both a, and b are large then this is a corner, otherwise it is not. Set

a threshold and this should detect corners.

Problem: Doesn’t work for these corners:

Harris Corner Detection

! = # $%

&

$%$' $'$% $'

&

= ( ) ) *

+,-./

! = 01

23 43

4& 01 Under a rotation M can be diagonalized 43 and 4& are the eigenvalues of M det ! − 4$ = 0 From your linear algebra class finding them requires solving

However no need to solve det(M-lambda I)=0

1 2 1 2

det trace M M l l l l = = +

Corner response function

“Corner” R > 0 “Edge” R < 0 “Edge” R < 0 “Flat” region |R| small

2 2 1 2 1 2

) ( ) ( trace ) det( l l a l l a +

• =
• =

M M R

α: constant (0.04 to 0.06)

Harris Detector Summary [Harris88]

• Second moment matrix

ú ú û ù ê ê ë é * = ) ( ) ( ) ( ) ( ) ( ) , (

2 2 D y D y x D y x D x I D I

I I I I I I g s s s s s s s µ

18

• 1. Image

derivatives

• 2. Square of

derivatives

• 3. Gaussian

filter g(sI)

Ix Iy Ix

2

Iy

2

IxIy g(Ix2) g(Iy2) g(IxIy)

2 2 2 2 2 2

)] ( ) ( [ )] ( [ ) ( ) (

y x y x y x

I g I g I I g I g I g +

• a

=

• =

] )) , ( [trace( )] , ( det[

2 D I D I

har s s µ a s s µ

• 4. Cornerness function – both eigenvalues are strong

har

• 5. Non-maxima suppression

1 2 1 2

det trace M M l l l l = = +

(optionally, blur first)

Alternative Corner response function

“flat” region l1 and l2 are small; “edge”: l1 >> l2 l2 >> l1 “corner”: l1 and l2 are large, l1 ~ l2;

1 2 1 2

det trace M M l l l l = = +

Szeliski Harmonic mean

Harris Detector: Steps

Harris Detector: Steps

Compute corner response R

Harris Detector: Steps

Find points with large corner response: R>threshold

Harris Detector: Steps

Take only the points of local maxima of R

Harris Detector: Steps

Invariance and covariance

• We want corner locations to be invariant to photometric

transformations and covariant to geometric transformations

• Invariance: image is transformed and corner locations do not

change

• Covariance: if we have two transformed versions of the same

image, features should be detected in corresponding locations

Slide by James Hays

Keypoint detection with scale selection

• We want to extract keypoints with characteristic

scale that is covariant with the image transformation

Slide by Svetlana Lazebnik

Basic idea

• Convolve the image with a “blob filter” at multiple

scales and look for extrema of filter response in the resulting scale space

• T. Lindeberg. Feature detection with automatic scale selection.

IJCV 30(2), pp 77-116, 1998.

Slide by Svetlana Lazebnik

Blob detection

• Find maxima and minima of blob filter response in

space and scale

*

=

maxima minima

Source: N. Snavely

Blob filter

• Laplacian of Gaussian: Circularly symmetric operator for blob

detection in 2D

2 2 2 2 2

y g x g g ¶ ¶ + ¶ ¶ = Ñ

Recall: Edge detection

g dx d f * f

g dx d

Source: S. Seitz

Edge Derivative

• f Gaussian

Edge = maximum

• f derivative

Edge detection, Take 2

g dx d f

2 2

* f

g dx d

2 2

Edge Second derivative

• f Gaussian

(Laplacian) Edge = zero crossing

• f second derivative

Source: S. Seitz

Scale-space blob detector

• 1. Convolve image with scale-normalized Laplacian at several scales

Scale-space blob detector: Example

Slide by Svetlana Lazebnik

Scale-space blob detector: Example

Slide by Svetlana Lazebnik

Scale-space blob detector

• 1. Convolve image with scale-normalized Laplacian at several scales
• 2. Find maxima of squared Laplacian response in scale-space

Slide by Svetlana Lazebnik

Scale-space blob detector: Example

Slide by Svetlana Lazebnik

Eliminating edge responses

• Laplacian has strong response along edge

Slide by Svetlana Lazebnik

• Approximating the Laplacian with a difference of Gaussians:

( )

2

( , , ) ( , , )

xx yy

L G x y G x y s s s = +

( , , ) ( , , ) DoG G x y k G x y s s =

• (Laplacian)

(Difference of Gaussians)

Efficient implementation

Slide by Svetlana Lazebnik

Efficient implementation

David G. Lowe. "Distinctive image features from scale-invariant keypoints.” IJCV 60 (2), pp. 91-110, 2004.

SIFT keypoint detection

• D. Lowe, Distinctive image features from scale-invariant keypoints, IJCV 60 (2),
• pp. 91-110, 2004.

Questions?

41