Lecture 2 Local Interest Point Detectors Lin ZHANG, PhD School of - PowerPoint PPT Presentation

Lecture 2 Local Interest Point Detectors Lin ZHANG, PhD School of Software Engineering Tongji University Spring 2020 Lin ZHANG, SSE, 2020

Content • Local Invariant Features • Motivation • Requirements • Invariance • Harris Corner Detector • Scale Invariant Point Detection • Automatic scale selection • Laplacian‐of‐Gaussian detector • Difference‐of‐Gaussian detector Lin ZHANG, SSE, 2020

Motivation Lin ZHANG, SSE, 2020

Motivation Application: Image Matching Lin ZHANG, SSE, 2020

Motivation Application: Image Matching Lin ZHANG, SSE, 2020

Motivation Application: Image Matching NASA Mars Rover Images Lin ZHANG, SSE, 2020

Motivation Application: Image Matching (Look for tiny colored squares) NASA Mars Rover images with SIFT matches Lin ZHANG, SSE, 2020

Motivation • Panorama stitching • We have two images – how do we combine them? Lin ZHANG, SSE, 2020

Motivation • Panorama stitching • We have two images – how do we combine them? Lin ZHANG, SSE, 2020

Motivation • Panorama stitching • We have two images – how do we combine them? Lin ZHANG, SSE, 2020

Motivation • Panorama stitching • We have two images – how do we combine them? Lin ZHANG, SSE, 2020

General Approach for Image Matching Source: B. Leibe Lin ZHANG, SSE, 2020

Characteristics of Good Features • Repeatability • The same feature can be found in several images despite geometric and photometric transformations • Saliency • Each feature has a distinctive description • Compactness and efficiency • Many fewer features than image pixels • Locality • A feature occupies a relatively small area of the image; robust to clutter and occlusion Lin ZHANG, SSE, 2020

Invariance: Geometric Transformations Lin ZHANG, SSE, 2020

Level of Geometric Invariance Lin ZHANG, SSE, 2020

Invariance: Photometric Transformations Lin ZHANG, SSE, 2020

Applications Feature points are used for: • Motion tracking • Image alignment • 3D reconstruction • Object recognition • Indexing and database retrieval • Robot navigation Lin ZHANG, SSE, 2020

Content • Local Invariant Features • Motivation • Requirements • Invariance • Harris Corner Detector • Scale Invariant Point Detection • Automatic scale selection • Laplacian‐of‐Gaussian detector • Difference‐of‐Gaussian detector Lin ZHANG, SSE, 2020

Finding Corners My office, 5:30PM, Sep. 18, 2011 Lin ZHANG, SSE, 2020

Finding Corners • Key property: in the region around a corner, image gradient has two or more dominant directions • Corners are repeatable and distinctive C. Harris and M. Stephens. “A Combined Corner and Edge Detector.“ Proceedings of the 4th Alvey Vision Conference : pages 147—151, 1988. Lin ZHANG, SSE, 2020

Corner Detection: Basic Idea • We should easily recognize the point by looking through a small window • Shifting a window in any direction should give a large change in intensity “ flat” region: “ edge”: “ corner”: no change in no change along significant change all directions the edge in all directions direction Lin ZHANG, SSE, 2020

Harris Detector: Basic Idea + ‐ > 0 Difference = 3 Demo of a point + with well distinguished neighborhood. Moving the window in any direction will result in a large intensity change. Lin ZHANG, SSE, 2020

Harris Detector: Basic Idea + - > 0 Difference = 2 Demo of a point + with well distinguished neighborhood. Moving the window in any direction will result in a large intensity change. Lin ZHANG, SSE, 2020

Harris Detector: Basic Idea + - > 0 Difference = 5 Demo of a point + with well distinguished neighborhood. Moving the window in any direction will result in a large intensity change. Lin ZHANG, SSE, 2020

Harris Detector: Basic Idea + - > 0 Difference = 2 Demo of a point + with well distinguished neighborhood. Moving the window in any direction will result in a large intensity change. Lin ZHANG, SSE, 2020

Harris Detector: Basic Idea + - > 0 Difference = 3 Demo of a point + with well distinguished neighborhood. Moving the window in any direction will result in a large intensity change. Lin ZHANG, SSE, 2020

Harris Detector: Basic Idea + - > 0 Difference = 2 Demo of a point + with well distinguished neighborhood. Moving the window in any direction will result in a large intensity change. Lin ZHANG, SSE, 2020

Harris Detector: Basic Idea + - > 0 Difference = 3 Demo of a point + with well distinguished neighborhood. Moving the window in any direction will result in a large intensity change. Lin ZHANG, SSE, 2020

Harris Detector: Basic Idea + - > 0 Difference = 2 Demo of a point + with well distinguished neighborhood. Moving the window in any direction will result in a large intensity change. Lin ZHANG, SSE, 2020

Harris Corner Detection: Mathematics Change in appearance of a local patch (defined by a   ( x , y ) window w ) centered at p for the shift :        2   S ( x , y ) ( ( , f x y ) f x ( x y , y )) w i i i i  ( x y , ) w i i Shifted Intensity intensity Window function Window function w = or 1 in window, 0 outside Gaussian Lin ZHANG, SSE, 2020

Harris Corner Detection: Mathematics        2 S ( x , y ) ( ( f x y , ) f x (  x y ,  y )) (1) w i i i i  ( x y , ) w i i 2          x f x y ( , ) f x y ( , )     f x y ( , ) f x y ( , ) i i , i i (2)       i i i i    y x y        ( x y , ) w i i 2         x  2 f x y ( , ) f x y ( , )  T  u u u (Due to )  , i i i i          x y y        ( x y , ) w i i      f x y ( , ) i i             x f x y ( , ) f x y ( , ) x            x ,  y i i i i      f x ( , y )      y x y        i i ( x y , ) w  i i     y        x       x y M    y   Lin ZHANG, SSE, 2020

Harris Corner Detection  M   2       f x y ( , )   f x y ( , ) f x y ( , )    i i i i i i           x  x y       ( x y , ) w ( x y , ) w i i i i   2        f x y ( , ) f x y ( , ) f x y ( , )       i i i i i i          x y y         ( x y , ) w ( x y , ) w i i i i • It is real symmetric Lin ZHANG, SSE, 2020

Harris Corner Detection   x     S (   x , y )   x , y M    y        2 ( I ) ( I I ) x x y     ( x y , ) w ( x y , ) w   M i i i i     2 ( I I ) ( I )   x y y     ( x y , ) w ( x y , ) w i i i i y    S ( x , ) 1 actually is the ellipse equation. The shape of the ellipse is determined by M . Lin ZHANG, SSE, 2020

Harris Corner Detection The “cornerness” of the window w is reflected in M Suppose there are two local windows w 1 and w 2 ; consider the cases when the moving of the two windows leads to the     x , y intensity change equals to 1. The moving vector of each window satisfies the ellipse equation. Thus,  y Which window For w 1 , has higher    x   cornerness?    x , y M 1  x   1  y   For w 2 ,  y    x      x , y M 1  x   2  y   Lin ZHANG, SSE, 2020

Harris Corner Detection    0 Why?   1 1 M R R Diagonalization of M :    0   2 The axis lengths of the ellipse are determined by the eigenvalues and the orientation is determined by R direction of the fastest change direction of the slowest change (  max ) -1/2 (  min ) -1/2 Lin ZHANG, SSE, 2020

Interpreting the eigenvalues Classification of image points using eigenvalues of M :  2 “ Edge”  2 >>  1 “ Corner”  1 and  2 are large,  1 ~  2 ; S increases in all directions  1 and  2 are small; “ Edge” S is almost constant “ Flat”  1 >>  2 in all directions region  1 Lin ZHANG, SSE, 2020

Corner response function Measure of corner response:   2   R det M k trace M    det M 1 2     trace M 1 2 ( k – empirical constant, k = 0.04‐0.06) Lin ZHANG, SSE, 2020