6.869 Advances in Computer Vision Matching with Invariant Features Prof. Bill Freeman March 3, 2005 Image and shape descriptors Darya Frolova, Denis Simakov – Affine invariant features – Comparison of feature descriptors The Weizmann Institute of Science – Shape context March 2004 Readings: Mikolajczyk and Schmid; Belongie et al http://www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/InvariantFeatures.ppt Example: Build a Panorama How do we build panorama? • We need to match (align) images M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003 Matching with Features Matching with Features •Detect feature points in both images •Detect feature points in both images •Find corresponding pairs 1
Matching with Features Matching with Features •Detect feature points in both images •Find corresponding pairs • Problem 1: •Use these pairs to align images – Detect the same point independently in both images no chance to match! We need a repeatable detector Matching with Features More motivation… • Feature points are used also for: • Problem 2: – Image alignment (homography, fundamental matrix) – For each point correctly recognize the – 3D reconstruction corresponding one – Motion tracking ? – Object recognition – Indexing and database retrieval – Robot navigation – … other We need a reliable and distinctive descriptor Contents • Harris Corner Detector An introductory example: – Description – Analysis • Detectors Harris corner detector – Rotation invariant – Scale invariant – Affine invariant • Descriptors – Rotation invariant – Scale invariant – Affine invariant C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988 2
Harris Detector: Basic Idea The Basic Idea • We should easily recognize the point by looking through a small window • Shifting a window in any direction should give a large change in intensity “flat” region: “edge”: “corner”: no change in no change along significant change all directions the edge direction in all directions Contents Harris Detector: Mathematics • Harris Corner Detector – Description Change of intensity for the shift [ u,v ]: – Analysis ∑ [ ] 2 • Detectors = + + − E u v ( , ) w x y ( , ) I x ( u y , v ) I x y ( , ) – Rotation invariant x y , – Scale invariant – Affine invariant Window Shifted Intensity function intensity • Descriptors – Rotation invariant – Scale invariant Window function w(x,y) = or – Affine invariant 1 in window, 0 outside Gaussian Harris Detector: Mathematics Harris Detector: Mathematics Intensity change in shifting window: eigenvalue analysis For small shifts [ u,v ] we have a bilinear approximation: ⎡ ⎤ u [ ] λ 1 , λ 2 – eigenvalues of M ⎡ ⎤ ≅ u E u v ( , ) u v , M [ ] ⎢ ⎥ ≅ E u v ( , ) u v , M ⎢ ⎥ ⎣ ⎦ v ⎣ ⎦ v direction of the fastest change where M is a 2 × 2 matrix computed from image derivatives: Ellipse E(u,v) = const direction of the slowest change ⎡ ⎤ 2 I I I ∑ ( λ max ) -1/2 = x x y ⎢ ⎥ M w x y ( , ) ⎢ 2 ⎥ I I I ( λ min ) -1/2 ⎣ ⎦ x y , x y y 3
Harris Detector: Mathematics Harris Detector: Mathematics λ 2 Measure of corner response: Classification of “Edge” λ 2 >> λ 1 image points using “Corner” ( ) = − 2 R det M k trace M eigenvalues of M : λ 1 and λ 2 are large, λ 1 ~ λ 2 ; E increases in all = λ λ directions det M 1 2 = λ + λ trace M 1 2 λ 1 and λ 2 are small; “Edge” E is almost constant “Flat” λ 1 >> λ 2 in all directions region ( k – empirical constant, k = 0.04-0.06) λ 1 Harris Detector: Mathematics cf David Lowe’s analysis λ 2 “Edge” “Corner” • R depends only on R < 0 eigenvalues of M • R is large for a corner R > 0 • R is negative with large magnitude for an edge • | R | is small for a flat region “Flat” “Edge” |R| small R < 0 λ 1 Harris Detector: Workflow Harris Detector • The Algorithm: – Find points with large corner response function R ( R > threshold) – Take the points of local maxima of R 4
Harris Detector: Workflow Harris Detector: Workflow Compute corner response R Find points with large corner response: R> threshold Harris Detector: Workflow Harris Detector: Workflow Take only the points of local maxima of R Contents Harris Detector: Summary • Harris Corner Detector – Description • Average intensity change in direction [ u,v ] can be – Analysis expressed as a bilinear form: • Detectors ⎡ ⎤ u [ ] – Rotation invariant ≅ E u v ( , ) u v , M ⎢ ⎥ ⎣ ⎦ v – Scale invariant – Affine invariant • Describe a point in terms of eigenvalues of M : • Descriptors measure of corner response – Rotation invariant ( ) 2 = λ λ − λ + λ R k 1 2 1 2 – Scale invariant • A good (corner) point should have a large intensity change – Affine invariant in all directions , i.e. R should be large positive 5
Harris Detector: Some Properties Harris Detector: Some Properties • Rotation invariance • Partial invariance to affine intensity change � Only derivatives are used => invariance to intensity shift I → I + b � Intensity scale: I → a I Ellipse rotates but its shape (i.e. eigenvalues) R R remains the same threshold Corner response R is invariant to image rotation x (image coordinate) x (image coordinate) Harris Detector: Some Properties Harris Detector: Some Properties • Quality of Harris detector for different scale • But: non-invariant to image scale ! changes Repeatability rate: # correspondences # possible correspondences All points will be Corner ! classified as edges C.Schmid et.al. “Evaluation of Interest Point Detectors”. IJCV 2000 Contents • Harris Corner Detector – Description – Analysis We want to: • Detectors – Rotation invariant detect the same interest points – Scale invariant regardless of image changes – Affine invariant • Descriptors – Rotation invariant – Scale invariant – Affine invariant 6
Contents Models of Image Change • Harris Corner Detector – Description • Geometry – Analysis – Rotation • Detectors – Similarity (rotation + uniform scale) – Rotation invariant – Scale invariant – Affine (scale dependent on direction) – Affine invariant valid for: orthographic camera, locally planar • Descriptors object – Rotation invariant • Photometry – Scale invariant – Affine intensity change ( I → a I + b ) – Affine invariant Contents Rotation Invariant Detection • Harris Corner Detector – Description • Harris Corner Detector – Analysis • Detectors – Rotation invariant – Scale invariant – Affine invariant • Descriptors – Rotation invariant – Scale invariant – Affine invariant C.Schmid et.al. “Evaluation of Interest Point Detectors”. IJCV 2000 Scale Invariant Detection Scale Invariant Detection • Consider regions (e.g. circles) of different sizes • The problem: how do we choose corresponding around a point circles independently in each image? • Regions of corresponding sizes will look the same in both images 7
Scale Invariant Detection Scale Invariant Detection • Solution: • Common approach: – Design a function on the region (circle), which is “scale Take a local maximum of this function invariant” (the same for corresponding regions, even if they are at different scales) Observation : region size, for which the maximum is Example : average intensity. For corresponding regions achieved, should be invariant to image scale. (even of different sizes) it will be the same. Important: this scale invariant region size is – For a point in one image, we can consider it as a function of region size (circle radius) found in each image independently! Image 1 Image 1 f f f f Image 2 Image 2 scale = 1/2 scale = 1/2 s 1 s 2 region size region size region size region size Scale Invariant Detection Scale Invariant Detection f = ∗ Kernel Image • A “good” function for scale detection: • Functions for determining scale has one stable sharp peak Kernels: ( ) = σ σ + σ 2 f f L G ( , , x y ) G ( , , x y ) f Good ! xx yy bad bad (Laplacian) region size region size region size = σ − σ DoG G x y k ( , , ) G x y ( , , ) (Difference of Gaussians) • For usual images: a good function would be a one which responds to contrast (sharp local intensity where Gaussian change) 2 + 2 x y − Note: both kernels are invariant to σ = σ 2 G x y ( , , ) 1 e 2 scale and rotation πσ 2 Scale Invariant Detection Scale Invariant Detectors ← Laplacian → scale • Harris-Laplacian 1 • Compare to human vision: eye’s response Find local maximum of: – Harris corner detector in y space (image coordinates) – Laplacian in scale ← Harris → x scale • SIFT (Lowe) 2 ← DoG → Find local maximum of: – Difference of Gaussians in y space and scale ← DoG → x 1 K.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 2001 2 D.Lowe. “Distinctive Image Features from Scale-Invariant Keypoints”. Accepted to IJCV 2004 Shimon Ullman, Introduction to Computer and Human Vision Course, Fall 2003 8
Recommend
More recommend
