Why Local Features? Scale and Affine Invariant Interest Point � Robust to noise, occlusion and clutter. Detectors � Distinctive and repeatable. � No explicit segmentation required – represent Krystian Mikolajczyk and Cordelia Schm id objects (classes). * * Sources: Schmid (CVPR’03), Tuytelaars (ECCV’06). � Invariance to image transformations + robust to illumination changes. � Applications: SLAM, object (class) recognition, matching… 1 2 Some keywords… The scale-adapted SMM � Harris corner detector. � Terms: differentiation scale, integration scale, based on variance of kernel. � Scale sensitive… ⎡ σ σ ⎤ 2 L ( , ) L L ( , ) � Difference-of-Gaussian (DoG). x x μ σ σ = σ σ ⎢ x D x y D ⎥ 2 ( , , ) g ( ) x � Lowe’s paper: approx. to normalized LoG. I D D I σ σ ⎢ 2 ⎥ ⎣ L L ( , ) L ( , ) ⎦ x x x y D y D σ = σ s � Laplacian-of-Gaussian (LoG). D I � Normalized = > Extrema in scale-space. � Ref: Elimination of edge responses in Lowe’s paper using eigen values… � Related to second moment matrix (SMM): second- order derivates of kernel-convolved image. 3 4 Characteristic Scale – scale invariance Characteristic scale selection � Apply local operator at scales: scale where operator best matches local structure. � Multi-scale Harris. � Characteristic scale with Laplacian. � LoG better than scale-adapted Harris. 5 6 1
Scale invariant feature selection Affine transforms – why? � Harris-Laplace (HL) detector: � Viewpoint changes ~ affine transform. � Harris measure, 8-neighborhood IP – larger scale � Scale changes by different amounts. ratio. � Harris, HL not affine invariant. � Iterate using LoG until convergence – smaller � Operate in affine Gaussian scale-space: ellipses as scale ratio. point neighborhoods. � Simplified HL: � Reduce scale diff, find IP, keep those with LoG extremum. � Simplified HL almost as good as HL. detected scale invariant projected region region 7 8 Affine Invariance – linear algebra 101 ☺ Affine Invariance – a picture = μ Σ = μ Σ ( x , ) M M ( x , ) � Basis: Anisotropy is affine-transformed isotropy. L L L R R R High-dim search space. → x A x Σ � Constraints on of Gaussian kernels: � recover affine shape, � reduce to orthogonal transform in normalized frames. 1 1 − − → → x 2 x x 2 x M M L R � Patterns in normalized frames are isotropic with 1 1 respect to SMM. = ( 2 x ) ( 2 x ) M R M R R L L Σ , L Σ � Estimation of - iterative algorithm. R Isotropic neighborhoods related by rotation 9 10 Affine Invariance – how? Affine Transformation detection � Step 1: Detect presence of affine � Eigen values – yes, again! � Ratio of eigen values of SMM: eigen values equal= > transformation. isotropy. λ μ ( ) = min Q λ μ ( ) max � Step 2: Transform IPs to normalized � Once more, a measure of the skew/ stretch. fram es, get to circular point neighborhoods, achieve affine invariance… � Ref: Lowe’s feature rejection based on the r-factor . 11 12 2
Σ , L Σ Algorithm ( ) – iterate until convergence Affine invariant Harris points R � Shape adaptation – normalize window using a � Iterative estimation of localization, scale, function of SMM. neighborhood σ � Select - remember characteristic scale . I σ � Select - equalize eigen values. D � Spatial localization of IP (interest point) – Harris detectors. � Compute SMM and update normalization matrix. Iteration #1 13 14 Affine invariant Harris points Affine invariant Harris points � Iterative estimation of localization, scale, � Iterative estimation of localization, scale, neighborhood neighborhood Iteration #2 Iteration #3, #4, ... 15 16 Notes Affine invariant Harris points � Convergence based on reasonable choice of scales and initial estimates. � Initial estimates of I Ps not affine invariant . � Averaging of similar features. � Only (20-30)% of initial IPs used. � Repeatability criterion. � More robust to large viewpoint changes. � Smallest number of features found . � Largest time complexity . affine Harris Harris-Laplace 17 18 3
Descriptors and Matching Matches – HarAff, large change in viewpoint � Normalized Gaussian gradient descriptors – weak ! � Cause of matching failure – use SIFT descriptors (ref: Moreels + Perona evaluation)… � Matching based on Mahalanobis distance and filters. � Comparable performance under scale changes and localization errors. � Performance much better under significant viewpoint changes. � Next, some ‘lab made’ image results ☺ 33 correct matches 19 20 Matches – SIFT, large change in viewpoint Images – difference in feature selection… 12 correct matches – hmm... 21 22 Images – difference in feature selection… Some SIFT matching – good… 23 24 4
SIFT Matching – not so good… SIFT Matching – not so good… 25 26 SIFT Matching – not so good… SIFT Matching – not so good… 27 28 Some other methods – MSER Some other methods – IBR 29 30 5
Observations… � Local features intuitively appealing – a lot of open That’s all folks ☺ questions still . � Scale, rotation, affine invariance, robust to viewpoint and illumination changes. � Depend on texture in images – absence of texture can make it unreliable. � Can add other features – Color? Texture? Structure? � Can combine with feature-learning approaches? 31 32 6
Recommend
More recommend
