Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Image Registration by Probabilistic Multi-Assignment Graph Matching Uri Okun Department of Electrical Engineering, Technion - Israel Institute of Technology Supervised by Prof. Israel Cohen and Prof. Yosi Keller November 18, 2014
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Outline Introduction 1 Common Matching Strategies 2 Epipolar Geometry Robust Estimation Graph Matching Discussion Our Contribution 3 Overview of the Method Multi-Assignment Graph Matching Unified Probabilistic Stereo Matching MAGMA Experimental Study 4 Synthetic Datasets Real Image Sequences Conclusion 5
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Wide-Baseline Image Registration Image matching - Finding a (large) coherent set of tie points between two images depicting the same scene Wide-Baseline Stereo Pair Two photos taken from disparate positions and viewing angles Often causing for severe perspective deformations
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Motivation Some applications: Three-dimensional scene reconstruction Image querying Object recognition Navigation and motion estimation Change detection Image-data fusion Object recognition Mapping ...
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Image Registration Challenges The two images can be of: Different sensors Different acquisition times Different lighting conditions Occlusions Repetitive patterns Different viewpoints: Rotations Scale changes Perspective transformations
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Image Matching Problem Statement Image registration algorithms often consist of the following steps: 1 Detection of interest points (features) 2 Assigning each feature with a descriptor 3 Matching the features from one image to the other The matching step is a difficult task - outliers may form due to false correspondences between repetitive image patterns or accidental feature similarity
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Points Matching by Descriptors Similarity A naive implementation: Each feature may be assigned to its nearest neighbor (most similar descriptor)
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Common Matching Strategies Two main approaches for solving the matching problem are based upon geometrical relations between feature points in both images: 1 Epipolar geometry robust estimation 2 Graph matching
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Epipolar Geometry The epipolar constraint is the geometric structure which holds for all pairs of pinhole cameras, regardless of the captured scene [Hartley and Zisserman, 2004] The pinhole camera model:
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Epipolar Geometry (Cont.) For stereo pair: The epipolar constraint is given by: x T F y = 0 x ∈ I 1 , y ∈ I 2 are points in homogeneous coordinates F is the 3 × 3 Fundamental matrix The epipolar lines are given by: l ′ T = x T F , l = F y
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Robust Model Estimation The Fundamental matrix can be calculated using the linear Normalized 8-point algorithm, or non-linear 7-point Algorithm Epipolar robust estimation techniques relay on estimating the fundamental matrix coefficients from sets of putative matches while simultaneously removing gross outliers RANSAC (RANdom Sampling Consensus) [Fischler and Bolles, 1981] and RANSAC-based algorithms are the most popular choice
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References RANSAC [Fischler and Bolles, 1981]
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References RANSAC [Fischler and Bolles, 1981] Select sample of s points at random Calculate model parameters that fit the data in the sample Calculate error function for each data point Select data that support current hypothesis Repeat sampling
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References RANSAC [Fischler and Bolles, 1981] Select sample of s points at random Calculate model parameters that fit the data in the sample Calculate error function for each data point Select data that support current hypothesis Repeat sampling
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References RANSAC [Fischler and Bolles, 1981] Select sample of s points at random Calculate model parameters that fit the data in the sample Calculate error function for each data point Select data that support current hypothesis Repeat sampling
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References RANSAC [Fischler and Bolles, 1981] Select sample of s points at random Calculate model parameters that fit the data in the sample Calculate error function for each data point Select data that support current hypothesis Repeat sampling
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References RANSAC [Fischler and Bolles, 1981] In order to ensure drawing at least one inlier-set with a certainty of c , the total number of samples is evaluated as following: N samples = log(1 − c ) log(1 − p i s ) , where p i is the inlier rate, and s is the sample size Example: Epipolar geometry estimation using the linear algorithm, with a certainty of 95%: p i N samples 4 . 57 e 3 0.4 1 . 17 e 6 0.2 3 e 8 0.1
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References RANSAC-Based Algorithms Improvements might... Use other variation of cost functions MLESAC [Torr and Zisserman, 2000] MAPSAC [Torr, 2002] Exploit each drawn sample to the fullest LO-RANSAC [Chum et al., 2003] Use guided sampling NAPSAC [Myatt et al., 2002] PROSAC [Chum and Matas, 2005] BEEM [Goshen and Shimshoni, 2006] BLOGS [Brahmachari and Sarkar, 2013] Decrease number of points per sample BEEM [Goshen and Shimshoni, 2006]
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Graph Matching Looking for an alignment which preserves geometrical consistency of small groups of points. Construct an affinity graph Extract prominent cluster within the graph The graph encodes spatial invariance properties: Pairwise affinities – encoding isometry (translation/rotation) Triplets affinities – encoding similarity (scale invariant)
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Pairwise Graph Matching In pairwise GM we calculate the affinity between each pair of correspondences, � c ii ′ , c jj ′ � � φ 2 � � x j , y j ′ �� φ 2 { x i , y i ′ } , = � − 1 � � � exp � � x i − x j � L 2 − � y i − y j � L 2 . � � ε � The goal is to find C = { c ii ′ } n i =1 , n ≤ min { N 1 , N 2 } such that, C ∗ = arg max � � c ii ′ , c jj ′ � φ 2 . C c ii ′ , c jj ′ ∈ C
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Pairwise Graph Matching (Cont.) The assignment matrix ( N 1 × N 2 ) represents the correspondences: � 1 if c ii ′ ∈ C Z i , i ′ = f c ii ′ / 0 ∈ C The matching problem can be formulated as: z ∗ = arg max z T Az � � z z ∈ { 0 , 1 } N 1 N 2 , s . t . � z ij ≤ 1 ∀ j , i � z ij ≤ 1 ∀ i j where z is a row-stacked replica of Z , and A is the affinity matrix ( A ∈ R N 1 · N 2 × N 1 · N 2 ), defined as: � i , i ′ ; j , j ′ � � A � i ( N 2 − 1) + i ′ , j ( N 2 − 1) + j ′ � � c ii ′ , c jj ′ � A = φ 2 .
Introduction Common Matching Strategies Our Contribution Experimental Study Conclusion References Pairwise Graph Matching (Cont.) A = φ ( c 11 , c 11 ) φ ( c 11 , c 12 ) · · · φ ( c 11 , c 1 N 2 ) φ ( c 11 , c 21 ) φ ( c 11 , c 22 ) · · · φ ( c 11 , c N 1 N 2 ) φ ( c 12 , c 11 ) φ ( c 12 , c 12 ) · · · φ ( c 12 , c 1 N 2 ) · · · φ ( c 12 , c N 1 N 2 ) . . ... . . . . φ ( c 1 N 2 , c 11 ) φ ( c 1 N 2 , c 12 ) φ ( c 1 N 2 , c 1 N 2 ) . . φ ( c 21 , c 11 ) . ... φ ( c 22 , c 11 ) . . . φ ( c N 1 N 2 , c 11 ) · · · φ ( c N 1 N 2 , c N 1 N 2 ) z ∗ = arg max z T Az � � z z ∈ { 0 , 1 } N 1 N 2 , s . t . � z ij ≤ 1 ∀ j , i � z ij ≤ 1 ∀ i j
Recommend
More recommend
