Estimation of Transformations Shao-Yi Chien Department of - PowerPoint PPT Presentation

Estimation of Transformations 簡韶逸 Shao-Yi Chien Department of Electrical Engineering National Taiwan University Fall 2018 1

Outline • Estimation – 2D Projective Transformation [Slides credit: Marc Pollefeys] 2

Parameter Estimation • 2D homography Given a set of (x i ,x i ’), compute H (x i ’=Hx i ) • 3D to 2D camera projection Given a set of (X i ,x i ), compute P (x i =PX i ) • Fundamental matrix Given a set of (x i ,x i ’), compute F (x i ’ T Fx i =0) • Trifocal tensor Given a set of (x i ,x i ’,x i ”), compute T

Number of Measurements Required • At least as many independent equations as degrees of freedom required • Example:        x h h h x 11 12 13       x'    Hx λ y h h h y       21 22 23       1 1       h h h 31 32 33 2 independent equations / point 8 degrees of freedom 4x2 ≥8

Approximate Solutions • Minimal solution • 4 points yield an exact solution for H • More points • Robust estimation algorithms, such as RANSAC • No exact solution, because measurements are inexact (“noise”) • Search for “best” according to some cost function • Algebraic or geometric/statistical cost

Gold Standard Algorithm • Cost function that is optimal for some assumptions • Computational algorithm that minimizes it is called “Gold Standard” algorithm • Other algorithms can then be compared to it

Direct Linear Transformation (DLT)   T 1     h x x    Hx x Hx 0  i     i i i i      T T  2 x , , Hx h x x y w   i i i i i i T   3   h x T T      3 2 h x h x i y w   i i i i   T T       1 3 x Hx h x h x w x   i i i i i i T T      2 1 h x h x x y   i i i i        T T T 1 0 x x h w y   i i i i         T T T 2 x 0 x h 0 w x   i i i i        T T T 3 x x 0 h y x     i i i i  A h 0 i

Direct Linear Transformation (DLT) • Equations are linear in h  A h 0 i • Only 2 out of 3 are linearly independent (indeed, 2 eq/pt)        T T T 1 0 x x h w y   i i i i         T T T 2 x 0 x h 0 w x   i i i i        T T T 3 x x 0 h y x     i i i i

Direct Linear Transformation (DLT) • Equations are linear in h  A h 0 i • Only 2 out of 3 are linearly independent (indeed, 2 eq/pt)   1 h        T T T 0 x x w y    2 i i i i   h 0    T T T    x 0 x  w x 3 i i i i h   • Holds for any homogeneous representation, e.g. ( x i ’, y i ’,1)

Direct Linear Transformation (DLT) • Solving for H   A 1   A   Ah   2 h 0 0   A 3    A  4 size A is 8x9 or 12x9, but rank 8 T is not interesting Trivial solution is h =0 9 1-D null-space yields solution of interest h  pick for example the one with 1

Direct Linear Transformation (DLT) • Over-determined solution   A 1   A    2 h 0 Ah  0       A  n No exact solution because of inexact measurement i.e. “noise” Find approximate solution h  - Additional constraint needed to avoid 0, e.g. 1 Ah  - Ah 0 not possible, so minimize

DLT Algorithm Objective Given n≥4 2D to 2D point correspondences {x i ↔x i ’}, determine the 2D homography matrix H such that x i ’=Hx i Algorithm For each correspondence x i ↔x i ’ compute A i . Usually (i) only two first rows needed. Assemble n 2x9 matrices A i into a single 2 n x9 matrix A (ii) (iii) Obtain SVD of A. Solution for h is last column of V (iv) Determine H from h

Inhomogeneous Solution Since h can only be computed up to scale, ~ h pick h j =1, e.g. h 9 =1, and solve for 8-vector        0 0 0 ' ' ' ' ' ' x w y w w w x y y y w y ~    i i i i i i i i i i i i h      ' ' ' 0 0 0 ' '   '  x w y w w w x x y x w x i i i i i i i i i i i i Solve using Gaussian elimination (4 points) or using linear least-squares (more than 4 points) However, if h 9 =0 this approach fails also poor results if h 9 close to zero Therefore, not recommended Note h 9 =H 33 =0 if origin is mapped to infinity   0       T l Hx 0 0 1 H 0 0    0    1 

Degenerate Configurations x 1 x 1 x 1 x 4 x 4 x 4 x 2 H? H’? x 2 x 2 x 3 x 3 x 3 (case B) (case A)    x Hx 0 Constraints: i =1,2,3,4 i i   * T H x l Define: 4       * T Then, H x x l x 0 , 1 , 2 , 3 i 4 i i       * T H x x l x x k 4 4 4 4 H * is rank-1 matrix and thus not a homography If H * is unique solution, then no homography mapping x i → x i ’(case B) If further solution H exist, then also α H * + β H (case A) (2-D null-space in stead of 1-D null-space)

Solutions from Lines 2D homographies from 2D lines   Ah  T l H l 0 i i Minimum of 4 lines 3D Homographies (15 dof) Minimum of 5 points or 5 planes 2D affinities (6 dof) Minimum of 3 points or lines Conic provides 5 constraints

Solutions from Mixed Type • 2D homography • cannot be determined uniquely from the correspondence of 2 points and 2 line • can from 3 points and 1 line or 1 point and 3 lines 16

Cost Functions • Algebraic distance • Geometric distance • Reprojection error • Comparison • Geometric interpretation • Sampson error

Algebraic Distance Ah DLT minimizes  Ah e residual vector e partial vector for each (x i ↔ x i ’) i algebraic error vector 2       T T T 0 x x   w y  2   2 i i i i   x , Hx h d e   alg   i i i T T T x 0 x   w x i i i i algebraic distance       2    T 2 2 x , x where a , , x x d a a a a a alg 1 2 1 2 1 2 3 1 2      2 2 2    2 x , Hx Ah d e e alg i i i i i Not geometrically/statistically meaningfull, but given good normalization it works fine and is very fast (use for initialization for non-linear minimization)

Geometric Distance x measured coordinates ˆ x estimated coordinates x true coordinates d (.,.) Euclidean distance (in image) Error in one image e.g. calibration pattern    ˆ   2 H argmin x , H x d i i H i Symmetric transfer error      ˆ 2     2 - 1 H argmin x , H x x , Hx d d i i i i H i Reprojection error        ˆ      2 2 ˆ ˆ ˆ ˆ H , x , x argmin x , x x , x d d i i i i i i  ˆ ˆ H, x , x ˆ i   i i ˆ ˆ subject to x H x i i

Symmetric Transfer Error v.s. Reprojection Error Symmetric Transfer Error     2    2 - 1 x, H x x , Hx d d Reprojection Error        d ˆ 2 ˆ 2 x, x x , x d

Comparison of Geometric and Algebraic Distances Error in one image                T ˆ ˆ ˆ ˆ T x , , x , , H x x y w x y w i i i i i i i i   1 h             ˆ ˆ   T T T y w w y 0 x x w y       2 i i i i i i i i   h A h e           ˆ ˆ T T T i i    x 0 x  w x   w x x w 3 i i i i h i i i i                       ˆ 2 ˆ ˆ 2 ˆ ˆ 2 x , x d y w w y w x x w alg i i i i i i i i i i         1 / 2               2 2 2 ˆ ˆ ˆ ˆ ˆ x , x / / / / d y w y w x w x w i i i i i i i i i i        ˆ ˆ x , x / d w w these two distance metrics are related, but not identical alg i i i i  ˆ    h 3 x 1 w w typical, but not , except for affinities i i i  For affinities, DLT can minimize geometric distance