Camera Calibration (Compute Camera Matrix P) Shao-Yi Chien - PowerPoint PPT Presentation

Camera Calibration (Compute Camera Matrix P) 簡韶逸 Shao-Yi Chien Department of Electrical Engineering National Taiwan University Fall 2019 1

Outline • Camera calibration [Slides credit: Marc Pollefeys] 2

Resectioning X  P ? x i i

Basic Equations x  PX i i   x i PX X  i Ap  0

Basic Equations Ap  0 minimal solution P has 11 dof, 2 independent eq./points  5½ correspondences needed (say 6) Over-determined solution n  6 points Ap minimize subject to constraint p  1 ˆ 3  p 1 or  P ˆ 3 p

Degenerate Configurations More complicate than 2D case (i) Camera and points on a twisted cubic (ii) Points lie on plane or single line passing through projection center

Data Normalization Less obvious (i) Simple, as before 3 2 (ii) Anisotropic scaling

Line Correspondences Extend DLT to lines   P T l (back-project line) i T PX T PX l l (2 independent eq.) i 2 i i 1 i

Geometric Error

Gold Standard Algorithm Objective Given n≥6 2D to 2D point correspondences {X i ↔x i ’}, determine the Maximum Likelyhood Estimation of P Algorithm (i) Linear solution: ~  ~  (a) Normalization: x Tx X UX i i i i (b) DLT: (ii) Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error: ~ ~ ~ ~  -1 (iii) Denormalization: P T P U

Calibration Example (i) Canny edge detection (ii) Straight line fitting to the detected edges (iii) Intersecting the lines to obtain the images corners typically precision <1/10 (HZ rule of thumb: 5 n constraints for n unknowns

Errors in the World   x P X i i Errors in the image and in the world

Geometric Interpretation of Algebraic error    2 ˆ ˆ w i d ( x , x ) i i i      ˆ ˆ ˆ ˆ ˆ 3 w p depth(X; P) w x , y , 1 PX i i i i i 3  ˆ therefore, if p 1 then ˆ ˆ ˆ w d ( x , x ) ~ fd ( X , X ) i i i i i

Estimation of Affine Camera Last row = (0, 0, 0, 1) note that in this case algebraic error = geometric error

Gold Standard Algorithm Objective Given n≥4 2D to 2D point correspondences {X i ↔x i ’}, determine the Maximum Likelyhood Estimation of P (remember P 3T =(0,0,0,1)) Algorithm ~  ~  (i) Normalization: x Tx X UX i i i i (ii) For each correspondence  A p b 8 8 (iii) solution is   p A b 8 8 ~  -1 P T P U (iv) Denormalization:

Restricted Camera Estimation Find best fit that satisfies • skew s is zero • pixels are square • principal point is known • complete camera matrix K is known Minimize geometric error  impose constraint through parametrization  Image only  9   2n , otherwise  3n+9   5n Minimize algebraic error  assume map from param q  P=K[R|-RC], i.e. p=g(q)  minimize ||Ag(q)||

Reduced Measurement Matrix One only has to work with 12x12 matrix, not 2nx12 ~ ^   T T Ap p A Ap A p

Restricted Camera Estimation Initialization • Use general DLT Clamp values to desired values, e.g. s=0,  x =  y • Note: can sometimes cause big jump in error Alternative initialization • Use general DLT • Impose soft constraints • gradually increase weights

Exterior Orientation Calibrated camera, position and orientation unkown  Pose estimation 6 dof  3 points minimal (4 solutions in general)

Covariance Estimation ML residual error Example: n=197, =0.365, =0.37

Radial Distortion short and long focal length

𝑦, ෤ ෤ 𝑧 : non-distorted projection 𝑦 𝑒 , 𝑧 𝑒 : distorted projection

Correction of Distortion Choice of the distortion function and center : interior parameters Computing the parameters of the distortion function (i) Minimize with additional unknowns (ii) Straighten lines (iii) …

Correction of Distortion After radial correction

Another Method of Calibration • Notation K ≡ • Homography between the model plane and its image Ref: Zhengyou Zhang , “ Flexible camera calibration by viewing a plane from unknown orientations ,” ICCV1999 . 29

Another Method of Calibration • Constraints on the intrinsic parameters r 1 and r 2 are orthonormal  30

Another Method of Calibration • Close-form solution • Let 31

Another Method of Calibration • Close-form solution • From the two constraints on the intrinsic parameters • V is a 2n x 6 matrix, if 𝑜 ≥ 3 , we will have in general a unique solution b defined up to a scale factor. Once b is estimated, we can compute the camera intrinsic matrix A . 32

Calibration Procedure http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples 33

Calibration Procedure http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples 34

Calibration Procedure http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples 35

Calibration Procedure http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples • If the location of the corners are not correct  adjust radial distortion manually 36

Calibration Procedure http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples 37

Calibration Procedure http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples 38