Camera Matrix 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University
2D to 2D Transform (last session)
3D object 3D to 2D Transform (today) 2D to 2D Transform (last session)
A camera is a mapping between the 3D world and a 2D image
A camera is a mapping between the 3D world and a 2D image x = PX 2D image camera 3D world point matrix point What do you think the dimensions are?
x = PX X X p 1 p 2 p 3 p 4 Y = Y p 5 p 6 p 7 p 8 Z Z p 9 p 10 p 11 p 12 1 homogeneous Camera homogeneous image matrix world point 3 x 1 3 x 4 4 x 1
The pinhole camera y image plane X x x z camera center principal axis z = f principal point What is the equation for image coordinate x (in terms of X )?
y What is the equation for image coordinate x (in terms of X )? image plane X Y ? z f Z
y image plane X Y fY Z z f Z Z ] > 7! [ fX/Z fY/Z ] > [ X Y
Pinhole camera geometry y image plane X x x z camera center principal axis z = f principal point What is the camera matrix P for a pinhole camera model? x = PX
Relationship from similar triangles… Z ] > 7! [ fX/Z fY/Z ] > [ X Y generic camera model X X p 1 p 2 p 3 p 4 Y = Y p 5 p 6 p 7 p 8 Z Z p 9 p 10 p 11 p 12 1 What does the pinhole camera model look like? ? ? ? ? P = ? ? ? ? ? ? ? ?
Relationship from similar triangles… Z ] > 7! [ fX/Z fY/Z ] > [ X Y generic camera model X X p 1 p 2 p 3 p 4 Y = Y p 5 p 6 p 7 p 8 Z Z p 9 p 10 p 11 p 12 1 What does the pinhole camera model look like? f 0 0 0 f P = 0 0 0 0 0 1 0
f 0 0 0 f P = 0 0 0 0 0 1 0 Camera origin and image origin might be different CCD array camera coordinate p system image coordinate system
CCD array camera coordinate p system image coordinate system f p x 0 0 f p y P = 0 0 0 0 1 0 Accounts for different origins
In general, the camera and image sensor have different coordinate systems X world point O image x image point O camera
In general, there are three different coordinate systems… X world point O image O world x image point O camera so you need the know the transformations between them
Can be decomposed into two matrices f p x 0 1 0 0 0 f p y P = 0 0 1 0 0 0 0 1 0 0 1 0 (3 x 3) (3 x 4) P = K [ I | 0 ] f p x 0 f p y K = 0 calibration matrix 0 0 1
Assumes that the camera and world share the same coordinate system f p x 0 1 0 0 0 f p y P = 0 0 1 0 0 0 0 1 0 0 1 0 What if they are different? How do we align them? y c x c y w Camera coordinate system x w z c World z w coordinate system
Assumes that the camera and world share the same coordinate system f p x 0 1 0 0 0 f p y P = 0 0 1 0 0 0 0 1 0 0 1 0 What if they are different? How do we align them? y c x c y w Camera coordinate system x w z c R t World z w coordinate 3R rotation and translation to align axis system
y c X w x c y w Camera coordinate system x w z c z w World coordinate system
y c X w x c y w Camera coordinate system x w z c C z w World Coordinate of the coordinate camera center in the system world coordinate frame
X c y c X w x c y w Camera coordinate system x w z c C z w World Coordinate of the coordinate camera center in the ( X w − C ) system world coordinate frame ( X w − C ) Translate
Why aren’t X c the points aligned? y c X w x c y w Camera coordinate system x w z c C z w World coordinate system ( X w − C ) Translate
What happens to points after alignment? Rotate X c R y c X w x c y w Camera coordinate system x w z c C z w World coordinate system R ( X w − C ) Rotate Translate
In inhomogeneous coordinates: X c = R ( X w − C ) Optionally in homogeneous coordinates: 2 3 2 3 X c X w R � − RC Y c Y w 6 7 6 7 5 = 6 7 6 7 0 Z c 1 Z w 4 4 5 1 1 (4 × 4) General mapping of a pinhole camera P = KR [ I | − C ]
Quiz What is the meaning of each matrix of the camera matrix decomposition? P = KR [ I | − C ] 3x3 intrinsics
Quiz What is the meaning of each matrix of the camera matrix decomposition? P = KR [ I | − C ] 3x3 3x3 intrinsics 3D rotation
Quiz What is the meaning of each matrix of the camera matrix decomposition? P = KR [ I | − C ] 3x3 3x3 3x3 intrinsics 3D rotation identity
Quiz What is the meaning of each matrix of the camera matrix decomposition? P = KR [ I | − C ] 3x3 3x3 3x3 3x1 intrinsics 3D rotation identity 3D translation
General mapping of a pinhole camera P = KR [ I | − C ] (translate first then rotate) Another way to write the mapping P = K [ R | t ] where t = − RC (rotate first then translate)
Quiz The camera matrix relates what two quantities?
Quiz The camera matrix relates what two quantities? x = PX
Quiz The camera matrix relates what two quantities? x = PX 3D points to 2D image points
Quiz The camera matrix relates what two quantities? x = PX 3D points to 2D image points The camera matrix can be decomposed into?
Quiz The camera matrix relates what two quantities? x = PX 3D points to 2D image points The camera matrix can be decomposed into? P = K [ R | t ]
Quiz The camera matrix relates what two quantities? x = PX 3D points to 2D image points The camera matrix can be decomposed into? P = K [ R | t ] intrinsic and extrinsic parameters
Generalized pinhole camera model P = K [ R | t ] f p x r 1 r 2 r 3 t 1 0 f p y r 4 r 5 r 6 t 2 P = 0 r 7 r 8 r 9 t 3 0 0 1 intrinsic parameters extrinsic parameters t 1 r 1 r 2 r 3 R = t = t 2 r 4 r 5 r 6 t 3 r 7 r 8 r 9 3D rotation 3D translation
Why do we need P ? to properly relate world points to image points (by taking into account different coordinate systems)
Recommend
More recommend