camera matrix

Camera Matrix 16-385 Computer Vision (Kris Kitani) Carnegie Mellon - PowerPoint PPT Presentation

Camera Matrix 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University 2D to 2D Transform (last session) 3D object 3D to 2D Transform (today) 2D to 2D Transform (last session) A camera is a mapping between the 3D world and a 2D


  1. Camera Matrix 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University

  2. 2D to 2D Transform (last session)

  3. 3D object 3D to 2D Transform (today) 2D to 2D Transform (last session)

  4. A camera is a mapping between the 3D world and a 2D image

  5. A camera is a mapping between the 3D world and a 2D image x = PX 2D image camera 3D world point matrix point What do you think the dimensions are?

  6. x = PX   X     X p 1 p 2 p 3 p 4 Y    = Y p 5 p 6 p 7 p 8      Z   Z p 9 p 10 p 11 p 12 1 homogeneous Camera homogeneous image matrix world point 3 x 1 3 x 4 4 x 1

  7. The pinhole camera y image plane X x x z camera center principal axis z = f principal point What is the equation for image coordinate x (in terms of X )?

  8. y What is the equation for image coordinate x (in terms of X )? image plane X Y ? z f Z

  9. y image plane X Y fY Z z f Z Z ] > 7! [ fX/Z fY/Z ] > [ X Y

  10. Pinhole camera geometry y image plane X x x z camera center principal axis z = f principal point What is the camera matrix P for a pinhole camera model? x = PX

  11. Relationship from similar triangles… Z ] > 7! [ fX/Z fY/Z ] > [ X Y generic camera model   X     X p 1 p 2 p 3 p 4 Y    = Y p 5 p 6 p 7 p 8      Z   Z p 9 p 10 p 11 p 12 1 What does the pinhole camera model look like?   ? ? ? ? P = ? ? ? ?   ? ? ? ?

  12. Relationship from similar triangles… Z ] > 7! [ fX/Z fY/Z ] > [ X Y generic camera model   X     X p 1 p 2 p 3 p 4 Y    = Y p 5 p 6 p 7 p 8      Z   Z p 9 p 10 p 11 p 12 1 What does the pinhole camera model look like?   f 0 0 0 f P = 0 0 0   0 0 1 0

  13.   f 0 0 0 f P = 0 0 0   0 0 1 0 Camera origin and image origin might be different CCD array camera coordinate p system image coordinate system

  14. CCD array camera coordinate p system image coordinate system   f p x 0 0 f p y P = 0 0   0 0 1 0 Accounts for different origins

  15. In general, the camera and image sensor have different coordinate systems X world point O image x image point O camera

  16. In general, there are three different coordinate systems… X world point O image O world x image point O camera so you need the know the transformations between them

  17. Can be decomposed into two matrices     f p x 0 1 0 0 0 f p y P = 0 0 1 0 0     0 0 1 0 0 1 0 (3 x 3) (3 x 4) P = K [ I | 0 ]   f p x 0 f p y K = 0 calibration matrix   0 0 1

  18. Assumes that the camera and world share the same coordinate system     f p x 0 1 0 0 0 f p y P = 0 0 1 0 0     0 0 1 0 0 1 0 What if they are different? How do we align them? y c x c y w Camera coordinate system x w z c World z w coordinate system

  19. Assumes that the camera and world share the same coordinate system     f p x 0 1 0 0 0 f p y P = 0 0 1 0 0     0 0 1 0 0 1 0 What if they are different? How do we align them? y c x c y w Camera coordinate system x w z c R t World z w coordinate 3R rotation and translation to align axis system

  20. y c X w x c y w Camera coordinate system x w z c z w World coordinate system

  21. y c X w x c y w Camera coordinate system x w z c C z w World Coordinate of the coordinate camera center in the system world coordinate frame

  22. X c y c X w x c y w Camera coordinate system x w z c C z w World Coordinate of the coordinate camera center in the ( X w − C ) system world coordinate frame ( X w − C ) Translate

  23. Why aren’t X c the points aligned? y c X w x c y w Camera coordinate system x w z c C z w World coordinate system ( X w − C ) Translate

  24. What happens to points after alignment? Rotate X c R y c X w x c y w Camera coordinate system x w z c C z w World coordinate system R ( X w − C ) Rotate Translate

  25. In inhomogeneous coordinates: X c = R ( X w − C ) Optionally in homogeneous coordinates: 2 3 2 3 X c X w  R � − RC Y c Y w 6 7 6 7 5 = 6 7 6 7 0 Z c 1 Z w 4 4 5 1 1 (4 × 4) General mapping of a pinhole camera P = KR [ I | − C ]

  26. Quiz What is the meaning of each matrix of the camera matrix decomposition? P = KR [ I | − C ] 3x3 intrinsics

  27. Quiz What is the meaning of each matrix of the camera matrix decomposition? P = KR [ I | − C ] 3x3 3x3 intrinsics 3D rotation

  28. Quiz What is the meaning of each matrix of the camera matrix decomposition? P = KR [ I | − C ] 3x3 3x3 3x3 intrinsics 3D rotation identity

  29. Quiz What is the meaning of each matrix of the camera matrix decomposition? P = KR [ I | − C ] 3x3 3x3 3x3 3x1 intrinsics 3D rotation identity 3D translation

  30. General mapping of a pinhole camera P = KR [ I | − C ] (translate first then rotate) Another way to write the mapping P = K [ R | t ] where t = − RC (rotate first then translate)

  31. Quiz The camera matrix relates what two quantities?

  32. Quiz The camera matrix relates what two quantities? x = PX

  33. Quiz The camera matrix relates what two quantities? x = PX 3D points to 2D image points

  34. Quiz The camera matrix relates what two quantities? x = PX 3D points to 2D image points The camera matrix can be decomposed into?

  35. Quiz The camera matrix relates what two quantities? x = PX 3D points to 2D image points The camera matrix can be decomposed into? P = K [ R | t ]

  36. Quiz The camera matrix relates what two quantities? x = PX 3D points to 2D image points The camera matrix can be decomposed into? P = K [ R | t ] intrinsic and extrinsic parameters

  37. Generalized pinhole camera model P = K [ R | t ]     f p x r 1 r 2 r 3 t 1 0 f p y r 4 r 5 r 6 t 2 P = 0     r 7 r 8 r 9 t 3 0 0 1 intrinsic parameters extrinsic parameters     t 1 r 1 r 2 r 3 R = t = t 2 r 4 r 5 r 6     t 3 r 7 r 8 r 9 3D rotation 3D translation

  38. Why do we need P ? to properly relate world points to image points (by taking into account different coordinate systems)

Recommend


More recommend