SLIDE 1
Camera Matrix
16-385 Computer Vision (Kris Kitani)
Carnegie Mellon University
2D to 2D Transform (last session)
Camera Matrix 16-385 Computer Vision (Kris Kitani) Carnegie Mellon - - PowerPoint PPT Presentation
Camera Matrix 16-385 Computer Vision (Kris Kitani) Carnegie Mellon - - PowerPoint PPT Presentation
Camera Matrix 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University 2D to 2D Transform (last session) 3D object 3D to 2D Transform (today) 2D to 2D Transform (last session) A camera is a mapping between the 3D world and a 2D
SLIDE 2
SLIDE 3
3D object 2D to 2D Transform (last session) 3D to 2D Transform (today)
SLIDE 4
A camera is a mapping between the 3D world and a 2D image
SLIDE 5
x = PX
camera matrix 3D world point 2D image point What do you think the dimensions are?
A camera is a mapping between the 3D world and a 2D image
SLIDE 6
x = PX
X Y Z = p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 X Y Z 1
homogeneous world point 4 x 1 homogeneous image 3 x 1 Camera matrix 3 x 4
SLIDE 7
camera center
X
image plane
x
principal axis
The pinhole camera x y z
z = f
What is the equation for image coordinate x (in terms of X)?
principal point
SLIDE 8
f
y
z
X
Y Z
image plane
?
What is the equation for image coordinate x (in terms of X)?
SLIDE 9
f
y
z
X
fY Z
Y Z
image plane
[X Y Z]> 7! [fX/Z fY/Z]>
SLIDE 10
camera center
X
image plane
x
principal axis
Pinhole camera geometry x y z
z = f
x = PX
What is the camera matrix P for a pinhole camera model?
principal point
SLIDE 11
X Y Z = p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 X Y Z 1
[X Y Z]> 7! [fX/Z fY/Z]>
Relationship from similar triangles…
P = ? ? ? ? ? ? ? ? ? ? ? ?
generic camera model What does the pinhole camera model look like?
SLIDE 12
P = f f 1
What does the pinhole camera model look like?
X Y Z = p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 X Y Z 1
[X Y Z]> 7! [fX/Z fY/Z]>
Relationship from similar triangles… generic camera model
SLIDE 13
P = f f 1
Camera origin and image origin might be different
camera coordinate system image coordinate system CCD array
p
SLIDE 14
P = f px f py 1
Accounts for different origins
camera coordinate system image coordinate system CCD array
p
SLIDE 15
X world point
Ocamera
Oimage
In general, the camera and image sensor have different coordinate systems x
image point
SLIDE 16
X world point
Ocamera
Oimage
Oworld
In general, there are three different coordinate systems… x
image point
so you need the know the transformations between them
SLIDE 17
P = f px f py 1 1 1 1
P = K[I|0]
Can be decomposed into two matrices K = f px f py 1
calibration matrix
(3 x 3) (3 x 4)
SLIDE 18
P = f px f py 1 1 1 1 Assumes that the camera and world share the same coordinate system What if they are different? How do we align them?
xc
yc zc zw xw yw
World coordinate system Camera coordinate system
SLIDE 19
P = f px f py 1 1 1 1 Assumes that the camera and world share the same coordinate system
xc
yc zc zw xw yw
R t
World coordinate system Camera coordinate system
3R rotation and translation to align axis
What if they are different? How do we align them?
SLIDE 20
xc
yc zc zw xw yw
World coordinate system Camera coordinate system
Xw
SLIDE 21
xc
yc zc zw xw yw
World coordinate system Camera coordinate system
Xw
C
Coordinate of the camera center in the world coordinate frame
SLIDE 22
xc
yc zc zw xw yw
World coordinate system Camera coordinate system
Xw
C
Coordinate of the camera center in the world coordinate frame
(Xw − C)
(Xw − C)
Translate Xc
SLIDE 23
xc
yc zc zw xw yw
World coordinate system Camera coordinate system
Xw
Why aren’t the points aligned?
C
(Xw − C)
Translate Xc
SLIDE 24
R(Xw − C)
What happens to points after alignment?
xc
yc zc zw xw yw
World coordinate system Camera coordinate system
Xw
C
R
Xc Rotate Translate Rotate
SLIDE 25
Xc = R(Xw − C)
Optionally in homogeneous coordinates: In inhomogeneous coordinates: 2 6 6 4 Xc Yc Zc 1 3 7 7 5 = R −RC 1
- 2
6 6 4 Xw Yw Zw 1 3 7 7 5 General mapping of a pinhole camera
P = KR[I| − C]
(4 × 4)
SLIDE 26
Quiz What is the meaning of each matrix of the camera matrix decomposition?
P = KR[I| − C]
3x3 intrinsics
SLIDE 27
Quiz What is the meaning of each matrix of the camera matrix decomposition?
P = KR[I| − C]
3x3 intrinsics 3x3 3D rotation
SLIDE 28
Quiz What is the meaning of each matrix of the camera matrix decomposition?
P = KR[I| − C]
3x3 intrinsics 3x3 3D rotation 3x3 identity
SLIDE 29
Quiz What is the meaning of each matrix of the camera matrix decomposition?
P = KR[I| − C]
3x3 intrinsics 3x3 3D rotation 3x3 identity 3x1 3D translation
SLIDE 30
General mapping of a pinhole camera
P = KR[I| − C]
Another way to write the mapping
t = −RC
where (translate first then rotate) (rotate first then translate)
P = K[R|t]
SLIDE 31
Quiz The camera matrix relates what two quantities?
SLIDE 32
Quiz The camera matrix relates what two quantities?
x = PX
SLIDE 33
Quiz The camera matrix relates what two quantities?
x = PX
3D points to 2D image points
SLIDE 34
Quiz The camera matrix relates what two quantities?
x = PX
3D points to 2D image points
The camera matrix can be decomposed into?
SLIDE 35
Quiz The camera matrix relates what two quantities?
x = PX
3D points to 2D image points
The camera matrix can be decomposed into?
P = K[R|t]
SLIDE 36
Quiz The camera matrix relates what two quantities?
x = PX
3D points to 2D image points
The camera matrix can be decomposed into?
P = K[R|t]
intrinsic and extrinsic parameters
SLIDE 37
P = f px f py 1 r1 r2 r3 t1 r4 r5 r6 t2 r7 r8 r9 t3 Generalized pinhole camera model
P = K[R|t]
R = r1 r2 r3 r4 r5 r6 r7 r8 r9 t = t1 t2 t3
extrinsic parameters intrinsic parameters 3D rotation 3D translation
SLIDE 38
Why do we need P? to properly relate world points to image points (by taking into account different coordinate systems)