1
2 CS 532: 3D Computer Vision Lecture 2 Enrique Dunn edunn@stevens.edu Lieb 310
Image Formation Based on slides by John Oliensis 3
Lecture Outline • Single View Geometry • 2D projective transformations – Homographies • Robust estimation – RANSAC • Radial distortion • Two-view geometry Based on slides by R. Hartley, A. Zisserman, M. Pollefeys and S. Seitz 4
Image Formation Pinhole camera light ray image plane (film) Object pinhole Virtual image 5
Projection Equation • 2D world è 1D image Object Image x camera center z f (focal length) “ Film ” 6
Projection Equation: 3D X Y Z , , ( ) ˆ Y ˆ X Y y x X ˆ Z Z f f x y f x , y = X , Y ( ) ( ) = = Similar triangles: Z X Y Z 7
Perspective Projection: Properties • 3D points è image points • 3D straight lines è image straight lines • 3D Polygons è image polygons 8
Polyhedra Project to Polygons (since lines project to lines) 9
Properties: Distant objects are smaller C ’ B ’ 10
Sin Single le Vie View Geome metry ry Rich ichard rd Hart rtle ley y and An Andre rew Zisse isserma rman Ma Marc rc Po Polle llefeys ys Modified by Philippos Mordohai 11
Homogeneous Coordinates • 3-D points represented as 4-D vectors (X Y Z 1) T • Equality defined up to scale – (X Y Z 1) T ~ (WX WY WZ W) T • Useful for perspective projection à makes equations linear m M 1 C M 2 12
Pinhole camera model T T ( X , Y , Z ) � ( fX / Z , fY / Z ) X X ⎛ ⎞ ⎛ ⎞ fX f 0 ⎜ ⎟ ⎜ ⎟ ⎛ ⎞ ⎡ ⎤ Y Y ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎢ ⎥ � fY f 0 = ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎢ ⎥ Z Z ⎜ ⎟ Z 1 0 ⎜ ⎟ ⎜ ⎟ ⎢ ⎥ ⎝ ⎠ ⎣ ⎦ ⎜ ⎟ ⎜ ⎟ 1 1 ⎝ ⎠ ⎝ ⎠ linear projection in homogeneous coordinates! 13
The Pinhole Camera fX x = Z fY y = Z 14
Principal Point Offset T T � ( X , Y , Z ) ( fX / Z p , fY / Z p ) + + x y T ( p x p , ) principal point y X X ⎛ ⎞ ⎛ ⎞ fX Zp f p 0 ⎜ ⎟ ⎜ ⎟ + ⎛ ⎞ ⎡ ⎤ x x Y Y ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎢ ⎥ � fY Zp f p 0 + = ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ y y ⎢ ⎥ Z Z ⎜ ⎟ Z 1 0 ⎜ ⎟ ⎜ ⎟ ⎢ ⎥ ⎝ ⎠ ⎣ ⎦ ⎜ ⎟ ⎜ ⎟ 1 1 ⎝ ⎠ ⎝ ⎠ 15
Principal Point Offset [ ] x = K I | 0 X cam X ⎛ ⎞ fX Zp f p 0 + ⎜ ⎟ ⎛ ⎞ ⎡ ⎤ x x Y ⎜ ⎟ ⎜ ⎟ ⎢ ⎥ fY Zp f p 0 + = ⎜ ⎟ ⎜ ⎟ x y ⎢ ⎥ Z ⎜ ⎟ Z 1 0 ⎜ ⎟ ⎢ ⎥ ⎝ ⎠ ⎣ ⎦ ⎜ ⎟ 1 ⎝ ⎠ f p ⎡ ⎤ x calibration matrix ⎢ ⎥ K f p = y ⎢ ⎥ 1 ⎢ ⎥ ⎣ ⎦ 16
Hands On: Image Formation • For a 640 by 480 image with focal length equal to 640 pixels, find 3D points that are marginally visible at the four borders of the image • Increase and decrease the focal length. What happens? 17
Camera Rotation and Translation ~ ~ ~ ( ) X R X - C cam = X ⎛ ⎞ ⎜ ⎟ ~ ~ Y R R C R R C ⎡ ⎤ ⎡ ⎤ − − ⎜ ⎟ X cam X = = ⎢ ⎥ ⎢ ⎥ ⎜ ⎟ Z 0 1 0 1 ⎣ ⎦ ⎣ ⎦ ⎜ ⎟ ⎜ ⎟ 1 ⎝ ⎠ 18
Camera Rotation and Translation X ⎛ ⎞ ⎜ ⎟ ~ ~ Y R R C R R C ⎡ ⎤ ⎡ ⎤ − − ⎜ ⎟ X cam X = = ⎢ ⎥ ⎢ ⎥ ⎜ ⎟ Z 0 1 0 1 x = PX ⎣ ⎦ ⎣ ⎦ ⎜ ⎟ ⎜ ⎟ 1 ⎝ ⎠ [ ] P = K R | t [ ] x = K I | 0 X ~ cam t R C = − ~ [ ] X x KR I | C = − 19
Intrinsic Parameters f s c af f cos( s ) u ⎡ ⎤ ⎡ ⎤ x x o ⎢ ⎥ ⎢ ⎥ or K f v K f c = = o ⎢ ⎥ y y ⎢ ⎥ 1 ⎢ ⎥ 1 ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ • Camera deviates from pinhole γ f sf x ⎡ ⎤ 0 s : skew ⎢ ⎥ K f y f x ≠ f y : different magnification in x and = y 0 ⎢ ⎥ (c x c y ): optical axis does not pierce 1 ⎢ ⎥ ⎣ ⎦ image plane exactly at the center • Usually: rectangular pixels: s = 0 square pixels: f f = principal point known: x y w h ⎛ ⎞ ( ) c , c , = ⎜ ⎟ x y 2 2 ⎝ ⎠ 20
Extrinsic Parameters R t ⎡ ⎤ (3x3) (3x1) M = Scene motion ⎢ ⎥ 0 1 ⎣ ⎦ (1x3) T T R - (R t) ⎡ ⎤ (3x3) 3x1 Camera motion M' = ⎢ ⎥ 0 1 ⎢ ⎥ ⎣ ⎦ (1x3) 21
Projection matrix • Includes coordinate transformation and camera intrinsic parameters X ⎡ ⎤ x p p p p ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ 11 12 13 14 Y ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ y p p p p λ = 21 22 23 24 ⎢ ⎥ ⎢ ⎥ Z ⎢ ⎥ 1 p p p p ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ 31 32 33 34 1 ⎣ ⎦ • Everything we need to know about a pinhole camera • Unambiguous • Can be decomposed into parameters 22
Projection matrix • Mapping from 2-D to 3-D is a function of internal and external parameters X ⎡ ⎤ x f s c ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ x x Y ⎢ ⎥ ⎢ ⎥ [ ] T T ⎢ ⎥ λ y 0 f c R | R t = − y y ⎢ ⎥ ⎢ ⎥ Z ⎢ ⎥ 1 0 0 1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ 1 ⎣ ⎦ [ ] X T − T λ x K R | R t = λ x = PX 23
Hands On: Camera Motion • Choose a few 3D points visible to a camera at the origin. (f=500, w=500, h=500) • Now, move the camera by 2 units of length on the z axis. What happens to the images of the points? • Rotate the points by 45 degrees about the z axis of the camera and then translate them by 5 units on the z axis away from the camera. What are the new images of the points? 24
Projective Transformations in 2D Definition: A projectivity is an invertible mapping h from P 2 to itself such that three points x 1 ,x 2 ,x 3 lie on the same line if and only if h (x 1 ), h (x 2 ), h (x 3 ) do. Theorem: A mapping h : P 2 → P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 reprented by a vector x it is true that h (x)= H x Definition: Projective transformation x ' h h h x ⎛ ⎞ ⎡ ⎤ ⎛ ⎞ 1 11 12 13 1 ⎜ ⎟ ⎜ ⎟ ⎢ ⎥ x ' h h h x x' H x = or = ⎜ ⎟ ⎜ ⎟ 2 21 22 23 2 ⎢ ⎥ ⎜ ⎟ ⎜ ⎟ x ' h h h x ⎢ ⎥ 8DOF ⎝ ⎠ ⎣ ⎦ ⎝ ⎠ 3 31 32 33 3 projectivity=collineation=projective transformation=homography 25
Mapping between planes central projection may be expressed by x’=Hx (application of theorem) 26
Removing Projective Distortion select four points in a plane with known coordinates x ' h x h y h x ' h x h y h + + + + x ' 1 11 12 13 y ' 2 21 22 23 = = = = x ' h x h y h x ' h x h y h + + + + 3 31 32 33 3 31 32 33 ( ) x ' h x h y h h x h y h + + = + + 31 32 33 11 12 13 (linear in h ij ) y ' ( h x h y h ) h x h y h + + = + + 31 32 33 21 22 23 (2 constraints/point, 8DOF ⇒ 4 points needed) Remarks: no calibration at all necessary, better ways to compute (see later) 27
A Hierarchy of Transformations Projective linear group Affine group (last row (0,0,1)) Euclidean group (upper left 2x2 orthogonal) Oriented Euclidean group (upper left 2x2 det 1) Alternatively, characterize transformation in terms of elements or quantities that are preserved or invariant e.g. Euclidean transformations leave distances unchanged 28
Class I: Isometries ( iso =same, metric =measure) x ' cos sin t x ε θ − θ ⎛ ⎞ ⎡ ⎤ ⎛ ⎞ x ⎜ ⎟ ⎜ ⎟ ⎢ ⎥ y ' sin cos t y 1 = ε θ θ ε = ± ⎜ ⎟ ⎜ ⎟ y ⎢ ⎥ ⎜ ⎟ ⎜ ⎟ 1 0 0 1 1 ⎢ ⎥ ⎝ ⎠ ⎣ ⎦ ⎝ ⎠ orientation preserving: 1 ε = 1 orientation reversing: ε = − t R ⎡ ⎤ x' H E x x T R R I = = = ⎢ ⎥ T 1 0 ⎣ ⎦ 3DOF (1 rotation, 2 translation) special cases: pure rotation, pure translation Invariants: length, angle, area 29
Recommend
More recommend
