Lecture 3: Cameras II Justin Johnson EECS 442 WI 2020: Lecture 3 - - PowerPoint PPT Presentation

Lecture 3: Cameras II Justin Johnson EECS 442 WI 2020: Lecture 3 - 1 January 16, 2020

Administrative HW0 is released will be due Friday 1/24 at 11:59pm Justin Johnson EECS 442 WI 2020: Lecture 3 - 2 January 16, 2020

Administrative HW0 is released will be due Friday 1/24 Wednesday 1/29 at 11:59pm (Had to split Cameras into 2 lectures; this makes HW0 due after linear algebra lectures) Justin Johnson EECS 442 WI 2020: Lecture 3 - 3 January 16, 2020

Recap: Pinhole Camera Model Focal length y f X (x,y,z) z O P x Coordinate system: O is origin, XY in image, Z sticks out. XY is image plane, Z is optical axis. (x,y,z) projects to (fx/z,fy/z) via similar triangles Source: L Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 3 - 4 January 16, 2020

Recap: Homogenous Coordinates Trick: add a dimension! This also clears up lots of nasty special cases Physical Homogeneous Physical Point Point Point 𝑣 𝑦 𝑣/𝑥 𝑤 𝑧 𝑤/𝑥 Concat Divide 𝑥 w=1 by w Justin Johnson EECS 442 WI 2020: Lecture 3 - 5 January 16, 2020

Recap: Homogenous Coordinates λ[x,y,w] Triple / Double / [x,y,w] Equivalent Equals 𝑣 𝑣 𝑣 ( 𝑣 ( y 𝑤 𝑤 ≡ 𝑤 ( ↔ = 𝜇 𝑤 ( 𝑥 𝑥 𝑥 ( 𝑥 ( z 𝜇 ≠ 0 Two homogeneous coordinates are x equivalent if they are proportional to each other. Not = ! Justin Johnson EECS 442 WI 2020: Lecture 3 - 6 January 16, 2020

Recap: Projection Matrix Projection (x, y, z) -> (fx/z, fy/z) is matrix multiplication f O 𝒚 𝑔 0 0 0 𝑔𝑦/𝑨 𝑔𝑦 𝒛 = ≡ 0 𝑔 0 0 𝑔𝑧 𝑔𝑧/𝑨 𝒜 𝑨 0 0 1 0 1 𝟐 3D homogenous 2D homogenous point point Slide inspired from L. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 3 - 7 January 16, 2020

Recap: Perspective Model Intrinsic Extrinsic Matrix K Matrix [R,t] 𝑔 0 𝑣 6 𝑺 898 𝒖 89; 𝑸 ≡ 𝒀 =9; 0 𝑔 𝑤 6 0 0 1 𝑸 ≡ 𝑳 𝑺 | 𝒖 𝒀 ≡ 𝑵 89= 𝒀 =9; Nice interactive demo: http://ksimek.github.io/2012/08/22/extrinsic/ Justin Johnson EECS 442 WI 2020: Lecture 3 - 8 January 16, 2020

Pinhole Model: Big Issue Photosensitive Material Film captures all the rays going through a point (a p encil of rays). How big is a point? Slide inspired by S. Seitz; image from Michigan Engineering Justin Johnson EECS 442 WI 2020: Lecture 3 - 9 January 16, 2020

Math vs Reality • Math: Any point projects to one point • Reality • Don’t image points behind the camera / objects • Don’t have an infinite amount of sensor material • Other issues • Light is limited • Spooky stuff happens with infinitely small holes Justin Johnson EECS 442 WI 2020: Lecture 3 - 10 January 16, 2020

Limitations of Pinhole Model Ideal Pinhole 1 point generates 1 image Low-light levels Finite Pinhole 1 point generates region Blurry. Why is it blurry? Slide inspired by M. Hebert Justin Johnson EECS 442 WI 2020: Lecture 3 - 11 January 16, 2020

Limitations of Pinhole Model Small pinhole gives sharper image (but also needs longer exposure time) When pinhole is too small, diffraction effects take over! Slide Credit: S. Seitz Justin Johnson EECS 442 WI 2020: Lecture 3 - 12 January 16, 2020

Adding a Lens • A lens focuses light onto the film Justin Johnson EECS 442 WI 2020: Lecture 3 - 13 January 16, 2020

Adding a Lens: Thin Lens Model • A lens focuses light onto the film • Thin lens model : • Rays passing through the center are not deviated (pinhole projection model still holds) Justin Johnson EECS 442 WI 2020: Lecture 3 - 14 January 16, 2020

Adding a Lens: Thin Lens Model focal point f • A lens focuses light onto the film • Thin lens model : • Rays passing through the center are not deviated (pinhole projection model still holds) • All rays parallel to the optical axis pass through the focal point Justin Johnson EECS 442 WI 2020: Lecture 3 - 15 January 16, 2020

What’s the catch? “circle of confusion” • There’s a distance where objects are “in focus” • Other points project to a “circle of confusion” Justin Johnson EECS 442 WI 2020: Lecture 3 - 16 January 16, 2020

Circle of Confusion Object is too close: Point projects to circle (blurry image) Object is just right: Point projects point (sharp image) Object is too far: Point projects to circle (blurry image) Question : How can we tell if the object is just right? Image Source: Wikipedia Justin Johnson EECS 442 WI 2020: Lecture 3 - 17 January 16, 2020

Thin Lens Formula Want relationship between y, D, D’, f that causes the object to be in focus D ʹ D f y focal point image object lens plane Diagram credit: F. Durand Justin Johnson EECS 442 WI 2020: Lecture 3 - 18 January 16, 2020

Thin Lens Formula Thin lens assumptions: 1. Rays through the lens center not deviated image object lens plane Diagram credit: F. Durand Justin Johnson EECS 442 WI 2020: Lecture 3 - 19 January 16, 2020

Thin Lens Formula Thin lens assumptions: 1. Rays through the lens center not deviated 2. Rays parallel to the optical axis pass through the focal point The object is in focus when both rays intersect on the image plane image object lens plane Diagram credit: F. Durand Justin Johnson EECS 442 WI 2020: Lecture 3 - 20 January 16, 2020

Thin Lens Formula Let’s derive the relationship between object distance D, image plane distance D’, and focal length f. D ʹ D f y y ʹ image object lens plane Diagram credit: F. Durand Justin Johnson EECS 442 WI 2020: Lecture 3 - 21 January 16, 2020

Thin Lens Formula 𝑧 = 𝐸 ( − 𝑔 𝐸 ( − 𝑔 = 𝑧 𝑧′ 𝑧′ One set of similar triangles: 𝑔 𝑔 D ʹ D f y y ʹ image object lens plane Diagram credit: F. Durand Justin Johnson EECS 442 WI 2020: Lecture 3 - 22 January 16, 2020

Thin Lens Formula 𝑧′ 𝑧 = 𝐸′ 𝐸′ = 𝑧 𝑧′ Another set of similar triangles: 𝐸 𝐸 D ʹ D f y y ʹ image object lens plane Justin Johnson EECS 442 WI 2020: Lecture 3 - 23 January 16, 2020

Thin Lens Formula 𝐸 + 1 1 𝐸′ = 1 𝐸′ 𝐸 = 𝐸′ − 𝑔 Set them equal: 𝑔 𝑔 D ʹ D f y y ʹ image object lens plane Justin Johnson EECS 442 WI 2020: Lecture 3 - 24 January 16, 2020

Thin Lens Formula Suppose I want to take a picture of a lion with D big? Which of D, D’, f are fixed? How do we take pictures of things at different distances? D ʹ D f 𝐸 + 1 1 𝐸′ = 1 𝑔 image lens object plane Diagram credit: F. Durand Justin Johnson EECS 442 WI 2020: Lecture 3 - 25 January 16, 2020

Depth of Field http://www.cambridgeincolour.com/tutorials/depth-of-field.htm Slide Credit: A. Efros Justin Johnson EECS 442 WI 2020: Lecture 3 - 26 January 16, 2020

Controlling Depth of Field Diagram: Wikipedia Changing the aperture size affects depth of field A smaller aperture increases the range in which the object is approximately in focus Justin Johnson EECS 442 WI 2020: Lecture 3 - 27 January 16, 2020

Controlling Depth of Field Diagram: Wikipedia If a smaller aperture makes everything focused, why don’t we just always use it? Justin Johnson EECS 442 WI 2020: Lecture 3 - 28 January 16, 2020

Varying the Aperture Small aperture = large DOF Large aperture = small DOF Slide Credit: A. Efros, Photo: Philip Greenspun Justin Johnson EECS 442 WI 2020: Lecture 3 - 29 January 16, 2020

Varying the Aperture Justin Johnson EECS 442 WI 2020: Lecture 3 - 30 January 16, 2020

Field of View 𝑔 𝜚 Photo. Material 𝑒 𝑒 𝜚 = tan I; 2𝑔 tan -1 is monotonic increasing. How can I get the FOV bigger? Justin Johnson EECS 442 WI 2020: Lecture 3 - 31 January 16, 2020

Field of View Slide Credit: A. Efros Justin Johnson EECS 442 WI 2020: Lecture 3 - 32 January 16, 2020

Field of View Slide Credit: A. Efros Justin Johnson EECS 442 WI 2020: Lecture 3 - 33 January 16, 2020

Field of View and Focal Length Large FOV, small f Camera close to car Small FOV, large f Camera far from the car Slide Credit: A. Efros, F. Durand Justin Johnson EECS 442 WI 2020: Lecture 3 - 34 January 16, 2020

Field of View and Focal Length standard wide-angle telephoto Slide Credit: F. Durand Justin Johnson EECS 442 WI 2020: Lecture 3 - 35 January 16, 2020

Dolly Zoom Change f and distance at the same time Video Credit: Goodfellas 1990 Justin Johnson EECS 442 WI 2020: Lecture 3 - 36 January 16, 2020

More Bad News • First a pinhole… • Then a thin lens model…. Slide: L. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 3 - 37 January 16, 2020

Radial Distortion Lens imperfections cause distortions as a function of distance from optical axis Less common these days in consumer devices Photo: Mark Fiala, U. Alberta Justin Johnson EECS 442 WI 2020: Lecture 3 - 38 January 16, 2020

Radial Distortion y f Photo. Material r z y' Ideal Distorted 𝑧′ = 𝑔 𝑧 𝑧′ = (1 + 𝑙 ; 𝑠 O + ⋯ ) 𝑧 𝑨 𝑨 Justin Johnson EECS 442 WI 2020: Lecture 3 - 39 January 16, 2020