Lecture 3: Cameras II Justin Johnson EECS 442 WI 2020: Lecture 3 - - - PowerPoint PPT Presentation

▶
lecture 3 cameras ii
SMART_READER_LITE
LIVE PREVIEW

Lecture 3: Cameras II Justin Johnson EECS 442 WI 2020: Lecture 3 - - - PowerPoint PPT Presentation

Nov 24, 2023 186 likes •739 views

Lecture 3: Cameras II Justin Johnson EECS 442 WI 2020: Lecture 3 - 1 January 16, 2020 Administrative HW0 is released will be due Friday 1/24 at 11:59pm Justin Johnson EECS 442 WI 2020: Lecture 3 - 2 January 16, 2020 Administrative HW0

slide-1
SLIDE 1

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 -

Lecture 3: Cameras II

1

slide-2
SLIDE 2

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 -

Administrative

2

HW0 is released will be due Friday 1/24 at 11:59pm

slide-3
SLIDE 3

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 -

Administrative

3

HW0 is released will be due Friday 1/24 Wednesday 1/29 at 11:59pm (Had to split Cameras into 2 lectures; this makes HW0 due after linear algebra lectures)

slide-4
SLIDE 4

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 -

Recap: Pinhole Camera Model

4

O P X (x,y,z)

Coordinate system: O is origin, XY in image, Z sticks out. XY is image plane, Z is optical axis.

z x y f

(x,y,z) projects to (fx/z,fy/z) via similar triangles

Source: L Lazebnik

Focal length

slide-5
SLIDE 5

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 -

Recap: Homogenous Coordinates

5

Trick: add a dimension!

This also clears up lots of nasty special cases

Physical Point

𝑦 𝑧

Homogeneous Point

𝑣 𝑤 𝑥

Concat w=1 Divide by w

𝑣/𝑥 𝑤/𝑥

Physical Point

slide-6
SLIDE 6

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 -

Recap: Homogenous Coordinates

6

z x y [x,y,w] λ[x,y,w]

Two homogeneous coordinates are equivalent if they are proportional to each other. Not = !

𝑣 𝑤 𝑥 ≡ 𝑣( 𝑤( 𝑥( ↔ 𝑣 𝑤 𝑥 = 𝜇 𝑣( 𝑤( 𝑥( 𝜇 ≠ 0

Triple / Equivalent Double / Equals

slide-7
SLIDE 7

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 7

Recap: Projection Matrix

Projection (x, y, z) -> (fx/z, fy/z) is matrix multiplication

𝑔 𝑔 1 𝒚 𝒛 𝒜 𝟐 = 𝑔𝑦 𝑔𝑧 𝑨 ≡ 𝑔𝑦/𝑨 𝑔𝑧/𝑨 1 O f

Slide inspired from L. Lazebnik

3D homogenous point 2D homogenous point

slide-8
SLIDE 8

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 -

Recap: Perspective Model

8

𝑸 ≡ 𝑔 𝑣6 𝑔 𝑤6 1 𝑺898 𝒖89; 𝒀=9; Intrinsic Matrix K Extrinsic Matrix [R,t]

𝑸 ≡ 𝑳 𝑺 | 𝒖 𝒀 ≡ 𝑵89=𝒀=9;

Nice interactive demo: http://ksimek.github.io/2012/08/22/extrinsic/

slide-9
SLIDE 9

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 9

Pinhole Model: Big Issue

Photosensitive Material

Film captures all the rays going through a point (a pencil of rays). How big is a point?

Slide inspired by S. Seitz; image from Michigan Engineering

slide-10
SLIDE 10

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 10

Math vs Reality

  • Math: Any point projects to one point
  • Reality
  • Don’t image points behind the camera / objects
  • Don’t have an infinite amount of sensor material
  • Other issues
  • Light is limited
  • Spooky stuff happens with infinitely small holes
slide-11
SLIDE 11

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 11

Limitations of Pinhole Model

Ideal Pinhole 1 point generates 1 image Low-light levels Finite Pinhole 1 point generates region Blurry. Why is it blurry?

Slide inspired by M. Hebert

slide-12
SLIDE 12

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 12

Limitations of Pinhole Model

Slide Credit: S. Seitz

Small pinhole gives sharper image (but also needs longer exposure time) When pinhole is too small, diffraction effects take over!

slide-13
SLIDE 13

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 13

Adding a Lens

  • A lens focuses light onto the film
slide-14
SLIDE 14

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 14

Adding a Lens: Thin Lens Model

  • A lens focuses light onto the film
  • Thin lens model:
  • Rays passing through the center are not deviated

(pinhole projection model still holds)

slide-15
SLIDE 15

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 15

Adding a Lens: Thin Lens Model

  • A lens focuses light onto the film
  • Thin lens model:
  • Rays passing through the center are not deviated

(pinhole projection model still holds)

  • All rays parallel to the optical axis pass through

the focal point focal point f

slide-16
SLIDE 16

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 16

What’s the catch?

“circle of confusion”

  • There’s a distance where objects are “in focus”
  • Other points project to a “circle of confusion”
slide-17
SLIDE 17

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 17

Circle of Confusion

Image Source: Wikipedia

Object is too close: Point projects to circle (blurry image) Object is just right: Point projects point (sharp image) Object is too far: Point projects to circle (blurry image)

Question: How can we tell if the object is just right?

slide-18
SLIDE 18

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 18

Thin Lens Formula

  • bject

image plane lens

Diagram credit: F. Durand

focal point

f D Dʹ y

Want relationship between y, D, D’, f that causes the object to be in focus

slide-19
SLIDE 19

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 19

Thin Lens Formula

  • bject

image plane lens

Diagram credit: F. Durand

Thin lens assumptions:

  • 1. Rays through the lens center not deviated
slide-20
SLIDE 20

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 20

Thin Lens Formula

  • bject

image plane lens

Diagram credit: F. Durand

Thin lens assumptions:

  • 1. Rays through the lens center not deviated
  • 2. Rays parallel to the optical axis pass through the focal point

The object is in focus when both rays intersect on the image plane

slide-21
SLIDE 21

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 21

Thin Lens Formula

f D Dʹ

  • bject

image plane lens

y yʹ

Let’s derive the relationship between object distance D, image plane distance D’, and focal length f.

Diagram credit: F. Durand

slide-22
SLIDE 22

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 22

Thin Lens Formula

f D Dʹ

  • bject

image plane lens

One set of similar triangles:

y yʹ

𝑧′ 𝐸( − 𝑔 = 𝑧 𝑔 𝑧′ 𝑧 = 𝐸( − 𝑔 𝑔

Diagram credit: F. Durand

slide-23
SLIDE 23

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 23

Thin Lens Formula

f D Dʹ

  • bject

image plane lens

y yʹ

𝑧′ 𝐸′ = 𝑧 𝐸

Another set of similar triangles:

𝑧′ 𝑧 = 𝐸′ 𝐸

slide-24
SLIDE 24

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 24

Thin Lens Formula

f D Dʹ

  • bject

image plane lens

y yʹ

Set them equal:

𝐸′ 𝐸 = 𝐸′ − 𝑔 𝑔 1 𝐸 + 1 𝐸′ = 1 𝑔

slide-25
SLIDE 25

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 25

Thin Lens Formula

Diagram credit: F. Durand

f D Dʹ

  • bject

image plane lens

1 𝐸 + 1 𝐸′ = 1 𝑔

Suppose I want to take a picture of a lion with D big? Which of D, D’, f are fixed? How do we take pictures of things at different distances?

slide-26
SLIDE 26

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 26

Depth of Field

http://www.cambridgeincolour.com/tutorials/depth-of-field.htm

Slide Credit: A. Efros

slide-27
SLIDE 27

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 27

Controlling Depth of Field

Changing the aperture size affects depth of field A smaller aperture increases the range in which the

  • bject is approximately in focus

Diagram: Wikipedia

slide-28
SLIDE 28

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 28

Controlling Depth of Field

Diagram: Wikipedia

If a smaller aperture makes everything focused, why don’t we just always use it?

slide-29
SLIDE 29

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 29

Varying the Aperture

Slide Credit: A. Efros, Photo: Philip Greenspun

Large aperture = small DOF Small aperture = large DOF

slide-30
SLIDE 30

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 30

Varying the Aperture

slide-31
SLIDE 31

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 31

Field of View

  • Photo. Material

𝜚 = tanI; 𝑒 2𝑔

𝜚 𝑔 𝑒

tan-1 is monotonic increasing. How can I get the FOV bigger?

slide-32
SLIDE 32

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 32

Field of View

Slide Credit: A. Efros

slide-33
SLIDE 33

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 33

Field of View

Slide Credit: A. Efros

slide-34
SLIDE 34

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 34

Field of View and Focal Length

Large FOV, small f Camera close to car Small FOV, large f Camera far from the car

Slide Credit: A. Efros, F. Durand

slide-35
SLIDE 35

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 35

Field of View and Focal Length

standard wide-angle telephoto

Slide Credit: F. Durand

slide-36
SLIDE 36

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 36

Dolly Zoom

Change f and distance at the same time

Video Credit: Goodfellas 1990

slide-37
SLIDE 37

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 37

More Bad News

  • First a pinhole…
  • Then a thin lens model….

Slide: L. Lazebnik

slide-38
SLIDE 38

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 38

Radial Distortion

Photo: Mark Fiala, U. Alberta

Lens imperfections cause distortions as a function of distance from optical axis Less common these days in consumer devices

slide-39
SLIDE 39

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 39

Radial Distortion

  • Photo. Material

r f z

Ideal 𝑧′ = 𝑔 𝑧 𝑨

y' y

Distorted 𝑧′ = (1 + 𝑙;𝑠O + ⋯ ) 𝑧 𝑨

slide-40
SLIDE 40

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 40

Vignetting

  • Photo. Material

Slide inspired by L. Lazebnik Slide

What happens to the light between the black and red lines? Doesn’t make it to the sensor! Image darkens toward the edge

slide-41
SLIDE 41

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 41

Vignetting

Photo credit: Wikipedia (https://en.wikipedia.org/wiki/Vignetting)

slide-42
SLIDE 42

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 42

Spherical Aberration

Lenses don’t focus light perfectly! Rays farther from the optical axis focus closer

Slide: L. Lazebnik

slide-43
SLIDE 43

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 43

Chromatic Aberration

Lens refraction index is a function of the

  • wavelength. Colors “fringe” or bleed

Image credits: L. Lazebnik, Wikipedia

slide-44
SLIDE 44

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 44

Chromatic Aberration

Researchers tried teaching a network about

  • bjects by forcing it to assemble jigsaws.

Slide Credit: C. Doersch

slide-45
SLIDE 45

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 45

From Photon to Photo

  • Each cell in a sensor array is a light-sensitive diode that converts

photons to electrons

  • Dominant in the past: Charge Coupled Device (CCD)
  • Dominant now: Complementary Metal Oxide

Semiconductor (CMOS)

Slide Credit: L. Lazebnik, Photo Credit: Wikipedia, Stefano Meroli

slide-46
SLIDE 46

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 46

From Photon to Photo

  • CCD Problem: Vertical Smear

Slide Credit: L. Lazebnik, Photo Credit: Wikipedia, Stefano Meroli

slide-47
SLIDE 47

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 47

From Photon to Photo

  • CMOS problem: Rolling Shutter

Slide Credit: L. Lazebnik, Photo Credit: Wikipedia, Stefano Meroli

slide-48
SLIDE 48

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 48

From Photon to Photo

Rolling Shutter: pixels read in sequence Can get global reading, but $$$

slide-49
SLIDE 49

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 49

Historic Milestones

  • Pinhole model: Mozi (470-390 BCE),

Aristotle (384-322 BCE)

  • Principles of optics (including lenses):

Alhacen (965-1039 CE)

  • Camera obscura: Leonardo da Vinci

(1452-1519), Johann Zahn (1631-1707)

  • First photo: Joseph Nicephore Niepce (1822)
  • Daguerréotypes (1839)
  • Photographic film (Eastman, 1889)
  • Cinema (Lumière Brothers, 1895)
  • Color Photography (Lumière Brothers, 1908)
  • Television (Baird, Farnsworth, Zworykin, 1920s)
  • First consumer camera with CCD

Sony Mavica (1981)

  • First fully digital camera: Kodak DCS100 (1990)

Slide Credit: S. Lazebnik

Niepce, “La Table Servie,” 1822 Alhacen’s notes Old television camera

slide-50
SLIDE 50

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 50

First Digitally Scanned Photgraph

  • 1957, 176 x 176 pixels

Slide Credit: http://listverse.com/history/top-10-incredible-early-firsts-in-photography/

slide-51
SLIDE 51

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 51

Historic Milestone

Sergey Prokudin-Gorskii (1863-1944) Photographs of the Russian empire (1909-1916)

Slide Credit: S. Maji

slide-52
SLIDE 52

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 52

Historic Milestone

Slide Credit: S. Maji

slide-53
SLIDE 53

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 53

Your Milestone: HW1

Your job in homework 1: Make the left look like the right.

slide-54
SLIDE 54

Justin Johnson January 16, 2020 EECS 442 WI 2020: Lecture 3 - 54

Human Luminance Sensitivity Function

Next Time: Light, Color