depth from X: defocus blur and binocular disparity Tues. Feb. 13, - PowerPoint PPT Presentation

COMP 546 Lecture 10 depth from X: defocus blur and binocular disparity Tues. Feb. 13, 2018 1

Depth from defocus blur 2

The eye controls 𝑔 𝑚𝑓𝑜𝑡 , so it “knows” 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 . 1 1 1 1 𝑑𝑝𝑠𝑜𝑓𝑏 + 𝑔 𝑚𝑓𝑜𝑡 = 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 + 𝑎 𝑡𝑓𝑜𝑡𝑝𝑠 𝑔 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 3

Recall: Depth and Defocus Blur 𝑎 0 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 1 − 1 blur width = aperture 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 𝑎 0 (See Exercise 2 Q 6 & Assignment 1) 4

Estimating Depth from Defocus Blur 𝑎 0 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 1 − 1 blur width = aperture 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 𝑎 0 5 2.) measured 1.) known 3.) estimated (details omitted) (controlled by vision system)

Estimating Depth from Defocus Blur 𝑎 0 ? 𝑎 0 ? 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 1 − 1 blur width = aperture 𝑎 𝑔𝑝𝑑𝑏𝑚𝑞𝑚𝑏𝑜𝑓 𝑎 0 There is a two-fold depth ambiguity.

Estimating Depth from Defocus Blur Varying accommodation resolves the two-fold ambiguity. 7

Blur on a slanted plane blur focal plane gradient intersection 8

Blur on a slanted plane 𝑎 = 𝑛 𝑍 + 𝑎 0 slanted plane focal plane sensor plane 9

Blur on a slanted plane 𝑎 = 𝑛 𝑍 + 𝑎 0 slanted plane image of slanted plane focal plane sensor plane See lecture notes: blur width radians = 𝐵 𝑛 𝑧 𝑎 0 𝑔 10

ASIDE: tricks with a tilt-shift lens in photography Tilt-shift lens can increase (or decrease) blur gradient slanted plane image of slanted plane focal plane sensor plane 11

Tilt-shift Illusion (Miniaturization) 12

Tilt-shift Illusion (Miniaturization) 13

We next examine how defocus blur is related to binocular disparity. 14

Recall: Binocular disparity and depth 𝑌 𝑠 𝑎 (𝑦 𝑠 , 𝑔) ( 𝑌 0 , 𝑎 0 ) 𝑌 𝑚 𝑈 𝑦 (𝑦 𝑚 , 𝑔 ) 𝑎 𝑦 𝑚 𝑔 − 𝑦 𝑠 𝑈 𝑦 disparity ≡ 𝑔 = 𝑎 0 15

Recall: Binocular vergence 𝜄 𝑠 < 0 ( 𝑌 𝑤 , 𝑎 𝑤 ) 𝑈 𝑦 Vergence point 𝜄 𝑚 > 0 16

Vergence angle 𝜄 𝑤 determines depth 𝜄 𝑠 < 0 ( 𝑌 𝑤 , 𝑎 𝑤 ) 𝜄 𝑚 - 𝜄 𝑠 Vergence point 𝜄 𝑚 > 0 𝑦 𝑚 𝑔 − 𝑦 𝑠 = 𝑈 𝑦 𝜄 𝑤 ≡ 𝜄 𝑚 - 𝜄 𝑠 ≈ 𝑔 𝑎 𝑤 17

Binocular disparity of vergence point is 0 (obvious). 𝜄 𝑠 < 0 ( 𝑌 𝑤 , 𝑎 𝑤 ) 𝑈 𝑦 Vergence point 𝜄 𝑚 > 0 ( 𝑦 𝑚 𝑔 − 𝜄 𝑚 ) − ( 𝑦 𝑠 𝑔 − 𝜄 𝑠 ) = 0 − 0 = 0 18

Binocular disparity depends on distance (diopters) from vergence depth. Near (crossed) Disparity ( 𝑌 𝑑 , 𝑎 𝑑 ) ( 𝑌 𝑤 , 𝑎 𝑤 ) 𝑦 ( 1 1 𝑒𝑗𝑡𝑞𝑏𝑠𝑗𝑢𝑧 = 𝑈 𝑎 𝑑 − 𝑎 𝑤 ) > 0 19

Binocular disparity depends on distance (diopters) from vergence depth. Far (uncrossed) disparity ( 𝑌 𝑣 , 𝑎 𝑣 ) ( 𝑌 𝑤 , 𝑎 𝑤 ) 𝑦 ( 1 1 𝑒𝑗𝑡𝑞𝑏𝑠𝑗𝑢𝑧 = 𝑈 𝑎 𝑣 − 𝑎 𝑤 ) < 0 20

Binocular Fusion (“Cyclopean Vision”) If disparity is sufficiently small, then we fuse the left and right eye images. Otherwise, we perceive two images ( “diplopia”). 21

Panum’s fusional area ( 𝑌 𝑤 , 𝑎 𝑤 ) ( 𝑌 𝑝 , 𝑎 𝑝 ) 1 − 1 disparity ≈ 𝑈 < 𝑒 𝑛𝑏𝑦 𝑦 𝑎 𝑝 𝑎 𝑤 22

Panum’s fusional area in disparity space 𝑒 < 0 𝑒 = 0 𝑦 𝑠𝑗𝑕ℎ𝑢 𝑒 > 0 𝑦 𝑚𝑓𝑔𝑢 23

Binocular disparity and blur 1 1 IOD − disparity = object dist vergence dist interocular distance or 𝑈 𝑌 1 1 − blur width = aperture object dist focal dist 24

Binocular disparity and blur 1 1 IOD − disparity = object dist vergence dist 1 1 − blur width = aperture object dist focal dist If vergence distance = focal distance then | disparity | IOD = aperture ≈ 10 blur 25

Vergence and accommodation systems are coupled. We verge at the same depth as we focus (even if one eye is closed) and vice-versa. 26

Monocular Binocular ≈ focal plane vergence plane

Limitations on 3D cinema vergence plane 28

3D stereo displays can create accommodation-vergence conflict [Hoffman et al 2008] 29

How to study binocular stereo vision ? Neuroscience • Show cats or monkeys different images to the left and right eyes and measure brain activity. Computational modelling • Write a computer program that finds matching points in left and right images, in a biologically plausible way Psychology • Show people different images to the left and right eyes and measure how well the people judge depth. 30

Random Dot Stereogram Bela Julesz , “Binocular depth perception without familiarity cues ” Science 1964. 31

32

33

How to make a random dot stereogram? image for image for left eye right eye 1.) shift patch left 2.) Fill empty patch 34

image for image for left eye right eye background background foreground square right eye only left eye only 35

image for image for perceived depths left eye right eye background background foreground square right eye only left eye only 36

neither eye left eye only right eye only 37

Disparity Space 𝑦 𝑠𝑗𝑕ℎ𝑢 𝑒 = 0 𝑒 < 0 𝑦 𝑚𝑓𝑔𝑢 𝑒 > 0 38

Disparity Space 𝑦 𝑠𝑗𝑕ℎ𝑢 𝑦 𝑚𝑓𝑔𝑢 Q: where are the eyes verging? 39