Dynamic Perspective CS 543 / ECE 549 Saurabh Gupta Spring 2020, - PowerPoint PPT Presentation

Dynamic Perspective CS 543 / ECE 549 – Saurabh Gupta Spring 2020, UIUC http://saurabhg.web.illinois.edu/teaching/ece549/sp2020/ Many slides adapted from J. Malik.

Perspective Projection P Y &' ( , &) 𝑌, 𝑍, 𝑎 → ( Z f O y Suppose the camera moves with p respect to the world… • Point P (𝑌, 𝑍, 𝑎) in the world moves relative to the camera, its projection in the image (𝑦, 𝑧) moves as well. • This movement in the image plane is called optical flow. • Suppose the image of the point (𝑦, 𝑧) moves to (𝑦 + ∆𝑦, 𝑧 + ∆𝑧) in time ∆𝑢 , then 12 13 , 14 are the two 13 components of the optical flow.

Outline • Relate optical flow to camera motion • Special cases

̇ ̇ ̇ How does a point X in the scene move? • Assume that the camera moves with a translational velocity 𝑢 = (𝑢 2 , 𝑢 4 , 𝑢 6 ) and angular velocity 𝜕 = 𝜕 2 , 𝜕 4 , 𝜕 6 . • Linear velocity of point 𝑄 = 𝑌, 𝑍, 𝑎 is given 𝑄 = −𝑢 − 𝜕×𝑄. by ̇ 𝜕 4 𝑎 − 𝜕 6 𝑍 𝑢 2 𝑌 𝑢 4 𝜕 6 𝑌 − 𝜕 2 𝑎 = − − 𝑍 𝑢 6 𝜕 2 𝑍 − 𝜕 4 𝑌 𝑎

̇ ̇ ̇ ̇ ̇ ̇ ̇ ̇ Now, lets consider the effect of projection 𝜕 4 𝑎 − 𝜕 6 𝑍 𝑢 2 𝑌 𝑢 4 𝜕 6 𝑌 − 𝜕 2 𝑎 = − − 𝑍 𝑢 6 𝜕 2 𝑍 − 𝜕 4 𝑌 𝑎 ' ) • Assume, 𝑔 = 1 , 𝑦 = ( , 𝑧 = ( . '(@ ̇ )(@ ̇ (' () • 𝑦 = 𝑧 = , ̇ ( A ( A 𝑌, ̇ 𝑍, ̇ 𝑎 , from equation above: • Substitute ̇ 𝜕 2 𝑢 2 − 1 + 𝑦 E 𝑧 = 1 𝑦𝑧 𝑧 𝑣 𝑦 −1 0 𝑦 𝑢 4 𝜕 4 𝑤 = + 0 −1 𝑧 (1 + 𝑧 E ) 𝑎 −𝑦𝑧 −𝑦 𝜕 6 𝑢 6

̇ ̇ Dynamic Perspective Equations 𝑢 2 𝜕 2 − 1 + 𝑦 E 𝑧 = 1 𝑦𝑧 𝑧 𝑣 𝑦 −1 0 𝑦 𝑢 4 𝜕 4 𝑤 = + 0 −1 𝑧 (1 + 𝑧 E ) 𝑎 −𝑦𝑧 −𝑦 𝜕 6 𝑢 6 Translation Component Rotation Component

̇ ̇ Optical flow for pure rotation 𝑢 2 𝜕 2 − 1 + 𝑦 E 𝑧 = 1 𝑦𝑧 𝑧 𝑣 𝑦 −1 0 𝑦 𝑢 4 𝜕 4 𝑤 = + 0 −1 𝑧 (1 + 𝑧 E ) 𝑎 −𝑦𝑧 −𝑦 𝜕 6 𝑢 6 𝜕 2 − 1 + 𝑦 E 𝑦𝑧 𝑧 𝑣 𝜕 4 𝑤 = • (1 + 𝑧 E ) −𝑦𝑧 −𝑦 𝜕 6 • We can determine 𝜕 from the flow field. • Flow field is independent of 𝑎(𝑦, 𝑧) .

̇ ̇ Optical flow for pure translation along Z-axis 𝑢 2 𝜕 2 − 1 + 𝑦 E 𝑧 = 1 𝑦𝑧 𝑧 𝑣 𝑦 −1 0 𝑦 𝑢 4 𝜕 4 𝑤 = + 0 −1 𝑧 (1 + 𝑧 E ) 𝑎 −𝑦𝑧 −𝑦 𝜕 6 𝑢 6 𝑦 𝑣 3 F • 𝑤 = 𝑧 ((2,4) • Optical flow vector is a scalar multiple of position vector. • Scale factor ambiguity, if 𝑢 6 → 𝑙𝑢 6 , and 𝑎 → 𝑙𝑎 , optical flow remains unchanged. • But, you can get time to collision, 𝑎/𝑢 6 .

̇ ̇ Optical flow for general translation 𝑢 2 𝜕 2 − 1 + 𝑦 E 𝑧 = 1 𝑦𝑧 𝑧 𝑣 𝑦 −1 0 𝑦 𝑢 4 𝜕 4 𝑤 = + 0 −1 𝑧 (1 + 𝑧 E ) 𝑎 −𝑦𝑧 −𝑦 𝜕 6 𝑢 6 @3 L J43 F • 𝑣 = @3 I J23 F ((2,4) , v = ((2,4)

Optical flow for points on a road Slide by J. Malik

Translating along X-axis in front of a wall Slide by J. Malik

Estimating Optical Flow from Images http://en.wikipedia.org/wiki/Barberpole_illusion

Estimating Optical Flow from Images http://en.wikipedia.org/wiki/Barberpole_illusion

Estimating Optical Flow from Images Aperture Problem

̇ ̇ Recap • Relate optical flow to camera motion 𝑢 2 𝜕 2 − 1 + 𝑦 E 𝑧 = 1 𝑦𝑧 𝑧 𝑣 𝑦 −1 0 𝑦 𝑢 4 𝜕 4 𝑤 = + 0 −1 𝑧 (1 + 𝑧 E ) 𝑎 −𝑦𝑧 −𝑦 𝜕 6 𝑢 6 • Special cases • Pure rotation / pure translation / time to collision