Dynamic Perspective CS 543 / ECE 549 Saurabh Gupta Spring 2020, - - PowerPoint PPT Presentation

Dynamic Perspective

CS 543 / ECE 549 – Saurabh Gupta Spring 2020, UIUC

http://saurabhg.web.illinois.edu/teaching/ece549/sp2020/

Many slides adapted from J. Malik.

Perspective Projection

𝑌, 𝑍, 𝑎 →

&' ( , &) (

f Z P y O p Y

Suppose the camera moves with respect to the world…

• Point P (𝑌, 𝑍, 𝑎) in the world moves relative to the

camera, its projection in the image (𝑦, 𝑧) moves as well.

• This movement in the image plane is called optical flow.
• Suppose the image of the point (𝑦, 𝑧) moves to

(𝑦 + ∆𝑦, 𝑧 + ∆𝑧) in time ∆𝑢, then 12

13 , 14 13

are the two components of the optical flow.

Outline

• Relate optical flow to camera motion
• Special cases

How does a point X in the scene move?

• Assume that the camera moves with a

translational velocity 𝑢 = (𝑢2, 𝑢4, 𝑢6) and angular velocity 𝜕 = 𝜕2, 𝜕4, 𝜕6 .

• Linear velocity of point 𝑄 = 𝑌, 𝑍, 𝑎 is given

by ̇ 𝑄 = −𝑢 − 𝜕×𝑄. ̇ 𝑌 ̇ 𝑍 ̇ 𝑎 = − 𝑢2 𝑢4 𝑢6 − 𝜕4𝑎 − 𝜕6𝑍 𝜕6𝑌 − 𝜕2𝑎 𝜕2𝑍 − 𝜕4𝑌

Now, lets consider the effect of projection

̇ 𝑌 ̇ 𝑍 ̇ 𝑎 = − 𝑢2 𝑢4 𝑢6 − 𝜕4𝑎 − 𝜕6𝑍 𝜕6𝑌 − 𝜕2𝑎 𝜕2𝑍 − 𝜕4𝑌

• Assume, 𝑔 = 1, 𝑦 =

' ( , 𝑧 = ) ( .

• ̇

𝑦 =

̇ '(@ ̇ (' (A

, ̇ 𝑧 =

̇ )(@ ̇ () (A

• Substitute ̇

𝑌, ̇ 𝑍, ̇ 𝑎 , from equation above:

𝑣 𝑤 = ̇ 𝑦 ̇ 𝑧 = 1 𝑎 −1 𝑦 −1 𝑧 𝑢2 𝑢4 𝑢6 + 𝑦𝑧 − 1 + 𝑦E 𝑧 (1 + 𝑧E) −𝑦𝑧 −𝑦 𝜕2 𝜕4 𝜕6

Dynamic Perspective Equations

𝑣 𝑤 = ̇ 𝑦 ̇ 𝑧 = 1 𝑎 −1 𝑦 −1 𝑧 𝑢2 𝑢4 𝑢6 + 𝑦𝑧 − 1 + 𝑦E 𝑧 (1 + 𝑧E) −𝑦𝑧 −𝑦 𝜕2 𝜕4 𝜕6 Translation Component Rotation Component

Optical flow for pure rotation

𝑣 𝑤 = ̇ 𝑦 ̇ 𝑧 = 1 𝑎 −1 𝑦 −1 𝑧 𝑢2 𝑢4 𝑢6 + 𝑦𝑧 − 1 + 𝑦E 𝑧 (1 + 𝑧E) −𝑦𝑧 −𝑦 𝜕2 𝜕4 𝜕6

• 𝑣

𝑤 = 𝑦𝑧 − 1 + 𝑦E 𝑧 (1 + 𝑧E) −𝑦𝑧 −𝑦 𝜕2 𝜕4 𝜕6

• We can determine 𝜕 from the flow field.
• Flow field is independent of 𝑎(𝑦, 𝑧).

Optical flow for pure translation along Z-axis

𝑣 𝑤 = ̇ 𝑦 ̇ 𝑧 = 1 𝑎 −1 𝑦 −1 𝑧 𝑢2 𝑢4 𝑢6 + 𝑦𝑧 − 1 + 𝑦E 𝑧 (1 + 𝑧E) −𝑦𝑧 −𝑦 𝜕2 𝜕4 𝜕6

• 𝑣

𝑤 =

3F ((2,4)

𝑦 𝑧

• Optical flow vector is a scalar

multiple of position vector.

• Scale factor ambiguity, if 𝑢6 →

𝑙𝑢6, and 𝑎 → 𝑙𝑎, optical flow remains unchanged.

• But, you can get time to

collision, 𝑎/𝑢6.

Optical flow for general translation

𝑣 𝑤 = ̇ 𝑦 ̇ 𝑧 = 1 𝑎 −1 𝑦 −1 𝑧 𝑢2 𝑢4 𝑢6 + 𝑦𝑧 − 1 + 𝑦E 𝑧 (1 + 𝑧E) −𝑦𝑧 −𝑦 𝜕2 𝜕4 𝜕6

• 𝑣 = @3IJ23F

((2,4) , v = @3LJ43F ((2,4)

Optical flow for points on a road

Slide by J. Malik

Translating along X-axis in front of a wall

Slide by J. Malik

Estimating Optical Flow from Images

http://en.wikipedia.org/wiki/Barberpole_illusion

Estimating Optical Flow from Images

http://en.wikipedia.org/wiki/Barberpole_illusion

Estimating Optical Flow from Images

Aperture Problem

Recap

• Relate optical flow to camera motion
• Special cases
• Pure rotation / pure translation / time to collision

𝑣 𝑤 = ̇ 𝑦 ̇ 𝑧 = 1 𝑎 −1 𝑦 −1 𝑧 𝑢2 𝑢4 𝑢6 + 𝑦𝑧 − 1 + 𝑦E 𝑧 (1 + 𝑧E) −𝑦𝑧 −𝑦 𝜕2 𝜕4 𝜕6