egomotion: translation & rotation, eye movements Thurs. Feb. 8, - - PowerPoint PPT Presentation

β–Ά
egomotion
SMART_READER_LITE
LIVE PREVIEW

egomotion: translation & rotation, eye movements Thurs. Feb. 8, - - PowerPoint PPT Presentation

Apr 09, 2023 285 likes β€’691 views

COMP 546 Lecture 9 egomotion: translation & rotation, eye movements Thurs. Feb. 8, 2018 1 What is the image motion seen by a moving observer? ( egomotion ) Translation Rotation 2 Motion field seen

slide-1
SLIDE 1

1

COMP 546

Lecture 9

egomotion:

translation & rotation, eye movements

  • Thurs. Feb. 8, 2018
slide-2
SLIDE 2

What is the image motion seen by a moving observer? (β€œegomotion”)

2

Translation Rotation

slide-3
SLIDE 3

The Yosemite sequence

Motion field seen by moving observer

For each image location (𝑦, 𝑧), there is a velocity (𝑀𝑦, 𝑀𝑧).

3

(Last lecture, I discussed how to estimate image velocities.)

slide-4
SLIDE 4

Motion field seen by a translating observer

4

(π‘ˆ

π‘Œ, π‘ˆπ‘, π‘ˆπ‘Ž)

π‘Œ π‘Ž 𝑍

eye π‘Œ0, 𝑍

0, π‘Ž0

slide-5
SLIDE 5

The path of the scene point in the eye’s coordinate system is: The relative 3D velocity of the scene point (βˆ’π‘ˆ

π‘Œ, βˆ’π‘ˆπ‘, βˆ’π‘ˆπ‘Ž)

5

π‘Œ 𝑒 , 𝑍 𝑒 , π‘Ž(𝑒) = π‘Œ0 βˆ’ π‘ˆ

𝑦 𝑒, 𝑍 0 βˆ’ π‘ˆ 𝑧 𝑒, π‘Ž0 βˆ’ π‘ˆ 𝑨 𝑒

(π‘ˆ

π‘Œ, π‘ˆπ‘, π‘ˆπ‘Ž)

π‘Œ π‘Ž 𝑍

eye π‘Œ0, 𝑍

0, π‘Ž0

slide-6
SLIDE 6

What is the image path of the scene point? (π‘Œ0, π‘Ž0) (π‘ˆ

π‘Œ, π‘ˆπ‘Ž)

π‘Ž = 𝑔 π‘Œ π‘Ž

𝑦 𝑒 𝑔 = π‘Œ(𝑒) π‘Ž(𝑒) = π‘Œπ‘ βˆ’ π‘ˆ

π‘Œ 𝑒

π‘Žπ‘ βˆ’ π‘ˆπ‘Ž 𝑒

6

Notation: here 𝑦 𝑒 is a position in the plane π‘Ž = 𝑔.

slide-7
SLIDE 7

𝑀𝑦

What is the image velocity of the scene point? (π‘Œ0, π‘Ž0) (π‘ˆ

π‘Œ, π‘ˆπ‘Ž)

π‘Ž = 𝑔 π‘Œ π‘Ž

𝑀𝑦 = 𝑒 𝑒𝑒 𝑦 𝑒 𝑔 |𝑒=0 = βˆ’ π‘ˆ

π‘Œ π‘Ž0 + π‘ˆπ‘Ž π‘Œ0

π‘Žπ‘

2

7

Notation: here 𝑀𝑦 𝑒 is an angular velocity (radians/sec, assuming small angle approximation) rather than a velocity in the plane π‘Ž = 𝑔.

slide-8
SLIDE 8

(𝑀𝑦, 𝑀𝑧) = βˆ’ π‘ˆ

π‘Œ π‘Ž0 + π‘ˆπ‘Ž π‘Œ0

π‘Žπ‘

2

, βˆ’ π‘ˆπ‘ π‘Ž0 + π‘ˆπ‘Ž 𝑍 π‘Žπ‘

2

+ π‘ˆπ‘Ž π‘Žπ‘ π‘Œπ‘ π‘Žπ‘ , 𝑍 π‘Žπ‘ = 1 π‘Žπ‘ (βˆ’π‘ˆ

π‘Œ, βˆ’π‘ˆπ‘)

Lateral component Forward component

𝑦 𝑔 , 𝑧 𝑔

8

slide-9
SLIDE 9

Lateral Component (π‘ˆπ‘Ž = 0)

9

𝑀𝑦, 𝑀𝑧 = 1 π‘Žπ‘ (βˆ’π‘ˆ

π‘Œ, βˆ’π‘ˆπ‘)

(π‘ˆ

π‘Œ, π‘ˆπ‘)

Example: wall (π‘Ž = 10), square (π‘Ž = 3)

eccentricity (deg.)

slide-10
SLIDE 10

Lateral Component (π‘ˆπ‘Ž = 0)

10

𝑀𝑦, 𝑀𝑧 = 1 π‘Žπ‘ (βˆ’π‘ˆ

π‘Œ, βˆ’π‘ˆπ‘)

(π‘ˆ

π‘Œ, π‘ˆπ‘)

Example: ground plane π‘Ž = βˆ’ π‘”β„Ž 𝑧

eccentricity (deg.)

slide-11
SLIDE 11

Lateral Motion and Balance

Holding this pose is more difficult when looking up than when looking down.

Why?

11

slide-12
SLIDE 12

Dizziness (β€˜height vertigo’)

Not to be confused with a more general β€˜acrophobia’ (fear of heights)

12

slide-13
SLIDE 13

π‘ˆπ‘Ž π‘Žπ‘ 𝑦 𝑔 , 𝑧 𝑔 𝑀𝑦, 𝑀𝑧 =

Forward Component (π‘ˆ

π‘Œ = π‘ˆπ‘ = 0)

13

What does image flow depend on ?

slide-14
SLIDE 14

π‘ˆπ‘Ž π‘Žπ‘ 𝑦 𝑔 , 𝑧 𝑔 𝑀𝑦, 𝑀𝑧 =

Forward Component (π‘ˆ

π‘Œ = π‘ˆπ‘ = 0)

wall π‘Žπ‘ = constant.

14

slide-15
SLIDE 15

π‘ˆπ‘Ž π‘Žπ‘ 𝑦 𝑔 , 𝑧 𝑔 𝑀𝑦, 𝑀𝑧 =

Forward Component (π‘ˆ

π‘Œ = π‘ˆπ‘ = 0)

wall π‘Žπ‘ = constant

15

ground plane (see Exercises)

slide-16
SLIDE 16

What does a pilot see when approaching the runway? (from JJ Gibson 1950)

16

slide-17
SLIDE 17

General Translation (when π‘ˆ

𝑨 β‰  0 )

17

π‘ˆπ‘Ž π‘Žπ‘ 𝑦 𝑔 , 𝑧 𝑔 𝑀𝑦, 𝑀𝑧 =

+ 1 π‘Žπ‘ (βˆ’π‘ˆ

π‘Œ, βˆ’π‘ˆπ‘)

forward lateral = π‘ˆπ‘Ž π‘Žπ‘ 𝑦 𝑔 βˆ’ π‘ˆ

𝑦

π‘ˆ

𝑨

, 𝑧 𝑔 βˆ’ π‘ˆ

𝑧

π‘ˆ

𝑨

slide-18
SLIDE 18

General Translation (when π‘ˆ

𝑨 β‰  0 )

18

π‘ˆπ‘Ž π‘Žπ‘ 𝑦 𝑔 , 𝑧 𝑔 𝑀𝑦, 𝑀𝑧 =

+ 1 π‘Žπ‘ (βˆ’π‘ˆ

π‘Œ, βˆ’π‘ˆπ‘)

forward lateral = π‘ˆπ‘Ž π‘Žπ‘ 𝑦 𝑔 βˆ’ π‘ˆ

𝑦

π‘ˆ

𝑨

, 𝑧 𝑔 βˆ’ π‘ˆ

𝑧

π‘ˆ

𝑨

π‘ˆ

𝑦

π‘ˆ

𝑨

, π‘ˆ

𝑧

π‘ˆ

𝑨

is called the β€œdirection of heading”.

slide-19
SLIDE 19

Example: π‘ˆ

π‘Œ, π‘ˆπ‘, π‘ˆπ‘Ž = (.3, 0, 1)

19

Eccentricity (deg)

slide-20
SLIDE 20

How can a translating observer estimate heading?

1) Estimate motion field 𝑀𝑦, 𝑀𝑧 . 2) Estimate the direction of heading

π‘ˆ

𝑦

π‘ˆ

𝑨 ,

π‘ˆ

𝑧

π‘ˆ

𝑨

to be the image point where all motion vectors point away from that point.

20

slide-21
SLIDE 21

How/where does the brain solve this problem?

V1: measure normal velocity components with Gabors οƒ  MT (middle temporal lobe): estimate velocities 𝑀𝑦, 𝑀𝑧

οƒ  MST (medial superior temporal lobe): estimate global motion field

V1

21

slide-22
SLIDE 22

MST cell β€˜templates’

(computation model)

22

Huge receptive fields Each vector represents a normal component of velocity (V1 cell). Each disk represents an MT cell (last lecture). [Perrone & Stone, 1998]

slide-23
SLIDE 23

23

COMP 546

Lecture 9

egomotion:

translation & rotation (eye movements)

  • Thurs. Feb. 8, 2018
slide-24
SLIDE 24

Motion field seen by a rotating observer ?

24

slide-25
SLIDE 25

25

slide-26
SLIDE 26

26

slide-27
SLIDE 27

27

slide-28
SLIDE 28

28

slide-29
SLIDE 29

Oculomotor nerve

… and its various branches

Mid-brain Thalamus (where LGN is) All eye movement motor (output) signals come from mid-brain e.g. oculomotor nerves. These nerves also control accommodation, blinks, pupil contraction.

29

slide-30
SLIDE 30

Types of eye movements

  • vestibulo-ocular reflex (VOR)
  • smooth pursuit
  • vergence (next lecture)
  • saccades (later in course)
  • OKN (optokinetic nystagmus) OMIT

30

slide-31
SLIDE 31

VOR

(eye rotations due to head movement)

31

slide-32
SLIDE 32

Vestibular System

(in the inner ear)

32

slide-33
SLIDE 33

Vestibular system: the brain’s IMU (inertial measurement unit – a term used in robotics)

It measures:

  • linear acceleration of head

𝑒 𝑒𝑒

π‘ˆ

π‘Œ, π‘ˆπ‘, π‘ˆ 𝑨

  • angular acceleration of head

𝑒 𝑒𝑒 (pan, tilt, roll)

33

slide-34
SLIDE 34

Rotation (angular acceleration)

34

Translation (linear acceleration)

  • toliths
slide-35
SLIDE 35

Translation

Vestibular System

(in the inner ear) Rotation Hearing

35

slide-36
SLIDE 36

Smooth Pursuit Eye Movements

Tracking a static object as the observer moves (or smoothly tracking a moving object) Reduces retinal motion of object to 0.

36

slide-37
SLIDE 37

Combined translation and rotation

37

translation only rotation only translation + rotation Zero image velocity (tracking) Q: where is direction of heading?

slide-38
SLIDE 38

Combined translation and rotation

38

Zero image velocity (tracking) translation only rotation only translation + rotation Direction of heading

slide-39
SLIDE 39

Discussion/Summary

  • Motion field seen by moving observer is the sum of translation & rotation

fields

  • Cells in MST are sensitive to global motion fields (huge receptive fields) and

are believed to be involved in estimating egomotion. [It is not entirely clear what exactly the computational role of these cells is. But there is a lot of evidence that these cells exist, and that they do play a role in estimating heading direction.]

  • VOR adds eye rotation to cancel the image motion that is due to head
  • motion. Smooth pursuit adds eye rotation that reduces the retinal velocity
  • f certain β€œinterest points” to zero, allowing a detailed β€œstill” image analysis.

In both cases, these rotations are known (controlled by the brain) so their effects can be accounted for, i.e. the translation and rotation components of the motion field can be disentangled.

39