SLIDE 1
1
COMP 546
Lecture 7
image motion
- Thurs. Jan. 31, 2018
π¦ π§ π’ π¦ π’ π¦ π§
Time varying images (XYT)
2
image motion Thurs. Jan. 31, 2018 1 Time varying images (XYT) - - PowerPoint PPT Presentation
image motion Thurs. Jan. 31, 2018 1 Time varying images (XYT) - - PowerPoint PPT Presentation
COMP 546 Lecture 7 image motion Thurs. Jan. 31, 2018 1 Time varying images (XYT) 2 Motion in XYT e.g. translating vertical edge 3 Motion in XYT e.g.
SLIDE 2
SLIDE 3
π¦ π§ π’ π¦ π’ π¦ π§
Motion in XYT
e.g. translating vertical edge
3
SLIDE 4
π¦ π§ π’ π¦ π’ π¦ π§
Motion in XYT
e.g. translating vertical bar
4
SLIDE 5
π¦ π§ π’
5
SLIDE 6
6
How do the eye and brain (retina, LGN, V1) measure image motion?
SLIDE 7
7
Photoreceptor response to a brief flash of light
(recall from lecture 3)
0 100 time (ms) flash of light
- 40 mV
- 50 mV
Response
SLIDE 8
Retinal Ganglion and LGN cells
OFF center, ON surround
+ + + + + +
- +
- ON center,
OFF surround
8
Our models up to now have been static only.
SLIDE 9
π¦ π’
XY and XT slices through DOG(x,y,t)
The response at t = 0 depends on the image intensities in the past (t < 0).
9
+
- π¦0
+
- π¦
π§
π¦0 π§0
SLIDE 10
π¦ π’
Space-time separable model
The response at t = 0 depends on the image intensities in the past (t < 0).
10
+
- π¦0
+
- π¦
π§
π¦0 π§0
π¦ π’
π¦0 π π¦, π§, π’ = πΈππ» π¦, π§ π(π’)
SLIDE 11
11
π¦ π’
π¦0
moving π¦ π’
π¦0
π¦ π’
π¦0
static
+
- This cell would
respond better to the static image.
SLIDE 12
π¦ π’
Space-time separable model (with temporal sensitivity)
12
π¦0 +
- π¦
π§
π¦0 π§0
π¦ π’
π¦0
+
- +
+
SLIDE 13
13
+
- +
+
π’
π¦0
π’
π¦0
moving π¦ π’
π¦0
moving π’
π¦0
static
This cell would respond better to motion than to static image.
SLIDE 14
14
+
- +
+
π’
π¦0
+
- +
+
π’
π¦0
This cell would respond better to slow motion. This cell would respond better to fast motion. π¦ π¦
SLIDE 15
For your interest... (not on exam) There are two classes of retinal ganglion cells...
Sensitive to temporal intensity variations. Large receptive fields. Insensitive to temporal intensity variations. Small receptive fields. Sensitive to color.
15
SLIDE 16
LGN
16
Layers 1, 2 Layers 3-6 L R L R R L
β¦ and these two classes map to distinct layers in the LGN.
SLIDE 17
V1 cells can detect oriented structure in XY. What about XT and YT ?
17
cosGabor(x,y) sinGabor(x,y)
SLIDE 18
+
- +
- +
- +
- π¦
π§
Hubel and Wieselβs idea for a simple cell (line detector).
18
SLIDE 19
π¦ π’
Reichardt (1950βs): Temporal delays combined with spatial shifts could produce motion direction sensitivity. Hubel and Wieselβs idea for a simple cell (line detector).
19
+
- +
- +
- +
- π¦
π§
π¦0 π¦0
+
- +
- +
SLIDE 20
π¦ π’
Alternatively, using time varying DOGs:
20
π¦0
+
- +
+
- +
+
- +
+
- +
+ +
- + - +
SLIDE 21
π’
Orientation and direction tuning in V1
21
π§
+ -
- +
+
Cell is selective for vertically line moving to the left. π¦ π¦
π¦0 π¦0
SLIDE 22
XT slice through receptive field profile
- f V1 cell
22
π’ π¦
π¦0
SLIDE 23
Simple and Complex Cells are Orientation Tuned (XY slice) and many are binocularly disparity tuned.
23
SLIDE 24
Many are also motion direction and speed tuned.
24
SLIDE 25
π¦ π§ π’ π¦ π’ π¦ π§
Vertical edge moving to the right
25
SLIDE 26
π¦ π§ π’ π¦ π’ π¦ π§
Vertical bar moving to the right
26
SLIDE 27
π¦ π§ π’ π¦ π’ π¦ π§
Vertically 2D sine moving to the right (wave)
27
SLIDE 28
Recall: 2D sine
28
sin 2π π (ππ¦ π¦ + ππ§ π§) π. π. ππ¦ = 8 ππ§ = 2 π = 256
SLIDE 29
sine wave in XYT
29
sin 2π π (ππ¦ π¦ + ππ§ π§) + 2π π π π’ Temporal frequency (cycles per π frames) Spatial frequency (cycles per π pixels)
Exercise: what is the speed of the wave?
SLIDE 30
3D sine in XYT
30
sin 2π π (ππ¦ π¦ + ππ§ π§) + 2π π π π’
2π π (ππ¦ π¦ + ππ§ π§) + 2π π π π’ = c
(
2π π ππ¦, 2π π ππ§, 2π π π) β ( π¦, π§, π’) = c
3D vector normal to the plane is the equation of a plane in XYT.
SLIDE 31
3D sine Gabor
31
sin
2π π (ππ¦ π¦ + ππ§ π§) + 2π π π π’
π»(π¦, π§, π’, ππ¦, ππ§, ππ’)
π¦ π’ π§
SLIDE 32
Normal Velocity
32
V1 cells only respond to the motion component that is normal (perpendicular) to their preferred orientation.
SLIDE 33
V1 : βAperture Problemβ
http://www.opticalillusion.net/optical-illusions/the-barber-pole-illusion/
The same issue arises with single bar, edge, or constant gradient.
33
SLIDE 34
To estimating image velocity (π€π¦ , π€π§) at π¦, π§ , the visual system needs to combine the responses of many V1 (normal velocity) cells.
34