A Computational Mo del for Represen tation of Image V elo - - PowerPoint PPT Presentation

▶
a computational mo del for represen tation of image v elo
SMART_READER_LITE
LIVE PREVIEW

A Computational Mo del for Represen tation of Image V elo - - PowerPoint PPT Presentation

Jan 05, 2024 171 likes •323 views

A Computational Mo del for Represen tation of Image V elo citi es Eero Simoncelli Computer and Information Science Departmen t Univ ersit y of P ennsylv ania Da vid Heeger Psyc hology Departmen t Stanford Univ

slide-1
SLIDE 1 A Computational Mo del for Represen tation
  • f
Image V elo citi es Eero Simoncelli Computer and Information Science Departmen t Univ ersit y
  • f
P ennsylv ania Da vid Heeger Psyc hology Departmen t Stanford Univ ersit y AR V O
slide-2
SLIDE 2 The Mo del

+ ÷ ÷

  • . . .

. . . . . . + +

  • σe
  • 2

Input: normalized STE Input: image intensities Output: normalized STE

σv

  • 2

Stage 1 Stage 2

Output: normalized VE

. . . + ÷ ÷

  • . . .

. . . . . .

  • First
stage computes spatiotemp
  • ral
energy STE
  • Second
stage computes v elo cit y energy VE
  • Simple
computation linear com bination squaring normalization AR V O
slide-3
SLIDE 3 Spatiotemp
  • ral
Energy STE Stage

t y x ωt ωx ωy

  • Eac
h unit is based
  • n
a linear com bination
  • f
image in tensities
  • Unit
resp
  • nse
is lo caliz ed in spatiotemp
  • ral
frequency
  • F
requency domain is co v ered b y a minimal n um b er
  • f
units Interme diate r esp
  • nses
may b e exactly interp
  • late
d AR V O
slide-4
SLIDE 4 STE Unit vs Complex Cell Stim ulus Cell Resp
  • nse
Mo del Resp
  • nse
Grating Plaid
  • P
  • lar
plots
  • f
resp
  • nse
vs stim ulus direction
  • f
mo v emen t
  • Single
cell recordings replotted from Mo vshon et al
  • AR
V O
slide-5
SLIDE 5 STE Unit Is Not V elo cit yT uned

ωt ωx ωy

  • Unit
resp
  • nds
equally w ell to a whole family
  • f
v elo cit i es ap erture problem
  • Equiv
alen tly
  • in
the F
  • urier
domain a family
  • f
planes cut through the unit AR V O
slide-6
SLIDE 6 V elo cit y Energy VE Stage

ωt ωx ωy

vy vx

  • Eac
h unit is based
  • n
a linear com bination
  • f
STE resp
  • nses
as in Albrigh t
  • Smith
et al
  • STE
resp
  • nses
are interp
  • late
d from previous stage
  • Unit
resp
  • nse
is lo caliz ed in v elo cit y space
  • V
elo cit y domain is co v ered b y a minimal n um b er
  • f
units Interme diate r esp
  • nses
may b e exactly interp
  • late
d AR V O
slide-7
SLIDE 7 VE Unit vs MT Cell Stim ulus Cell Resp
  • nse
Mo del Resp
  • nse
Grating Plaid
  • P
  • lar
plots
  • f
resp
  • nse
vs stim ulus direction
  • f
mo v emen t
  • MT
cell recordings replotted from Mo vshon et al
  • AR
V O
slide-8
SLIDE 8 VE Unit vs MT Cell Cell Resp
  • nse
Mo del Resp
  • nse
  • 3
  • 2
  • 1

1 2 3 0.0 0.2 0.4 0.6 0.8 1.0

  • 3
  • 2
  • 1

1 2 3 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

log sp eed log sp eed
  • Resp
  • nse
to an
  • rien
ted bar as a function
  • f
log sp eed
  • MT
cell recordings replotted from Maunsell and V an Essen
  • AR
V O
slide-9
SLIDE 9 VE P
  • pulation
Resp
  • nse
Single Motion

vx vy

v

  • Stim
ulus is a translating random dot pattern
  • F
ull v elo cit yspace is in terp
  • lated
from resp
  • nses
  • f
a small n um b er
  • f
units
  • Resp
  • nse
is unimo dal AR V O
slide-10
SLIDE 10 VE P
  • pulation
Resp
  • nse
T ransparen t Dots

vx vy

  • Stim
ulus is t w
  • random
dot patterns translating in dieren t di rections
  • VE
Resp
  • nse
is bimo dal indicating presence
  • f
t w
  • motions
AR V O
slide-11
SLIDE 11 VE P
  • pulation
Resp
  • nse
T ransparen t Noise P atterns

vx vy

  • Stim
ulus is t w
  • additiv
ely com bined fractal noise patterns mo ving in dieren t directions
  • VE
Resp
  • nse
is bimo dal indicating presence
  • f
t w
  • motions
AR V O
slide-12
SLIDE 12 VE P
  • pulation
Resp
  • nse
Sine Grating Plaids

vx vy vx vy

  • Steep
er plaids lo
  • k
more transparen t
  • Consisten
t with Adelson
  • Mo
vshon
  • AR
V O
slide-13
SLIDE 13 VE P
  • pulation
Resp
  • nse
Square Grating Plaids

vx vy vx vy

  • T
ransparency p ercept is inuenced b y luminance
  • f
in tersections
  • Consisten
t with Stoner et al
  • AR
V O
slide-14
SLIDE 14 Conclusions
  • Simple
t w
  • stage
distributed computation F
  • r
eac h stage
  • Linear
  • p
erators squared and normalized
  • Resp
  • nse
space is minimally sampled
  • Resp
  • nses
smo
  • thly
co v er the space
  • In
termediate resp
  • nses
ma y b e exactly in terp
  • lated
  • Mo
del is consisten t with ph ysiology
  • f
Complex
  • MT
cells
  • Mo
del is capable
  • f
represen ting m ultiple motions
  • Mo
del is consisten t with plaid transparency p erception AR V O