V1 MT Hubel and Wiesel, 1968 Maunsell and V an Essen, 1983 - - PowerPoint PPT Presentation

Maunsell and V an Essen, 1983

MT

Hubel and Wiesel, 1968

V1

0% coherence 50% coherence 100% coherence

Relating MT responses to visual discrimination

Newsome and Paré, 1988

Downing & Movshon, 1989

V1 V 2 M T FST AITv CITv 7a STP LIP, VIP MST DP V4

PITd

PITv CITd AITd VOT VP FEF

Visual input Record

V1 V 2 M T FST AITv CITv 7a STP LIP, VIP MST DP V4

PITd

PITv CITd AITd VOT VP FEF

Visual input Record

Visual stimulus Neuronal response Behavioral judgement

Fixation Point Pref target Null target

10 deg

Receptive field Dots Aperture Fix Pt Dots Targets 1 sec

+ recording in MT

Britten, Shadlen, Newsome & Movshon, 1993

MT responses depend on motion coherence

25 50 75 100 50 100 150 200 10 20 30 40 50 50 100 25 50 75 100 10 20 30 40 50 10 20 30 40 25 50 75 100

Firing rate (impulses/trial) Coherence (%) Preferred direction Null direction

Britten, Shadlen, Newsome & Movshon, 1992

Coherence (%) Proportion correct

Behavioral performance from one session

Britten, Shadlen, Newsome & Movshon, 1992 Barlow, Levick and Y

• on, 1971

Britten, Shadlen, Newsome & Movshon, 1992

Coherence (%) Proportion correct

20 40 60 Number of cells 0.1 1 10 Threshold ratio (neuron/behavior)

MT cells are as sensitive as monkeys to visual motion

Neuronal threshold, choice (%) Neuronal threshold, fixation (%)

1 10 100 1 10 100 Britten, Shadlen, Newsome & Movshon, 1992

MT cell firing does not require the observer to make a decision

Visual stimulus Neuronal response Behavioral judgement

Neurometric function Psychometric function

Visual stimulus Neuronal response Behavioral judgement

Neurometric function Psychometric function

?

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

MT cell firing is correlated with behavioral choice

Choose “preferred” Choose “anti” 0% coherence

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

MT cell firing is correlated with behavioral choice

Choose “preferred” Choose “anti” 0% coherence

MT cell firing is correlated with behavioral choice

Response of "preferred" neuron Response of "antipreferred" neuron

MT cell firing is correlated with behavioral choice

Response of "preferred" neuron Response of "antipreferred" neuron Choose "preferred" Choose "antipreferred"

MT cell firing is correlated with behavioral choice

Response of "preferred" neuron Response of "antipreferred" neuron Choose "preferred" Choose "antipreferred" Response of "preferred" neuron Choice probability ~ 0.85

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

Correlation of activity to choice is not accidental

500 1000 1500 2000 Time (msec) 0.0 0.2 0.4 Mean normalized response 500 1000 1500 2000 Time (msec)

• 0.1

0.0 0.1 Difference in normalized response Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

Choice-related activity has a “forward” time course

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

The most sensitive neurons are most correlated to choice

–50 –25 25 50 0.5 1

–0.5 0.5 –0.5 0.5 –0.5 0.5 –0.5 0.5 –0.5 0.5 0.5

Probability Percentage of near choices Added signal: –50% –25% Disparity (º) Added signal (%) 0% 25% 50% 10 20 30 40 2,000 –0.3 0.3

a d

2 Time (s) Time (ms)

c b

Fixation marker Stimulus Choice targets Trial start Trial end Disparity (º)

Figure 1 | Methods. a, Sketch of the sequence of events during one trial. b, Example time series of the stimulus. c, Probability mass distributions of the stimuli for one experiment (probability as a function of disparity), with signal disparities of 20.3u and 0.15u. Each plot depicts one signal condition (negative percentages indicate near signal disparities). d, The monkey’s performance for this experiment (in percentage ‘near’ choices as a function

• f percentage added signal).

d

Disparity (º)

a

n = 17,200 1,000 2,000 0.3 –0.3 –0.01 –0.005 0.005 0.01 1,000 2,000 0.54 Time (ms) Mean choice probability n = 57 cells

c

1 2 Amplitude

b

0.5 0.4 0.6 0.8 –0.5 0.5 Choice probability R n = 76 Temporal integration time (ms) 20 40 60

Figure 2 | Psychophysical kernel and choice-related signal have different time courses. a, Psychophysical kernel (averaged over 76 experiments; n 5 17,200 trials; two monkeys) as a function of disparity and time. Colour represents amplitude (in occurrences per frame). b, Normalized amplitude

• f the psychophysical kernels decreases over time. c, Averaged choice-related

signal over time. Shaded grey areas in b and c, 61 standard error. d, The correlation coefficient, R, over time between choice probability (for individual neurons) and the amplitude of the mean psychophysical kernel, plotted against a neuron’s mean choice probability. Colour represents temporal integration time (Supplementary Methods); bold symbols, significant R (P , 0.05, by resampling); circles, data from monkey 1; squares, data from monkey 2.

Choice probability for depth discrimination in V2

Visual stimulus Neuronal response Behavioral judgement

Neurometric function Psychometric function Choice probability

Albright, 1984

V1 V 2 M T FST AITv CITv 7a STP LIP, VIP MST DP V4

PITd

PITv CITd AITd VOT VP FEF

Visual input Stimulate

Fixation Point Pref target Null target

10 deg

Receptive field Dots Aperture Fix Pt Dots Targets 1 sec Elect Stim

+ microstimulation in MT

Salzman, Murasugi, Britten and Newsome, 1992

Coherence (%) Proportion of choices of the preferred direction

Salzman, Murasugi, Britten and Newsome, 1992

Equivalent visual coherence Number of cases

Visual stimulus Neuronal response Behavioral judgement

Neurometric function Psychometric function Choice probability Microstimulation

X

up 1

X

up 2

X

up 3

• X

up N

Pooled MT Signal

〈 X 〉

up

X

down 1

X

down 2

X

down 3

• X

down N

〈 X 〉

Pooled MT Signal

down

Decision

up

〈 X 〉 〈 X 〉

down

> Britten, Newsome, Shadlen, Celebrini & Movshon, 1996 Shadlen, Britten, Newsome & Movshon, 1996 Cohen & Newsome, 2009

Decoding MT neurons for visual motion discrimination

X

up 1

X

up 2

X

up 3

• X

up N

Pooled MT Signal

〈 X 〉

up

X

down 1

X

down 2

X

down 3

• X

down N

〈 X 〉

Pooled MT Signal

down

Decision

up

〈 X 〉 〈 X 〉

down

> Shadlen, Britten, Newsome & Movshon, 1996 Cohen & Newsome, 2009

Decoding MT neurons for visual motion discrimination

X

up 1

X

up 2

X

up 3

• X

up N

Pooled MT Signal

〈 X 〉

up

X

down 1

X

down 2

X

down 3

• X

down N

〈 X 〉

Pooled MT Signal

down

Decision

up

〈 X 〉 〈 X 〉

down

> Shadlen, Britten, Newsome & Movshon, 1996 Cohen & Newsome, 2009

Decoding MT neurons for visual motion discrimination

90 180 Difference in preferred direction (deg) 0.0 0.5 Interneuronal correlation Zohary, Shadlen & Newsome, 1994

Lateral intra-parietal area (LIP) V1 V4 7a 7b V2d V2v MT/MST Frontal eye field (FEF)

Where is sensory activity converted into decision and actions?

LIP receives projections from MT and projects to areas that are known to contribute to the generation of saccadic eye movements

• ✙
• ✙
• Fixate

350 msec Targets appear 500 msec Random dot motion 2 sec Delay 500-1000 msec Saccade

✙

• Target 2

Target 1

✙

task_panels_vert_noMF.isl

• Target 2

Target 1

✙

FP

A B

Shadlen and Newsome, 2001

+ recording in LIP

• 0.5

0.5 1 10 15 20 25 30 35 40 45 50

• 0.5

0.5 10 15 20 25 30 35 40 45 50

Time (s) motion onset saccade Mean response (sp/s)

51.2% 25.6% 12.8% 6.4% 0% N=106

Mike Shadlen

Responses in a reaction-time version of the direction discrimination task

Mike Shadlen

High motion strength High motion strength Low motion strength Time ~1 sec Stimulus on Stimulus off Spikes/s Time ~1 sec Stimulus on Stimulus off Spikes/s Low motion strength

LIP – decision formation Accumulation of evidence “ramp” Threshold

Mike Shadlen

MT – sensory evidence Motion energy “step”

Bounded accumulation of evidence

Momentary evidence e.g., ∆Spike rate: MTRight– MTLeft Accumulated evidence for Rightward and against Leftward Criterion to answer “Right” Criterion to answer “Left”

Diffusion to bound model

Palmer et al (2005) Shadlen et al (2006)

µ = kC

C is motion strength (coherence)

P = 1 1+ e

−2k C B

t(C) = B kC tanh(BkC) + tnd

Responses in a reaction-time version of the direction discrimination task are well described by the “race” model of integration to a decision boundary

Firing rate (sp/s)

saccade

choose Tin choose Tout

RT (ms)

Time from saccade (ms) Firing rate (sp/s)

30 40 50 60 70 –1000 –500

Grouped by RT

Roitman & Shadlen, 2002

Saccade Motion Targets Fixation Time

]

Reaction Time

Speed-Accuracy Tradeoff

Hanks, Kiani & Shadlen, 2014

a b c

20 40 60 0.5 0.6 0.7 0.8 0.9 1 Motion strength (% coh) Proportion correct 20 40 60 0.4 0.5 0.6 0.7 Motion strength (% coh) Reaction time (s)

Speed-Accuracy Tradeoff

Hanks, Kiani & Shadlen, 2014

c

Bound for Tin

A

Bound for Tout

A

Tin accumulator Tout accumulator

Speed-Accuracy Tradeoff

Hanks, Kiani & Shadlen, 2014

Speed-Accuracy Tradeoff

−400 −200 0.5 1 Time from saccade (ms) Normalized firing rate Accuracy condition −400 −200 0.5 1 Time from saccade (ms) Speed condition 20 40 60 0.8 1 1.2 Motion strength (% coh) Normalized FR before saccade

Faster Slower Faster Slower

a b c

Speed-Accuracy Tradeoff

Hanks, Kiani & Shadlen, 2014

Lateral intra-parietal area (LIP) V1 V4 7a 7b V2d V2v MT/MST Frontal eye field (FEF)

Where is sensory activity converted into decision and actions?

Voluntary saccade Motion Fixation Evoked saccade Time Fixation point Targets Motion Electrical stim Eye position 200 ms

Gold and Shadlen, 2000

+ microstimulation in FEF

• 5

5 10 X eye position (degrees)

• 5

5 Y eye position (degrees) ←Stimulation alone Choice alone →

Gold and Shadlen, 2000

• 5

5 10 X eye position (degrees)

• 5

5 Y eye position (degrees) ←Stimulation alone Choice alone → ← Choice and stimulation

Gold and Shadlen, 2000

• 4

4

• 4

4

• 4

4

• 4

4

• 5

5 10 X eye position (degrees)

• 5

5 Y eye position (degrees) ←Stimulation alone Choice alone → ← Choice and stimulation Weak visual signal Strong visual signal

Gold and Shadlen, 2000

100 300 500 0.5 1.0 1.5 Viewing duration (ms) Magnitude of deviation (degrees) Strong visual signal Weak visual signal

Gold and Shadlen, 2000

100 300 500 0.5 1.0 1.5 Viewing duration (ms) Magnitude of deviation (degrees) Strong visual signal Weak visual signal

Gold and Shadlen, 2000

Saccade vector map (FEF) Rightward Leftward Direction of motion map (MT)

DECISION

V1 V 2 M T FST AITv CITv 7a STP LIP, VIP MST DP V4

PITd

PITv CITd AITd VOT VP FEF

Visual input Eye movement

Mmm .... left ... right?

Motion

How confident am I?

Response

Tin Topp

Stimulus duration: 100-900 ms (truncated exponential) Sure target delay: 500-750 ms Sure reward / correct reward ≈ 0.8

• o o

Certainty task

Kiani & Shadlen, 2009

Post-decision wagering

sure bet high stakes choice

Mmm .... left ... right?

Motion

How confident am I?

Response

Kiani & Shadlen, 2009

Kiani & Shadlen, 2009

Momentary evidence e.g., ∆Spike rate: MTRight– MTLeft

Decision variable (v)

Link 1992 Ratcliff & Smith, 2004 Palmer et al, 2005 Laming, 1968 Luce, 1986

Choose right Choose le7 Time

– + Momentary evidence μ= kC

accumulaBon-to-bound model

M

• s

t l i k e l y c

• r

r e c t U n c e r t a i n

Kiani & Shadlen, 2009

Decline sure target Decline sure target Choose sure target

+B

• B

μ= kC

Three free parameters: k, sensiBvity coefficient B, bound height θ, criterion on log-odds correct

Kiani & Shadlen, 2009

Model fits

Kiani & Shadlen, 2009

Model fits PredicBons

Kiani & Shadlen, 2009

saccade 0.5 1 1.5 dots 1 sure target saccade 0.5 1 1.5 dots 1 sure target

choose Tin choose Tout choose Tsure

100 ms

Firing rate (normalized) motion sure target eye movement weak motion strength N = 70 neurons two directions remember 2 dashed lines. Waive good Decline good Reject bad

Kiani & Shadlen, 2009