Generalized Linear Models (GLMs) II Jonathan Pillow 1 <latexit - - PowerPoint PPT Presentation

Generalized Linear Models (GLMs) II

Statistical modeling and analysis of neural data NEU 560, Spring 2018 Lecture 10 Jonathan Pillow

1

Summary:

ˆ k = (XT X)−1XT Y

• 1. “Linear-Gaussian” GLM:

Y |X,~ k ∼ N(X~ k, 2I)

• 2. Bernoulli GLM:
• 3. Poisson GLM:

yt|~ xt,~ k ∼ Ber(f(~ xt · ~ k))

2

stimulus filter Poisson spiking

stimulus

k f

λ(t)

conditional intensity (spike rate) exponential nonlinearity

Linear-Nonlinear-Poisson

3

Fitting the nonlinearity

Filter k specifies a direction in stimulus space 
 (i.e. a 1D subspace)

4

1) Project onto subspace spanned by k

Fitting the nonlinearity

5

projection onto uk

1) Project onto subspace spanned by k

Fitting the nonlinearity

2) take histogram of projected stimuli

6

STA response

1) Project onto subspace spanned by k

Fitting the nonlinearity

2) take histogram of projected stimuli 3) ML estimate of Poisson rate in each bin is # spikes / # stimuli

projection onto uk

7

= stimuli and spike trains such that falls in bin j

= Poisson firing rate in bin j

Fitting the nonlinearity: derivation

Log-likelihood:

ML estimate: = # spikes / # stim

• piecewise constant model of nonlinearity f
• more histogram bins ⇒ more flexible model

Model:

• r

8

LNP (Linear-Nonlinear-Poisson)
 cascade model

Characterization Procedure:

1. Fit filter k using maximum likelihood under 
 assumed nonlinearity.

• 2. Project stimuli onto k, compute spike rate 


(mean # spikes / stimulus) in each histogram bin.

stimulus filter Poisson spiking

stimulus

k f

λ(t)

exponential nonlinearity

projection onto uk

9

• output: Poisson process
• problem: assumes spiking depends only on stimulus!

stimulus filter Poisson spiking

stimulus

k f

λ(t)

conditional intensity (spike rate) exponential nonlinearity

LNP (Linear-Nonlinear-Poisson)
 cascade model

10

conditional intensity (spike rate)

(Truccolo et al 04)

• output: no longer a Poisson process

Poisson GLM with spike-history dependence

post-spike filter exponential nonlinearity probabilistic spiking

stimulus

stimulus filter

+ k h

f

11

filter output

traditional IF

filter output

∞

“hard threshold” “soft-threshold” IF

spike rate

• interpretation: “soft-threshold” integrate-and-fire model

Poisson GLM with spike-history dependence

post-spike filter exponential nonlinearity probabilistic spiking

stimulus

stimulus filter

+ k h

f

12

GLM dynamic behaviors

post-spike filter h(t) stimulus p(spike)

• irregular spiking

filter outputs

(“currents”)

13

GLM dynamic behaviors

post-spike filter h(t) stimulus p(spike)

• regular spiking

filter outputs

(“currents”) (Weber & Pillow 2016)

14

GLM dynamic behaviors

post-spike filter h(t)

• bursting

filter outputs

(“currents”)

p(spike) stimulus

15

GLM dynamic behaviors

post-spike filter h(t) stimulus filter outputs

(“currents”)

p(spike)

• adaptation

16

GLM dynamic behaviors (from Izhikevich)

A B C D E F G H I J K L M N O P

tonic spiking phasic spiking tonic bursting phasic bursting mixed mode type I type II spike latency resonator integrator rebound spike rebound burst variability bistability I bistability II

50 ms

spike frequency adaptation threshold

(Weber & Pillow 2017) Izhikevich neuron GLM spikes stimulus GLM parameters

17

multi-neuron GLM

exponential nonlinearity probabilistic spiking

stimulus

neuron 1 neuron 2 post-spike filter stimulus filter

+ +

18

multi-neuron GLM

exponential nonlinearity probabilistic spiking coupling filters

stimulus

neuron 1 neuron 2 post-spike filter stimulus filter

+ +

19

...

time

t

GLM equivalent diagram:

spike rate

20

Uzzell et al (J Neurophys 04)

• stimulus = binary flicker

• parasol retinal ganglion cell spike responses

Example dataset

21

Uzzell et al (J Neurophys 04)

• stimulus = binary flicker

• parasol retinal ganglion cell spike responses

Example dataset

22

Y X~ k

time time lag

model

Stimulus-only GLM

spike response design matrix

23

time

model model

stimulus portion spike-history portion

Stimulus + SpikeHistory GLM

Y X~ k

spike response design matrix

24

time

model model stimulus portion neuron 1 spike-hist neuron 2 spike-hist neuron 3 spike-hist neuron 4 spike-hist

Y X~ k

spike response design matrix

Stimulus + History + 3 Neuron Coupling GLM

25

Fitting: Maximum Likelihood

• find filters that maximize the

log-conditional probability of
 the observed data GLM parameters Data

• log-likelihood is concave
• no local maxima [Paninski 04]

log P(Y |X) = X

t

yt log λt − λt

• smooth basis for

coupling filters

26

• Concavity: having everywhere downward curvature
• Convexity: having everywhere upward curvature
• “f = concave” iff “-f = convex”
• Maximizing concave function = minimizing a convex function
• both preclude (non-global) local optima

convexity and concavity

concave convex

27

• A function f is convex if, for any x1, x2, a in [0,1]:
• for continuous scalar functions:
• for vector functions: all eigenvalues of Hessian are ≥0, or

convexity: formal definition properties of convex functions:

• affine maps:
• sums:
• linear functions: concave and convex

28

log-concavity of GLM likelihood

Theorem (Paninski 2004) The GLM log-likelihood is concave in the parameters {k,h}, for any stimulus X and any spike data Y, if the nonlinearity f satisfies:

• f is convex
• f is log-concave (i.e., log f(x) is a concave function)

Proof

weighted sum of concave funcs: concave sum of convex funcs: convex concave function negative of convex func: concave sum of two concave functions is concave ⇒ log-likelihood is concave! log-likelihood

29

log-concavity of GLM likelihood

Theorem (Paninski 2004) The GLM log-likelihood is concave in the parameters {k,h}, for any stimulus X and any spike data Y, if the nonlinearity f satisfies:

• f is convex
• f is log-concave (i.e., log f(x) is a concave function)

Examples of acceptable nonlinearities:

• condition: f must grow at least linearly and at most exponentially

Why this matters: no restriction on choice of stimuli! Can easily find ML fits to GLM parameters for any stimuli + spikes

30