Generalized Linear Models (GLMs) Jonathan Pillow 1 Example 3: - - PowerPoint PPT Presentation

Generalized Linear Models (GLMs)

Statistical modeling and analysis of neural data NEU 560, Spring 2018 Lecture 9 Jonathan Pillow

1

Example 3: unknown neuron

• 25

25 25 50 75 100 (contrast) (spike count)

Be the computational neuroscientist: what model would you use?

2

Example 3: unknown neuron

More general setup:

• 25

25 25 50 75 100 (contrast) (spike count)

for some nonlinear function f

3

Answer: stimulus likelihood function - useful for ML stimulus decoding!

Quick Quiz:

• 1. as a function of y?
• 2. as a function of ?

The distribution P(y|x, ) can be considered as a function of y, x, or .

spikes stimulus parameters

Answer: encoding distribution - probability distribution over spike counts Answer: likelihood function - the probability of the data given model params

• 3. as a function of x?

What is P(y|x, ) :

4

20 40 20 40 60

(contrast) (spike count)

stimulus decoding likelihood

5

20 40 20 40 60

(contrast)

20 40 60

Stimulus likelihood function (for decoding)

(spike count)

What is this?

6

GLMs

• Be careful about terminology:

Linear Linear General Linear Model Generalized Linear Model

GLM GLM ≠

(Nelder 1972)

7

2003 interview with John Nelder...

Stephen Senn: I must confess to having some confusion when I was a young statistician between general linear models and generalized linear models. Do you regret the terminology? John Nelder: I think probably I do. I suspect we should have found some more fancy name for it that would have stuck and not been confused with the general linear model, although general and generalized are not quite the same. I can see why it might have been better to have thought of something else.

Senn, (2003). Statistical Science

8

Moral: Be careful when naming your model!

9

Linear Noise

“Dimensionality Reduction”

(exponential family)

Examples:

• 1. Gaussian
• 2. Poisson

y = ~ ✓ · ~ x + ✏

• 1. General Linear Model

10

• 2. Generalized Linear Model

Linear Examples:

• 1. Gaussian
• 2. Poisson

Noise

(exponential family)

Nonlinear

y = f(~ ✓ · ~ x) + ✏

11

• 2. Generalized Linear Model

Linear Noise

(exponential family)

Nonlinear Terminology: “distribution function” “parameter” = “link function”

12

1 1 0 0 0 0 1 1 01 0 0 00 0 0 0

stimulus response

From spike counts to spike trains:

linear filter vector stimulus at time t

time

response at time t

first idea: linear-Gaussian model!

yt = ~ k · ~ xt + ✏t

yt = ~ k · ~ xt + noise

N(0, σ2)

13

~ xt yt

1 1 0 0 0 0 1 1 01 0 0 00 0 0 0

stimulus response

linear filter vector stimulus at time t

yt = ~ k · ~ xt + noise

time

response at time t

t = 1

walk through the data

• ne time bin at a time

N(0, σ2)

14

~ xt yt

1 1 0 0 0 0 1 1 01 0 0 00 0 0 0

linear filter vector stimulus at time t

yt = ~ k · ~ xt + noise

time

response at time t

t = 2

walk through the data

• ne time bin at a time

stimulus response

N(0, σ2)

15

~ xt yt

1 1 0 0 0 0 1 1 01 0 0 00 0 0 0

linear filter vector stimulus at time t

yt = ~ k · ~ xt + noise

time

response at time t

t = 3

walk through the data

• ne time bin at a time

stimulus response

N(0, σ2)

16

~ xt

yt

1 1 0 0 0 0 1 1 01 0 0 00 0 0 0

linear filter vector stimulus at time t

yt = ~ k · ~ xt + noise

time

response at time t

t = 4

walk through the data

• ne time bin at a time

stimulus response

N(0, σ2)

17

~ xt yt

1 1 0 0 0 0 1 1 01 0 0 00 0 0 0

linear filter vector stimulus at time t

yt = ~ k · ~ xt + noise

time

response at time t

t = 5

walk through the data

• ne time bin at a time

stimulus response

N(0, σ2)

18

~ xt yt

1 1 0 0 0 0 1 1 01 0 0 00 0 0 0

linear filter vector stimulus at time t

yt = ~ k · ~ xt + noise

time

response at time t

t = 6

walk through the data

• ne time bin at a time

stimulus response

N(0, σ2)

19

Build up to following matrix version:

…

Y

X~ k

= + noise =

time design matrix

…

~ k

1

20

Build up to following matrix version:

1 …

Y

X~ k

= + noise =

time

ˆ k = (XT X)−1XT Y

stimulus covariance spike-triggered avg (STA)

(maximum likelihood estimate for “Linear-Gaussian” GLM) least squares solution:

…

~ k

21

Formal treatment: scalar version

model:

N(0, σ2)

yt = ~ k · ~ xt + ✏t

equivalent to writing:

yt|~ xt,~ k ∼ N(~ xt · ~ k, 2)

p(yt|~ xt,~ k) =

1 √ 2πσ2 e− (yt−~

xt·~ k)2 22

• r

p(Y |X,~ k) =

T

Y

t=1

p(yt|~ xt,~ k)

For entire dataset:

(independence across time bins)

= (2πσ2)− T

2 exp(− PT

t=1 (yt−~ xt·~ k)2 22

)

Guassian noise with variance σ2

log P(Y |X,~ k) = − PT

t=1 (yt−~ xt·~ k)2 22

+ const

log-likelihood

22

Formal treatment: vector version

…

Y

X~ k

=

…

~ k

=

time

+ ~ ✏

N(0, σ2I)

…

+

iid Gaussian noise vector

✏1 ✏2

✏3

equivalent to writing:

• r

Y |X,~ k ∼ N(X~ k, 2I) P(Y |X,~ k) =

1 |2πσ2I|

T 2 exp

⇣ −

1 2σ2 (Y − X~

k)>(Y − X~ k) ⌘

Take log, differentiate and set to zero.

1

23

…

…

~ k

≈

time

probability of spike at bin t

Bernoulli GLM:

pt = f(~ xt · ~ k)

(coin flipping model, y = 0 or 1)

p(yt = 1|~ xt) = pt

nonlinearity

Equivalent ways of writing:

yt|~ xt,~ k ∼ Ber(f(~ xt · ~ k)) p(yt|~ xt,~ k) = f(~ xt · ~ k)yt ⇣ 1 − f(~ xt · ~ k) ⌘1−yt

• r

But noise is not Gaussian!

log-likelihood:

L = PT

t=1

⇣ yt log f(~ xt · ~ k) + (1 − yt) log(1 − f(~ xt · ~ k)) ⌘

f( )

1

24

Logistic regression

Logistic regression:

f(x) = 1 1 + e−x

logistic function

• so logistic regression is a special case of a Bernoulli GLM

…

…

~ k

≈

time

probability of spike at bin t

Bernoulli GLM:

pt = f(~ xt · ~ k)

(coin flipping model, y = 0 or 1)

p(yt = 1|~ xt) = pt

nonlinearity

f( )

1

25

Poisson regression

firing rate

Poisson GLM:

(integer y≥0 ) nonlinearity

t = f(~ xt · ~ k) yt|~ xt,~ k ∼ Poiss(∆t)

time bin size

log-likelihood: encoding distribution:

26

Summary:

ˆ k = (XT X)−1XT Y

• 1. “Linear-Gaussian” GLM:

Y |X,~ k ∼ N(X~ k, 2I)

• 2. Bernoulli GLM:
• 3. Poisson GLM:

yt|~ xt,~ k ∼ Poiss(∆t) yt|~ xt,~ k ∼ Ber(f(~ xt · ~ k))

27