SLIDE 1
Estimation
parameter measured dataset
(“sample”)
An estimator is a function
model
- often we will write or just
Maximum Likelihood Jonathan Pillow Mathematical Tools for - - PowerPoint PPT Presentation
Maximum Likelihood Jonathan Pillow Mathematical Tools for - - PowerPoint PPT Presentation
Maximum Likelihood Jonathan Pillow Mathematical Tools for Neuroscience (NEU 314) Spring, 2016 lecture 16 Estimation model measured dataset parameter (sample) An e stimator is a function often we will write or just
SLIDE 2
SLIDE 3
Properties of an estimator
bias:
- “unbiased” if bias=0
variance:
- “consistent” if bias and variance both go
to zero asymptotically “expected” value (average over draws of m)
mean squared error (MSE)
SLIDE 4
Example 1: linear Poisson neuron
spike count spike rate encoding model: stimulus parameter
SLIDE 5
20 40 20 40 60
(contrast) (spike count)
20 40 60
conditional distribution
p(y|x)
SLIDE 6
20 40 20 40 60
(contrast) (spike count)
20 40 60
conditional distribution
p(y|x)
SLIDE 7
20 40 20 40 60
(contrast) (spike count)
20 40 60
conditional distribution
p(y|x)
SLIDE 8
20 40 20 40 60
(contrast) (spike count)
Maximum Likelihood Estimation:
- given observed data , find that maximizes
p(y|x)
SLIDE 9
20 40 20 40 60
(contrast) (spike count)
Maximum Likelihood Estimation:
- given observed data , find that maximizes
p(y|x)
SLIDE 10
20 40 20 40 60
(contrast) (spike count)
Maximum Likelihood Estimation:
- given observed data , find that maximizes
p(y|x)
SLIDE 11
likelihood
Likelihood function: as a function of
Because data are independent:
1 2
SLIDE 12
1 2
log-likelihood log
Likelihood function: as a function of
Because data are independent:
1 2
likelihood
SLIDE 13
- Closed-form solution (exists for “exponential family” models)
1 2
log-likelihood
SLIDE 14
Properties of the MLE (maximum likelihood estimator)
- consistent
(converges to true in limit of infinite data)
- efficient
(converges as quickly as possible, i.e., achieves minimum possible asymptotic error)
SLIDE 15
Example 2: linear Gaussian neuron
spike count spike rate encoding model: stimulus parameter
SLIDE 16
20 40 20 40 60
(contrast) (spike count)
20 40 60
All slices have same width encoding distribution
SLIDE 17
Log-Likelihood Differentiate and set to zero: Maximum-Likelihood Estimator:
(“Least squares regression” solution) (Recall that for Poisson, )
SLIDE 18
Example 3: unknown neuron
- 25
25 25 50 75 100 (contrast) (spike count)
What model would you use to fit this neuron?
SLIDE 19
Example 3: unknown neuron
More general setup:
- 25
25 25 50 75 100 (contrast) (spike count)