Maximum Likelihood Jonathan Pillow Mathematical Tools for - PowerPoint PPT Presentation

Maximum Likelihood Jonathan Pillow Mathematical Tools for Neuroscience (NEU 314) Spring, 2016 lecture 16

Estimation model measured dataset 
 parameter (“sample”) An e stimator is a function • often we will write or just

Properties of an estimator “expected” value 
 (average over draws of m) bias: • “unbiased” if bias=0 variance: • “consistent” if bias and variance both go 
 to zero asymptotically mean squared error (MSE)

Example 1: linear Poisson neuron spike count spike rate parameter stimulus encoding model:

60 p(y|x) conditional distribution (spike count) 40 20 0 0 20 40 0 20 40 60 (contrast)

60 p(y|x) conditional distribution (spike count) 40 20 0 0 20 40 0 20 40 60 (contrast)

60 p(y|x) conditional distribution (spike count) 40 20 0 0 20 40 0 20 40 60 (contrast)

Maximum Likelihood Estimation: • given observed data , find that maximizes 60 p(y|x) (spike count) 40 20 0 0 20 40 (contrast)

Maximum Likelihood Estimation: • given observed data , find that maximizes 60 p(y|x) (spike count) 40 20 0 0 20 40 (contrast)

Maximum Likelihood Estimation: • given observed data , find that maximizes 60 p(y|x) (spike count) 40 20 0 0 20 40 (contrast)

Likelihood function: as a function of Because data are independent: likelihood 0 1 2

Likelihood function: as a function of Because data are independent: likelihood 0 1 2 log log-likelihood 0 1 2

log-likelihood 0 1 2 • Closed-form solution (exists for “exponential family” models)

Properties of the MLE (maximum likelihood estimator) • consistent (converges to true in limit of infinite data) • e ffi cient 
 (converges as quickly as possible, 
 i.e., achieves minimum possible asymptotic error)

Example 2: linear Gaussian neuron spike count spike rate parameter stimulus encoding model:

60 encoding distribution 40 (spike count) 20 0 0 20 40 0 20 40 60 (contrast) All slices have same width

Log-Likelihood Differentiate and set to zero: Maximum-Likelihood Estimator: (“Least squares regression” solution) (Recall that for Poisson, )

Example 3: unknown neuron 100 75 (spike count) 50 25 0 -25 0 25 (contrast) What model would you use to fit this neuron?

Example 3: unknown neuron 100 75 (spike count) 50 25 0 -25 0 25 (contrast) More general setup: for some nonlinear function f