Topics in Brain Computer Interfaces Topics in Brain Computer - - PowerPoint PPT Presentation

Michael J. Black - CS295-7 2005 Brown University

Topics in Brain Computer Interfaces Topics in Brain Computer Interfaces CS295 CS295-

• 7

7

Professor: MICHAEL BLACK TA: FRANK WOOD Spring 2005 Probabilistic Inference

Michael J. Black - CS295-7 2005 Brown University

Reza Shadmehr

Michael J. Black - CS295-7 2005 Brown University

Reza Shadmehr

After 300 trails “Catch” trails Forces used to counter field.

Michael J. Black - CS295-7 2005 Brown University

Kinarm

Michael J. Black - CS295-7 2005 Brown University

Kinarm project

• Connect Kinarm with TG2 game

software (partially done).

• Calibrate Kinarm (partially done)
• Develop a high-level way of

programming different force fields (not done).

• Develop a basic set of fields.

Michael J. Black - CS295-7 2005 Brown University

Encoding Review

k k

y y x x k y k x k

v h v h h h h speed h z + + = + + = )) sin( ) cos( ( θ θ Moran & Schwartz (’99):

(Linear in velocity)

x

v

x

v

y

v

y

v

Michael J. Black - CS295-7 2005 Brown University

Encoding Review

k y k x k

y b x b b z + + =

Kettner et al (’88):

(Linear in position)

Flament et al (‘88): Firing rate is also related to hand

acceleration y y x x

Michael J. Black - CS295-7 2005 Brown University

Notation

⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ =

k n k k k

z z z z

, , 2 , 1

M v

Firing rates of n cells at time k

⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ =

k y k x k y k x k k k

a a v v y x x

, , , ,

v

Hand kinematics at time k

) , , , (

1 1

z z z Z

k k k

v K v v v

−

= ) , , , (

1 1

x x x X

k k k

v K v v v

−

=

Michael J. Black - CS295-7 2005 Brown University

Decoding Methods

Direct decoding methods:

,...) , (

1 −

=

k k k

z z f x v v v

Simple linear regression method

d k k T k d k k T k

Z f y Z f x

− −

= =

: 2 : 1 v

v v v

Michael J. Black - CS295-7 2005 Brown University

Decoding Methods

Direct decoding methods: In contrast to generative encoding models: Need a sound way to exploit generative models for decoding.

,...) , (

1 −

=

k k k

z z f x v v v ) (

k k

x f z v v =

Michael J. Black - CS295-7 2005 Brown University

Uncertainty

Typically there will be uncertainty in our models

noise ) ( + =

k k

x f z v v

But we want to estimate something about the state x given noisy measurements z

) ,..., | ( ) | (

1

z z x p Z x p

j t t j t t

v v v v v

− −

=

This defines the likelihood of the observations given the state:

) | (

k k x

z p v v

Michael J. Black - CS295-7 2005 Brown University

Probability Review

Probability

1

a

2

a

3

a

5

a

4

a

7

a

6

a

Let X be a random variable that can take on one of the discrete values

] , , [

7 1

a a X K ∈

Michael J. Black - CS295-7 2005 Brown University

Basic facts

Probability

) ( just

• r

) (

i i

a p a X p =

1 ) ( ≤ = ≤

i

a X p 1 ) (

7 1

=

∑

= i i

a p

1

a

2

a

3

a

5

a

4

a

7

a

6

a

Michael J. Black - CS295-7 2005 Brown University

Basic facts

Probability

∑

= =

x

x x p x E ) ( ] [ µ

Expected value or expectation of a random variable

∑

− = − = =

x

x p x x E x E x ) ( ) ( ] )) ( [( ] var[

2 2 2

µ σ

?

1

a

2

a

3

a

5

a

4

a

7

a

6

a

Michael J. Black - CS295-7 2005 Brown University

Joint Probability

) , ( ) , (

, 2 , 1 , 2 2 , 1 1 j i j i

a a p a X a X p = = =

1 ) , (

, 1 , 2

, 2 , 1

=

∑∑

i j

a a j i a

a p

Statistical independence

) ( ) ( ) , ( y p x p y x p =

• knowing y tells you nothing about x

Michael J. Black - CS295-7 2005 Brown University

Conditional Probability

Dependence - Knowing the value of one random variable tells us something about the other.

) , ( ) ( ) | ( ) ( ) , ( ) | ( B A p B p B A p B p B A p B A p = =

Michael J. Black - CS295-7 2005 Brown University

Statistical Independence

) ( ) ( ) ( ) ( ) ( ) , ( ) | ( A p B p B p A p B p B A p B A p = = = ?

If A and B are statistically independent?

Michael J. Black - CS295-7 2005 Brown University

Statistical Independence

) ( ) | ( ) ( ) | ( B p A B p A p B A p = =

) ( ) ( ) ( ) | ( ) , ( B p A p B p B A p B A p = =

A and B are statistically independent if and only if

) ( ) ( ) ( ) ( ) ( ) , ( ) | ( A p B p B p A p B p B A p B A p = = =

Michael J. Black - CS295-7 2005 Brown University

Conditional Independence

) ( ) | ( ) | ( ) ( ) | , ( ) , , ( C p C B p C A p C p C B A p C B A p = =

A is independent of B, conditioned on C If I know C, then knowing B doesn’t give me any more information about A. This does not mean that A and B are statistically independent

Michael J. Black - CS295-7 2005 Brown University

More generally

) | ( ) , | ( ) | , ( C B p C B A p C B A p = ) | ( ) , | ( ) | , ( ) | ( C B p C B A p C B A p C A p

B B

∑ ∑

= =

Marginalizing over a random variable

Michael J. Black - CS295-7 2005 Brown University

Bayes’ Theorem

) ( ) ( ) | ( ) | ( ) ( ) | ( ) ( ) | ( ) , ( B p A p A B p B A p A p A B p B p B A p B A p = = =

• Revd. Thomas Bayes, 1701-1761

Michael J. Black - CS295-7 2005 Brown University

normalization constant (independent of mouth) Prior (a priori – before the evidence) Likelihood (evidence)

Bayesian Inference

Posterior a posteriori probability (after the evidence)

) firing ( ) kinematics ( ) kinematics | firing ( ) firing | kinematics ( p p p p =

We infer hand kinematics from uncertain evidence and our prior knowledge of how hands move.

Michael J. Black - CS295-7 2005 Brown University

GENERATIVE MODEL

k k k

q x f z v v v + = ) (

1 k k k

w x f x v v v + =

− )

(

1 2

neural firing rate of N=42 cells in M=70ms behavior (e.g. hand position, velocity, acceleration) noise (e.g. Normal or Poisson)

Encoding:

linear, non-linear?

Michael J. Black - CS295-7 2005 Brown University

… …

“cell 8” “cell 18”

Michael J. Black - CS295-7 2005 Brown University

… …

noise x H z

t t

+ = v v

Michael J. Black - CS295-7 2005 Brown University

… …

noise x H z

t t

+ = v v

⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ + ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡

n t y t t a n y n x n a y x a y x t n t

a y x h h h h h h h h h z z

y y y

η η η M M L M L L M

2 1 , , , , , 2 , 2 , 2 , 1 , 1 , 1 , , 1

Michael J. Black - CS295-7 2005 Brown University

⎟ ⎟ ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛

− − − 42 2 1 j k j k j k

z z z M firing rate vector (zero mean, sqrt) 42 X 6 matrix

⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛

k k k k k k

y x y x

a a v v y x

system state vector (zero mean)

GENERATIVE MODEL

Observation Equation:

k k j k

q x H z v v v + =

−

Michael J. Black - CS295-7 2005 Brown University

Assumption

Gaussian distribution:

) , ( ~ ) , ( ~ Q N q x H z Q x H N z

k k k k k

v v v v v = −

Recall:

) / ) ( 2 1 exp( 2 1 ) (

2 2 σ

µ σ π − − = x x p

Michael J. Black - CS295-7 2005 Brown University

Gaussian

∏

=

=

n i t t i t t n t t

x z p x z z z p

1 , , , 2 , 1

) | ( ) | , , , ( v v K

) / )) ( ( 2 1 exp( 2 1 ) | (

2 2 , , , , , ,

σ σ π

t y a i t y i t x i t i t t i

a h y h x h z x z p

y

+ + + − − = L v For a single cell: What about multiple cells? If the firing rates are conditionally independent:

If we know xt, then the firing rates of the other cells tell us nothing more about zi,t

Michael J. Black - CS295-7 2005 Brown University

Covariance

∑∑

− − = − − =

x y y x y x xy

y x p y x y x ) , ( ) )( ( )] )( [( E µ µ µ µ σ

∑∑

− = − = =

x y x x x

y x p x x x ar ) , ( ) ( )] [( E ] [ v

2 2

µ µ σ

first moment second moment

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = y x x

y x

v v µ µ µ

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = − − =

yy yx xy xx T

x x C σ σ σ σ µ µ ] ) )( [( E v v v v

Michael J. Black - CS295-7 2005 Brown University

Covariance

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − − − =

−

) ( ) ( 2 1 exp | | ) 2 ( 1 ) (

1 2 / 1 2 /

µ µ π w v v v v x C x C x p

T D

Mahalanobis distance

2

∆ Multivariate Gaussian (Normal)

Michael J. Black - CS295-7 2005 Brown University

Covariance Ellipse

hyperellipsoids of constant Mahalanobis distance

2

∆

Michael J. Black - CS295-7 2005 Brown University

Some Facts

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = − − =

yy yx xy xx T

x x C σ σ σ σ µ µ ] ) )( [( E v v v v

If x and y are statistically independent then σxy=0. If σxy=0, then x and y are uncorrelated. Uncorrelated does not imply statistically independent. Uncorrelated and Gaussian does. PCA de-correlates the directions but unless the data is Gaussian, the coefficients are not statistically independent.

Michael J. Black - CS295-7 2005 Brown University

Approximation: Linear Gaussian (generative) model

… …

) , ( ~

t t j t

Q x H z v v Ν

−

• bservation model

noise x H z

t t

+ = v v

Full covariance Q matrix models correlations between cells. H models how firing rates relate to full kinematic model (position, velocity, and acceleration).

Michael J. Black - CS295-7 2005 Brown University

… …

likelihood

)) ( ) ( exp( 1 ) | (

1 T 2 1 t t t t t t t

x H z Q x H z D x z p v v v v v v − − =

− −

Approximation: Linear Gaussian (generative) model

noise x H z

t t

+ = v v

Michael J. Black - CS295-7 2005 Brown University

TRANSFORMED RATE

Firing rates are not normally distributed.

Poisson probability

) 1 ( 1 ~ + − + =

k k k

z mean z z

k k j k

q x H z v v v + =

−

~

We’ll come back to Poisson models…

Michael J. Black - CS295-7 2005 Brown University

⎟ ⎟ ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛

− − − 42 2 1 j k j k j k

z z z M firing rate vector (zero mean, sqrt) 42 X 42 matrix

L

v

, 2 , 1 ,

) , ( ~

= k k

Q N q

42 X 6 matrix

⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛

k k k k k k

y x y x

a a v v y x

system state vector (zero mean) 6 X 6 matrix

GENERATIVE MODEL

Observation Equation:

6 X 6 matrix

System Equation:

k k k

w x A x v v v + =

+ 1

L

v

, 2 , 1 ,

) , ( ~

= k k

W N w

k k j k

q x H z v v v + =

−

Michael J. Black - CS295-7 2005 Brown University

Model Fitting

How do we fit H and A?

2

|| || argmin ∑ − =

k k k H

x H z H v v

2 1

|| || argmin ∑ − =

+ k k k A

x A x A v v

Linear regression:

Michael J. Black - CS295-7 2005 Brown University

T 1 1 1

) )( ( ) } ({

k k k k k k k

A A x A x W x x x x cov − − = − =

+ + +

v v

T

) )( ( ) } ({ x z x z cov H H x H z Q

k k k

− − = − = v v

TRAINING

Centralize the training data, such that

}) ({ E , }) ({ E = =

k k

x z v v

What about the covariance matrices?

Michael J. Black - CS295-7 2005 Brown University

Matlab

z = rates'; x = kinematics'; H = z*x'*inv(x*x'); Q = (z-H*x)*(z-H*x)'/size(z,2);

2

|| || argmin ∑ − =

k k k H

x H z H v v

Michael J. Black - CS295-7 2005 Brown University

normalization constant (independent of mouth) Prior (a priori – before the evidence) Likelihood (evidence)

Bayesian Inference

Posterior a posteriori probability (after the evidence)

) firing ( ) kinematics ( ) kinematics | firing ( ) firing | kinematics ( p p p p =

We infer hand kinematics from uncertain evidence and our prior knowledge of how hands move.

Michael J. Black - CS295-7 2005 Brown University

Two Tasks