Reach and grasp by people with tetraplegia using a neurally - PowerPoint PPT Presentation

Leigh R. Hochberg et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm Nature, 17 May 2012 Paper overview Ilya Kuzovkin 11 April 2014, Tartu

etc…

How it works? etc…

Reach and grasp by people with tetraplegia using a neurally controlled 2012 robotic arm Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems 2011 Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia Neural control of computer cursor velocity by decoding motor cortical 2010 spiking activity in humans with tetraplegia Neuronal ensemble control of prosthetic devices by a human with tetraplegia 2006 Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter

Reach and grasp by people with tetraplegia using a neurally controlled 2012 robotic arm Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems 2011 Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia Neural control of computer cursor velocity by decoding motor cortical 2010 spiking activity in humans with tetraplegia Neuronal ensemble control of prosthetic devices by a human with tetraplegia 2006 Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter

“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.” “Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”

“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”

“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”

“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.” Posterior probability Likelihood Prior probability Hypothesis Marginal likelihood (can be (hand motion) ignored since it is the same Evidence (sequence of for all hypothesis) observed firing rates)

“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”

“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.” “ Likelihood term models the probability of firing rates given a particular hand motion”

“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.” “ Likelihood term models the probability of firing rates given a particular hand motion” “ linear Gaussian model could be used to approximate this likelihood and could be readily learned from a small amount of training data ”

“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.” “ Likelihood term models the probability of firing rates given a particular hand motion” “ linear Gaussian model could be used to approximate this likelihood and could be readily “The prior term defines a learned from a small amount probabilistic model of hand of training data ” kinematics and was also taken to be a linear Gaussian model.”

Neural Coding

Neural Coding of Hand Kinematics

Neural Coding of Hand Kinematics

Neural Coding of Hand Kinematics

Neural Coding of Hand Kinematics Experiment 1: 23/25 neurons are correctly described by equations (4) and (5) � Experiment 2: 39/42 neurons correctly described by (4) and (5)

Neural Coding of Hand Kinematics The relationship between the kinematics of the arm and the behavior of the neurons is strong Experiment 1: 23/25 neurons are correctly described by equations (4) and (5) � Experiment 2: 39/42 neurons correctly described by (4) and (5)

Learning the model

Detour: Multivariate normal distribution

Detour: Multivariate normal distribution

Detour: Multivariate normal distribution Why covariance matrix and not just a vector of variances?

Definitions

Definitions

Parameters of the model

Parameters of the model H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise

Parameters of the model H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise A is the relation between the state at time k+1 and the state at time k W is covariance matrix of the noise

Parameters of the model H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise A is the relation between the state at time k+1 and the state at time k W is covariance matrix of the noise Matrices A , H, Q, W is what we want to learn from the training data

The Learning

Decoding

“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …” Note that now x and z and everything else refer to the test data

“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”

“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …” The probability that the hand can move in the way it did

“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …” The probability The that hand can end up probability that the in the state where it hand can move in was in time k-1 the way it did

“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …” “… the Kalman filter operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state.” (Wikipedia)

“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”

“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”

“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”

Results

Reach and grasp by people with tetraplegia using a neurally controlled 2012 robotic arm Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems 2011 Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia Neural control of computer cursor velocity by decoding motor cortical 2010 spiking activity in humans with tetraplegia Neuronal ensemble control of prosthetic devices by a human with tetraplegia 2006 Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter

2006 2010 2013 2011

The steady-state Kalman filter significantly increases the computational efficiency for even relatively simple neural spiking data sets from a human NIS. <…> The decoding complexity is reduced dramatically by the SSKF, resulting in approximately seven-fold reduction in the execution time for decoding a typical neuronal firing rate signal.

Summary http://braingate2.org/publications.asp