SLIDE 1
Reach and grasp by people with tetraplegia using a neurally controlled robotic arm
Ilya Kuzovkin
11 April 2014, Tartu
Leigh R. Hochberg et al.
Nature, 17 May 2012
Reach and grasp by people with tetraplegia using a neurally - - PowerPoint PPT Presentation
Reach and grasp by people with tetraplegia using a neurally - - PowerPoint PPT Presentation
Leigh R. Hochberg et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm Nature, 17 May 2012 Paper overview Ilya Kuzovkin 11 April 2014, Tartu etc How it works? etc Reach and grasp by people with
SLIDE 2
SLIDE 3
SLIDE 4
SLIDE 5
SLIDE 6
SLIDE 7
etc…
SLIDE 8
etc…
How it works?
SLIDE 9
2012 Reach and grasp by people with tetraplegia using a neurally controlled robotic arm 2011 Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia 2010 Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia 2006 Neuronal ensemble control of prosthetic devices by a human with tetraplegia Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter
SLIDE 10
2012 Reach and grasp by people with tetraplegia using a neurally controlled robotic arm 2011 Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia 2010 Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia 2006 Neuronal ensemble control of prosthetic devices by a human with tetraplegia Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter
SLIDE 11
“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.” “Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
SLIDE 12
“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”
SLIDE 13
“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”
SLIDE 14
“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”
Hypothesis (hand motion) Evidence (sequence of
- bserved firing rates)
Posterior probability Prior probability Likelihood Marginal likelihood (can be ignored since it is the same for all hypothesis)
SLIDE 15
“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”
SLIDE 16
“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”
“Likelihood term models the probability of firing rates given a particular hand motion”
SLIDE 17
“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”
“Likelihood term models the probability of firing rates given a particular hand motion” “linear Gaussian model could be used to approximate this likelihood and could be readily learned from a small amount
- f training data”
SLIDE 18
“… uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.”
“Likelihood term models the probability of firing rates given a particular hand motion” “linear Gaussian model could be used to approximate this likelihood and could be readily learned from a small amount
- f training data”
“The prior term defines a probabilistic model of hand kinematics and was also taken to be a linear Gaussian model.”
SLIDE 19
Neural Coding
SLIDE 20
Neural Coding of Hand Kinematics
SLIDE 21
Neural Coding of Hand Kinematics
SLIDE 22
Neural Coding of Hand Kinematics
SLIDE 23
Neural Coding of Hand Kinematics
Experiment 1: 23/25 neurons are correctly described by equations (4) and (5)
- Experiment 2: 39/42 neurons correctly
described by (4) and (5)
SLIDE 24
Neural Coding of Hand Kinematics
Experiment 1: 23/25 neurons are correctly described by equations (4) and (5)
- Experiment 2: 39/42 neurons correctly
described by (4) and (5)
The relationship between the kinematics of the arm and the behavior of the neurons is strong
SLIDE 25
Learning the model
SLIDE 26
Detour: Multivariate normal distribution
SLIDE 27
Detour: Multivariate normal distribution
SLIDE 28
Detour: Multivariate normal distribution
Why covariance matrix and not just a vector of variances?
SLIDE 29
Definitions
SLIDE 30
Definitions
SLIDE 31
Parameters of the model
SLIDE 32
Parameters of the model
H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise
SLIDE 33
Parameters of the model
H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise A is the relation between the state at time k+1 and the state at time k W is covariance matrix of the noise
SLIDE 34
Parameters of the model
H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise A is the relation between the state at time k+1 and the state at time k W is covariance matrix of the noise Matrices A, H, Q, W is what we want to learn from the training data
SLIDE 35
The Learning
SLIDE 36
Decoding
SLIDE 37
“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
Note that now x and z and everything else refer to the test data
SLIDE 38
“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
SLIDE 39
“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
The probability that the hand can move in the way it did
SLIDE 40
“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
The probability that the hand can move in the way it did The probability that hand can end up in the state where it was in time k-1
SLIDE 41
“… the Kalman filter operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state.” (Wikipedia)
“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
SLIDE 42
“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
SLIDE 43
“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
SLIDE 44
“Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference …”
SLIDE 45
Results
SLIDE 46
SLIDE 47
SLIDE 48
2012 Reach and grasp by people with tetraplegia using a neurally controlled robotic arm 2011 Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia 2010 Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia 2006 Neuronal ensemble control of prosthetic devices by a human with tetraplegia Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter
SLIDE 49
2006 2010 2011 2013
SLIDE 50
The steady-state Kalman filter significantly increases the computational efficiency for even relatively simple neural spiking data sets from a human NIS. <…> The decoding complexity is reduced dramatically by the SSKF, resulting in approximately seven-fold reduction in the execution time for decoding a typical neuronal firing rate signal.
SLIDE 51