Japanese Humanoids Understanding Brain by Creating Brain Robots - - PowerPoint PPT Presentation

SLIDE 1

Predictions by Cerebellar Internal Models

Mitsuo Kawato

ATR Computational Neuroscience Labs

Discovery Channel

Topics

• 1. Understanding brain by creating brain
• 2. Basics of cerebellum
• 3. Internal models are necessary
• 4. Feedback-error-learning model
• 5. Cerebellar STDP
• 6. Neurophysiology of ocular following

responses

• 7. Chaotic resonance in inferior olive
• 8. Human cerebellum and MOSAIC

Japanese Humanoids

Understanding Brain by Creating Brain

• Robots and computers are much inferior to humans.
• This demonstrates that we do not understand brain

functions.

• Only if we try to create a brain, we can understand

information processing in the brain.

• Creating only a brain does not make sense, and a

body and its environment are essential.

SLIDE 2

Computational Neuroscience

We elucidate information processing of the brain to the extent that artificial machines, either computer programs or robots, can be built to solve the same computational problems that are solved by the brain, essentially in the same principle. We elucidate information processing of the brain to the extent that artificial machines, either computer programs or robots, can be built to solve the same computational problems that are solved by the brain, essentially in the same principle. We elucidate information processing of the brain to the extent that artificial machines, either computer programs or robots, can be built to solve the same computational problems that are solved by the brain, essentially in the same principle. We elucidate information processing of the brain to the extent that artificial machines, either computer programs or robots, can be built to solve the same computational problems that are solved by the brain, essentially in the same principle. Artificial Intelligence, Robotics Neuroscience

Biped

Schaal S, Sternad D, Osu R, Kawato M: Rhythmic arm movement is not discrete. Nature Neuroscience, 7, 1137-1144 (2004). Nakanishi J, Morimoto J, Endo G, Cheng G, Schaal S, Kawato M: Learning from demonstration and adaptation of biped locomotion, J. Robotics and Autonomous Systems, 47, 79-91 (2004).

Endo G, Nakanishi J, Morimoto J, Cheng G: Experimental studies of a neural oscillator for biped locomotion with QRIO. IEEE International Conference on Robotics and Automation, 598-604 (2005) By SONY IDL and ATR Collaborative Research

Humanoid “DB” (Dynamic Brain)

• 30 DOFs
• 190 cm height
• 80 kg weight
• Compliant
• Biomimetic
• culomotor system
• Co-designed by SRC

and KDB

Shibata, T. and Vijayakumar, S. and Conradt, J. and Schaal, S.: Biomimetic Oculomotor

• Control. Adaptive Behavior, 9, 189-208 (2001).

!"#$%! "&#'$(!

New Humanoid “CB” with SARCOS

!"#$%&'()$(*+,-.(/%0&&1

• .1(0$2&'3

!4+##5(-+)$.$,$+% !6789,(-.1(78(:; !<&9*-.=9-##5(9$,0#=-.)

An ATR/SARCOS development

SLIDE 3

Understanding Hierarchical Sensory-Motor Control by the Brain through Robot Control with Neural Decoding

Prefrontal decoding Parietal decoding CBL decoding M1 decoding

Decision Intention Internal model Muscle activity

ROBOT Human

SARCOS, ATR, CMU, NiCT

Topics

• 1. Understanding brain by creating brain
• 2. Basics of cerebellum
• 3. Internal models are necessary
• 4. Feedback-error-learning model
• 5. Cerebellar STDP
• 6. Neurophysiology of ocular following

responses

• 7. Chaotic resonance in inferior olive
• 8. Human cerebellum and MOSAIC

Cerebellum and Cerebral Cortex

)*+,-*.'/0,1 23++34'/0,1 5,60*.'/0,1 Cerebellum 7Cerebral Cortex

Weight

7777 130g 771to10771,300g

Surface Area7 50,000mm27 1to2780,000mm2 Number of Neurons7 7 1011 > 1011 Expansion from Average Primates 2.8 778' 3.2

$,-,9,..:4'*;6'<:4*;'=;+,..0>,;?,

$,-,9,..*-'$3-+,@ A,;+*+, A3-B34,60*. A,;+*+, /,;+-3.*+,-*.

C'$,-,9,..*-'(:?.,0

'5,60*.'DEFG'A,;+*+,'HEI'-,.*+0J,'+3'K-3B040*;

C'$,-,9,..*-'!0L,'*;6' '''''<:4*;'=;+,..0>,;?,

''''''''M*-*60B3'NOPPQR S0;>,-'+*KK0;>

• TDEUU''

KVDEDI /,-9*.'4,43-W7

• TDEUQ'

KVDEDU X,;,-*.'=Y77

• TDEOP'

KVDEDQ NZ"=![#R

(3'?3--,.*+03;'\3-'.,\+'+,4K3-*.'.39,

'236W77''''''A,;+*+,'(:?.,:B Z,0>]+'''/3.:4,''''''''Z'^'<''''''''),;>E ''_I`>7OGO_Q44a7'B*4,7''''+10?, ''HI`>7'''HI_44a7'B*4,77'']*.\

<:4*; $]04K

SLIDE 4

$,-,9-3[ $,-,9,..*- $344:;0?*+03; )33K

A,;+*+,'63-B*. &]*.*4:B'/M)3 M-04*-W'53+3- A,;+*+,'.*+,-*. f /,;+-*.'M-,43+3- A,;+*+,'J,;+-*. 5Ag/) M-,\-3;+*.

=;+,-4,60*+, A,;+*+,

4,6 63-B

"-,*'H_ "-,*'P' .*+,-*.

$9.4'?3-+,@ M( $9.4'( &]*.*4:B $,-,9-*.'?3-+,@

$,-,9,..*-'A,,K'(:?.,0 M3;+0;,'(:?.,:B $,-,9-*.'$3-+,@ '$,-,9-*.'$3-+,@

Private Communication between Cerebrum and Cerebellum with Closed-loop Circuits

Parallel fiber(PF) Climbing fiber(CF) Purkinje cell

Granule cell

Neural Circuit of Cerebellar Cortex

(,:-*.'$0-?:0+G'!W;*K+0?'M.*B+0?0+W'*;6' 536,.B'3\'$,-,9,..:4

'5*--[".9:B[=+3'NbOPQDR

c$.0490;>'\09,-'0;K:+B'*B'+,*?],- ''''N,--3-'B0>;*.R cM*-*..,.[\09,-[M:-`0;d,[?,..'BW;*KB, ''''?]*;>,B'0+B',\\0?*?W

)3;>'&,-4'A,K-,BB03;G')&MG'#M'NbOPFUR =;+,-;*.'536,.'&],3-W'NbOPFHR

c$,-,9,..*-'?3-+,@'*?e:0-,B'0;+,-;*.'436,.B ''''9W'.,*-;0;> c$.0490;>'\09,-'-,K-,B,;+B'43+3-'?344*;6' '''',--3-

53BBW'\09,-B M*-*..,.'\09,-B M:-`0;d,'?,.. =;\,-03-'3.0J,' ;,:-3;B 53BBW'\09,- X-*;:.,'?,.. M*-*..,.'\09,- M:-`0;d, ?,.. M:-`0;d, ?,.. !+,..*+,'?,..B 2*B`,+'?,.. $.0490;> \09,- M:-`0;d, $,..'*@3; X3.>0 ?,.. X.34,-:.:B $.0490;>'\09,-B X-*;:.,'?,..B

SLIDE 5

Parallel Fiber and Climbing Fiber Inputs to Purkinje cells: Simple Spikes and Complex Spikes

A

Purkinje cell Climbing fiber (CF) Parallel fiber (PF)

Complex spike (CS)

Error signal Input signal Output signal Error signal

B

Complex spike (CS) Error signal Simple spike (SS) Internal model output

Topics

• 1. Understanding brain by creating brain
• 2. Basics of cerebellum
• 3. Internal models are necessary
• 4. Feedback-error-learning model
• 5. Cerebellar STDP
• 6. Neurophysiology of ocular following

responses

• 7. Chaotic resonance in inferior olive
• 8. Human cerebellum and MOSAIC

Cerebellar Internal Model Theory

• The cerebellum consists of many modules

(micro-zones) which perform different input-

• utput transformations.
• Synaptic weights change and different

transformations can be learned.

• Supervised learning guided by an error signal
• Different modules acquire internal models of

controlled objects, tools, other brains, etc.

Pole and CBLM

"''S,,69*?`'?3;+-3. 2''S,,6\3-1*-6'?3;+-3.

$3;+-3..,6 39d,?+

A,.*W

X*0; A,B0-,6 +-*d,?+3-W #,*.0L,6 +-*d,?+3-W S,,69*?` 43+3- ?344*;6 !+0\\;,BBG'/0B?3B0+W A,B0-,6 +-*d,?+3-W $3;+-3..,6 39d,?+ =;J,-B, 436,. S,,6\3-1*-6 43+3- ?344*;6 #,*.0L,6 +-*d,?+3-W

SLIDE 6

Neuroscience: Tilting Against a Major Theory of Movement Control

Stiffness Measurement by PFM

Hiroaki Gomi Mechanical perturbations in 8 directions were given during point-to-point movement and reaction forces were measured. Then stiffness and viscosity were estimated.

2''S,,6\3-1*-6'?3;+-3.'9W'0;J,-B,'436,.

A,B0-,6 +-*d,?+3-W 53+3-' ?344*;6 $3;+-3..,6' 39d,?+

h [

#,*.0L,6 +-*d,?+3-W S3-1*-6' 436,. $:--,;+'B+*+, S,,69*?` ?3;+-3..,- A,B0-,6 +-*d,?+3-W $3;+-3..,6' 39d,?+ #,*.0L,6 +-*d,?+3-W 53+3-' ?344*;6 =;J,-B,' 436,.

"''=;+,-;*.'\,,69*?`'?3;+-3.'9W'\3-1*-6'436,.

iB+04*+,6' +-*d,?+3-W

SLIDE 7

A,B0-,6 +-*d,?+3-W

1 2 1 2

1 2

1 = (M2L1

2 + 2M2L1S2cos2 + I1 + I2)˙˙

1 + (M2L

1S2cos 2 + I2)˙˙

2 M2L

1S2(2 ˙

1 + ˙ 2) ˙ 2sin 2 + B

1 ˙

1 2 = (M2L1S2cos2 + I2)˙˙ 1 + I2˙˙ 2 + M2L1S2 ˙ 1

2sin 2 + B 2 ˙

2

• 1
• 2

˙

• 1

˙ 2 ˙˙

• 1

˙˙

• 2

1 2

1 2 ˙ 1 ˙ 2

1 2

1

• 2

˙ 1 ˙

• 2

" 2 $

=;J,-B,'AW;*40?B'536,. S3-1*-6'AW;*40?B'536,.

$3;+-3..,6 39d,?+ =;J,-B, 436,. S,,6\3-1*-6 43+3- ?344*;6 #,*.0L,6 +-*d,?+3-W

Topics

• 1. Understanding brain by creating brain
• 2. Basics of cerebellum
• 3. Internal models are necessary
• 4. Feedback-error-learning model
• 5. Cerebellar STDP
• 6. Neurophysiology of ocular following

responses

• 7. Chaotic resonance in inferior olive
• 8. Human cerebellum and MOSAIC

Feedback Error Learning in Cerebellum

(1) Parallel-fiber inputs represent desired trajectory (2) Simple spike represents feedforward motor command (3) Cerebellar cortex constitutes inverse model (4) Complex spike represents error in motor-command coordinate (5) LTD time window (optimal if PF leads CF)

Feedback controller

+- + +

Controlled

• bject

Actual trajectory Motor command Feedback motor command Desired trajectory Trajectory error Feedforward Motor command

Inverse model

Motor Command error Cerebellar Cortex Simple spikes Complex spikes Mossy fibers Granule cells Purkinje cells Parallel fiber Climbing fiber Feedforward motor command Jun Nakanishi and Stefan SchaaljFeedback error learning and nonlinear adaptive control. Neural Networks, 17, 1453-1451 (2004)

$34K*-0B3;'3\'S,,69*?`'i--3-'),*-;0;>'10+]' !:K,-J0B,6'),*-;0;>'\3-'53+3-'$344*;6

!""#$%&'("))*)(+"%),-,. /01")2-3"#(+"%),-,.

NOR NUR =\'+,*?]0;>'B0>;*.'\3-'43+3-'?344*;6'0B'>0J,;'*B'''''''''''''''G'+],;'+], B+,,K,B+'6,B?,;+'60-,?+03;'3\'''''\3-'+],'\3..310;>'Be:*-,6',--3-G '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''0B'>0J,;'*B'\3..31BE

desired

E = 1 2( desired ff)T (desired ff )

• d dt = (ff )T( desired ff)

d dt = (ff )Tfb

$34K*-0;>',e:*+03;B'NOR'*;6'NURG''''''*KK-3@04*+,B'''''''''''''''''''''''''0;'+],' \,,69*?`',--3-'.,*-;0;>'B?],4,E''&]*+'0BG'+],'\,,69*?`'43+3-'?344*;6 K.*WB'*'-3.,'3\'+],',--3-'B0>;*.'\3-'+],'43+3-'?344*;6E'''&]0B'13-`B'3;.W 1],;'+],'\,,69*?`'?3;+-3..,-'0B'*;'*KK-3@04*+,'0;J,-B,'436,.'3\'+],' ?3;+-3..,6'39d,?+E''&],'\3..310;>'+13'?3;60+03;B'B]3:.6'9,'B*+0B\0,6E' '''''&],'?3443;'?33-60;*+,'\-*4,'\3-''''''''*;6'''''''''''E '''''&],'+13'+,4K3-*.'1*J,\3-4B''''''*;6''''''''''''''''''''''''*-,'B040.*-E

(desired ff ) fb desired fb fb desired ff

SLIDE 8

)&A'*;6')&M'*B'!W;*K+0?'M.*B+0?0+W'\3-' +],'S,,69*?`'i--3-'),*-;0;>'!?],4,

/-41+"(+-,"%)(-,1056*05105(4*#"+(*7(80)'-,9"(&"++ 536,.'3\')&A'*;6')&M

NOR NUR

:,5")1)"5%5-*,($;(7""#$%&'("))*)(+"%),-,.

NaR <05105=((((((((((((((((!-)-,.(*7(-65>(1%)%++"+(7-$")(-,105=(((((

• 65>(3;,%15-&(?"-.>5=((((((((((((((((((((@04$")(*7(-,1053=!

y n xi i

y = i xi

• n

A>"(#-77")",&"(*7(&+-4$-,.(7-$")(-,1053(7)*4(-53(31*,5%,"*03(+"2"+ ((((((((((((((((((",&*#"3(5>"(7""#$%&'(4*5*)(&*44%,#B!!!!

C C

spont

di dt = xi(C Cspont)

di dt = (ff i) fb = ((y) i)fb = xi(C Cspont)

"KK-3@04*+,'50--3-[=4*>,'#,.*+03;B]0K 2,+1,,;'$!'*;6'!!

(1) (2) (3) (4)

SS t

• =

wi

n

• t
• xi t
• dwi t
• dt = xi t
• {CS t
• CSspont} wi t
• wi t
• ~ xi t
• {CS t
• CSspont}

SS t

• =

xi t

• {CS t
• CSspont}

n

• xi t
• ~ {CS t
• CSspont}

M-,60?+03; S,,69*?`G'"\+,-'),*-;0;> k'40--3-'04*>, S,,69*?`G'2,\3-,'),*-;0;> k';3'-,.*+03; S,,6\3-1*-6G'A,.*WG'"\+,-'),*-;0;> k'$!'*+'9,>0;;0;> S,,6\3-1*-6G'(3'A,.*WG'"\+,-'),*-;0;>k'$!'J*;0B],B

Topics

• 1. Understanding brain by creating brain
• 2. Basics of cerebellum
• 3. Internal models are necessary
• 4. Feedback-error-learning model
• 5. Cerebellar STDP
• 6. Neurophysiology of ocular following

responses

• 7. Chaotic resonance in inferior olive
• 8. Human cerebellum and MOSAIC

Spike Timing, Ca2+, and LTD

Climbing fiber (CF) Parallel fiber (PF) Spine PF-CF input t = +100ms [Ca2+]i increase AMPA Receptor # decreases at the PF synapse

SLIDE 9

Computational Theory of Cerebellar STDP

• Computational theory requires that the

temporal window should exist in cerebellar LTD

• Behavioral learning experiments suggest

the temporal window in cerebellar LTD

• Computational model of Ca2+ dynamics

within spine coherently reproduces LTD data.

• Signal transduction pathway model of LTD

Adaptation to Various Force Fields

• Forward reaching with 25 cm

and 600 msec

• Two force fields VF and DF

suddenly changed from NF

• Hand path, joint angle, joint

torque measured and estimated

• Stiffness ellipse measured

Burdet E, Osu R, Franklin D, Milner T, Kawato M: The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature, 414 446-449 (2001). Osu R, Burdet E, Franklin DW, Milner TE , Kawato M: Different mechanisms involved in adaptation to stable and unstable

• dynamics. Journal of Neurophysiology, 90, 3255-3269 (2003).

Franklin DW, Osu R, Burdet E, Kawato M, Milner TE: Adaptation to stable and unstable environments achieved by combined impedance control and inverse dynamics model. Journal of Neurophysiology, 90, 3270-3282 (2003). Franklin DW, Burdet E, Osu R, Kawato M, Milner TE: Functional significance of stiffness in adaptation of multijoint arm movements to stable and unstable dynamics. Experimental Brain Research, 151 145-157 (2003). Osu R, Hirai S, Yoshioka T, Kawato M: Random presentation enables subjects to adapt to two opposing forces on the hand. !Nature Neuroscience, 7, 111-112 (2004).

Data supporting FEL with Time Advance and Cross-Muscle Reflex

A: incorporation of feedback command into feedforward command in next trial with phase advance for the stretched and main muscle compensating VF B: delayed feedback in un- stretched antagonist muscle, incorporated into feedforward command in next trial.

CHand path and EMG of the first few movements

EMG Temporal Waveforms Reconstruction by the Model

SLIDE 10

Signaling Networks in Cerebellar LTD

Masao Ito, Nat Rev Neurosci 3, 896-902 (2002)

Simulation of Cerebellar LTD

NO Glu AMPA-R VGCC mGluR Gq PLC PKC DAG IP3 Ca2+ Store IP3R PLA2 MAPK MEK Raf AA GC Ca2+ CRHR cGMP PKG G-substrate PP2A Lyn

CF PF

Positive Feedback Loop CRF

Doi T, Kuroda S, Michikawa T, Kawato M: IP3-dependent Ca2+ threshold dynamics detect spike-timing in cerebellar Purkinje Cells. Journal of Neuroscience, 25, 950-961 (2005).

Ca2+ Dynamics Model for Coherent Understanding of Diverse and Confusing Experimental Data on LTD Time Window

• Cerebellar learning theories require LTD time

window where CF is delayed with respect to PF for 100-200 msec.

• Several experimental supports to this prediction
• Strong bundle stimulation to PF alone, uncaging
• f Ca2+ or IP3 can induce LTD without PF-CF

conjunction or time delay.

• Some experiments even reported CF preceding

PF is optimal.

• Serious doubts and criticisms on LTD as a

cellular basis of cerebellar supervised learning

Ca2+ Imaging in Purkinje-cell Spines

Wang et al., Nat Neurosci 3, 1266-1273 (2000)

SLIDE 11

Signal-Transduction Pathways detect Input Spike Timing within a Spine

PF input ! Ca2+ influx & IP3 production CF input ! Ca2+ influx

Block Diagram of Ca2+ Signaling Formulation of Biochemical Reaction

(1) Binding Reaction kf kb

d[AB]/dt = + kf[A][B] - kb[AB] Dissociation Constant, Kd=kb / kf : equilibrium point Time Constant, =1/(kf + kb): speed for convergence

A + B AB (2) Enzymatic Reaction (Michaelis-Menten) E + S ES E + P k1 k-1 kcat

E: Enzyme, S: Substrate, P: Product

(1) Binding Reaction A + B AB kf kb

Km = k-1+kcat

k1

Formulation of Biochemical Reaction

SLIDE 12

Example: GluD4GluRDEq

Glu mGluR Gq

IP3 Receptor Kinetic Model

based on Adkins and Taylor (1999)

• IP3R-opening requires binding to both IP3 and Ca2+
• Excessive Ca2+ inactivates IP3R
• Ca2+-dependent activation is assumed faster than

Ca2+-dependent inactivation

Variables and Parameters

31 / 34 Time Constant, 3 / 29 Dissociation Constant, Kd Michaelis Constant, Km 3 / 21 Concentration, [A] 3 / 12 Enzyme Turnover, Vmax Unknown/Total

• 53 ordinary differential equations and 96 parameters
• None of the unknown parameters has significant effects on

peak-time or width of the temporal window.

• Four unknown parameters (time constants and maximum

enzyme velocity regarding mGluR-Gq & IP3-Ca2+) were determined from peak-height of the temporal window.

Supralinear Ca2+ signal appears when PF is followed by CF

Ca2+ Imaging

Wang et al., (2000) Nat Neurosci

Simulation

SLIDE 13

Temporal Window of PF and CF Inputs for Ca2+ Signaling and LTD

Wang et al., (2000) Nat Neurosci 3, 1266-1273

Ca2+ Signaling is determined by Timing of PF and CF Inputs Time Window of Ca2+ Response

IP3-Increase Time Course determines LTD Time Window

Fast IP3 increase Slow IP3 increase

SLIDE 14

IP3-dependent Ca2+ Dynamics

Ca2+ Dynamics and LTD Experiments

(i) CF alone does not induce LTD (ii) PF alone does not induce LTD (iii)Conjunctive PF-CF induces LTD (iv) Ca2+ uncaging induces LTD (v) IP3 uncaging induces LTD (vi) Massive PF stimulus induces LTD

• Delay in slow PF metabotropic pathway compared with fast

CF electrical pathway as a mechanism for timing-detection

• Fast positive feedback loop generates large Ca2+ signals.
• Slow negative feedback loop shuts down the Ca2+ increase.

Schematic Model of Timing-detection Summary of Ca2+ Dynamics Model

• LTD temporal window reproduced by Ca2+ dynamics
• Under physiological conditions, only the input order PF!

CF generates large Ca2+ increase.

• Ca2+ threshold dynamics dependent on IP3
• Qualitatively different Ca2+ dynamics dependent on

different IP3 concentrations can coherently reproduce diverse data.

• Several experiments to test model predictions
• Only heterosynaptic (PF-CF) conjunctive LTD can

implement supervised learning. Homosynatic (PF alone,

• r CF alone) LTD may implement meta-learning or hyper-

parameter tuning.

• Chemical plasticity of temporal window itself is possible.

SLIDE 15

Computational Theory of Cerebellar STDP

• Computational theory requires that the

temporal window should exist in cerebellar LTD

• Behavioral learning experiments suggest the

temporal window in cerebellar LTD

• Computational model of Ca2+ dynamics

within spine coherently reproduces LTD data.

• Signal transduction pathway model of LTD

Kuroda S, Schweighofer N, Kawato M: Exploration of signal transduction pathways in cerebellar long-term depression by kinetic simulation. Journal of Neuroscience, 21 5693-5702 (2001). Purkinje cells

Climbing fiber (CF)>

AMPA R Glutamate

[Ca2+]

PLC MEK Na/Ca DAG AA MAP kinase Lyn PLA2 IP3 Raf Gq Ica

Positive feedback loop

PKC CRHR PP2A GC PKG cGMP G substrate mGlu R CRF

membrane

NO

Parallel fiber (PF)>

NO AMPA R Glutamate

[Ca

MEK Na/Ca AA MAP kinase Lyn PLA2 IP3 Raf Gq Ica

Positive feedback loop

PKC CRHR GC PKG cGMP mGlu R CRF

membrane

Na/Ca mGlu R AMPA R PKC Raf MAP kinase PLA2 AA MEK PLC Ica 2+] PP2A AA PLA2 MAP kinase G substrate G substrate PP2A PLC

[Ca2+]

DAG IP PKG 3

[Ca2+]

CRF NO Glutamate

Climbing fiber (CF)> Parallel fiber (PF)>

PKC Raf MEK AMPA R cGMP Na/Ca Lyn Gq GC mGlu R CRHR Ica AMPA R

LTD Signal Transduction Pathways AMPAR Phosphorylation

Topics

• 1. Understanding brain by creating brain
• 2. Basics of cerebellum
• 3. Internal models are necessary
• 4. Feedback-error-learning model
• 5. Cerebellar STDP
• 6. Neurophysiology of ocular following

responses

• 7. Chaotic resonance in inferior olive
• 8. Human cerebellum and MOSAIC

%?:.*-'S3..310;>'#,BK3;B,B'*;6' #,.*+,6'2-*0;'#,>03;B

i@K,-04,;+B'9W'l*1*;3G'!]06*-*G'&*`,4:-*G'l39*W*B]0',+'*.E'Ni&)'*+'&B:`:9*R #,\.,@',W,'43J,4,;+B 0;6:?,6'9W'J0B:*.'43+03; 3\'.*->,'\0,.6 A)M( 5!& /MS)

SLIDE 16

Purkinje Cell Activity of Awake Monkeys during Eye Movement Tasks

Eye movements are recorded by implanted search coil.

Ocular Following Responses: (Reflex eye movement induced by movement of large visual field)

Simple and Complex Spikes of Purkinje Cells in Monkeys during Ocular Following Responses

(Kandel 1989) (Takemura et al, 2001)

• +

+

• +

7 $,-,9,..*-'$3-+,@

X-*;:., $,..B M:-`0;d, $,..B

$,-,9-*.'$3-+,@

5& 5!&

Brain Stem

!&!

/0B:*.'$3-+,@

)X(

/0B:*.' !+04:.:B

#,+0;*

"%!

M&'''(%&

=;\,-03-'%.0J,

53BBW S09,-

M*-*..,. S09,-

$.0490;>'S09,- 53+3-'$344*;6'i--3- iW, 53J,4,;+

i%5( A)M(

IDmgn

A0-,?+03;'!,.,?+0J0+W'3\'!04K.,'*;6'$34K.,@'!K0`,B

iW,'43J,4,;+' '''''''''''''J,.3?0+W A31;'FD'6,>oB pK'FD'6,>oB !04K.,'BK0`, $34K.,@'BK0`, &04,N4BR &04,N4BR ID6,>oB

SLIDE 17

Simple spikes (purple) are maximally induced by ipsiversive or downward stimulus movement. Complex spikes (blue) are opposite. Electrical stimulation evokes ipsiversive or downward movement.

(Takemura 2001) (Kawano and Shidara1993)

Firing rate is related to magnitude of eye movement. #,?3;B+-:?+03;'3\'S0-0;>'S-,e:,;?W'\3-' A0\\,-,;+'!+04:.:B'!K,,6B'*;6'A:-*+03;

'!"#$k'\0-0;>'\-,e:,;?W "#$k',W,'43J,4,;+

f t

( ) = M˙

˙ t +

( ) + B ˙

t +

( ) + K t + ( ) + fbias

!e:*-,6'$3--,.*+03;'$3,\\0?0,;+''DEQF

SLIDE 18

Simple spike encodes eye movement rather than image motion.

(Takemura et al, 2001)

Simple spike encodes eye movement, but MST/DLPN neurons encode image motion.

(Takemura et al, 2001)

Comparing simple spike and complex spike, temporal pattern is opposite, and firing rate is very different (100 spikes/sec vs. 1 spike/sec)

A3'?.0490;>'\09,-'0;K:+B'?*--W K33-'3-'-0?]'0;\3-4*+03;q

O8U'BK0`,B'K,-'B,?3;6 M33-

cp;,@K,?+,6',J,;+'6,+,?+3- cDoO c#,0;\3-?,4,;+'.,*-;0;>

#0?]

ci--3-'B0>;*.'10+]'4*>;0+:6,'*;6'60-,?+03; cS0-0;>'K-39*90.0+W'K-3\0.,'N]0>][\-,e:,;?W ''''+,4K3-*.'K*++,-;R c\,,69*?`[,--3-[.,*-;0;>

SLIDE 19

=;B+*;+*;,3:B'\0-0;>'\-,e:,;?W'0B 0;\3-4*+03;'?*--0,-'\3-'?34K.,@ BK0`,B'N$!R'*;6'B04K.,'BK0`,B'N!!RE

! "!! #!! ! $! "!!

A

! "!! #!! ! $!! "!!!

B

! "!! #!! ! $ "!

C

D ODD UDD D DEDI DEO DEOI

i

D ID

!!'3;,'?,..

D ODD UDD D DEDU

F

D OD UD

$!'3;,'?,..

D ODD UDD D DEDI DEO

G

D ID

!!'P'?,..B

D ODD UDD 'D DEDO

&04,'N4BR $!'P'?,..B

H

Spikes/s

D I

&04,N4BR

D ODD UDD

D

aD UD OD D HD

&04,N4BR r&-0*.B N6,>R N6,>LoBR N6,>

U

• BR

J,.3?0+W *??,.,-*+03; K3B0+03; M-39*90.0+W S0-0;>'\-,e:,;?W

• Temporal profile of simple spike firing and complex spike firing for each neuron is

very similar (just opposite in sign and firing rate is different). – Complex spike firing may be a template for simple spike firing.

– Simple spike of individual cell is prescribed by complex spike.

Mirror Image Relationships between CS and SS

Complex Spike as Individual Error Signal for Simple Spike

Teacher shapes student’s action by hetero-synaptic plasticity (Long-term depression/potentiation of parallel fiber synapse by climbing fiber activation ).

SLIDE 20

$ A = U $ A = U p $ A = $ A p =

D ODD UDD D DEDI DEO D ID D ODD UDD 'D DEDO D I

• +

+

• +

7 5!& A)M( <3-0L3;+*.'?,..B /,-+0?*.'?*..B /MS)'M:-`0;d,'?,..B !04K.,'BK0`, $34K.,@'BK0`, S0-0;>'\-,e:,;?W M-39*90.0+W &04,'N4BR !K0`,oB 5&''''5!& $,-,9-*.'$3-+,@ !&! /0B:*.'$3-+,@ )X( Retina 53BBW \09,- X-*;:., ?,..B PT NOT AOS Inferior Olive EOMN Brain stem iW,' 53J,4,;+ A)M( $,-,9,..*-'$3-+,@ M:-`0;d, ?,..B M*-*..,.'\09,- $.0490;>'\09,- Motor Command Error /0B:*. !+04:.:B

• +

+

• +

! "

5!& A)M( /,-+0?*.'?,..B <3-0L3;+*.'?,..B /MS)''M:-`0;d,'$,..B $,-,9,..*-'$3-+,@ X-*;:., $,..B M:-`0;d, $,..B A)M( $,-,9-*.'$3-+,@ 5& 5!& !&! /0B:*.'$3-+,@ )X( /0B:*. !+04:.:B #,+0;* "%! M&'''(%& =;\,-03-'%.0J, 2-*0; !+,4 i%5( iW, 53J,4,;+ ?.0490;>'S09,- 53+3-'$344*;6'i--3-

" $ 2

M*-*..,.' S09,-

!04K.,'BK0`, $34K.,@'BK0`,

&04,'N4BR D7777ODD777UDD D7777ODD777UDD DEO DEDI D ID I DEDO D D D 53BBW' S09,-

M-39*90.0+W S0-0;>'\-,e:,;?W !K0`,oB

• 1. IDM representation
• 2. Mirror between CS and SS
• 3. Population coding to rate coding
• 4. Two opposite axes for CS and SS
• 5. Full learning simulation

Shidara M, Kawano K, Gomi H, Kawato M: Inverse- dynamics model eye movement control by Purkinje cells in the cerebellum. Nature, 365, 50-52 (1993). Kawato M: Internal models for motor control and trajectory planning. Current Opinion in Neurobiology, 9, 718-727 (1999).

$ A = U $ A = U

A*+*'!:KK3-+0;>'=;+,-;*.'536,.'&],3-W *;6'S,,69*?`[i--3-[),*-;0;>

NOR'!04K.,'BK0`,B'N?,-,9,..*-'3:+K:+R'*-,'1,..'-,?3;B+-:?+,6'9W'*; 0;J,-B,'6W;*40?B'436,.E NUR'/0B:*.'0;K:+B'+3'+],'?,-,9,..:4'N5!&'*;6'A)M(R'*-,';3+'1,..

• ,?3;B+-:?+,6'9W'*;'0;J,-B,'6W;*40?B'436,.G'9:+'1,..'-,?3;B+-:?+,6

9W'-,+0;*.'B.0KBE NaR'$.0490;>'\09,-'0;K:+B'N?34K.,@'BK0`,BR'3\':.+-*[.31'\0-0;>'-*+,B ?3;J,W']0>][\-,e:,;?W'43+3-'?344*;6',--3-BE NHR'A-*B+0?'?]*;>,'3\';,:-*.'?36,B'\-34'K3K:.*+03;'?360;>'+3'\0-0;>'-*+, ?360;>'*+'K*-*..,.[\09,-[M:-`0;d,[?,..'BW;*KB,B'N60-,?+03;G'BK,,6G 1*J,\3-4R'0B'B:K,-J0B,6'9W'?.0490;>'\09,-'0;K:+B'N*??,BB3-W'3K+0? BWB+,4RE

!"###

!"#$%" "# ""& !"#$&' "# "($ !"#$)* "# "&' "# $(" "# $(" "# $++ "# ('+ "# (*& !"#$(" !"#$(" !"#$(" !"#$(" !"#$(" !"#$(" "# $(" !"#$(" "# $(" !"#$(" "# $(" "# $(" "# $(" "# $(" "# $(" "# $(" !"#$(" !"#$(" !"#$(" !"#$(" !"#$("

E2 E3 I1 I 2 I 3

V1 A

1

V

2

V

3

A2 A

3

10000 p

2 +300 p +10000

36000 p

2 +120 p + 90000

p p2

r t ( ) v t ( ) $ %

p

E1

e

0.039 p

SLV t

( )

e

0.012 p

t ( )

5!&'o'A)M( $,-,9,..*-'$3-+,@

/0B:*. B+04:.:B #,+0;*. B.0K

$34K.,@'BK0`, iW, 43J,4,;+

=;60-,?+'K*+]1*W

!04K.,'BK0`,

M*-*..,.'\09,- M:-`0;d,'?,.. $.0490;>'\09,-

X,;,-*.0L,6 )0;,-'436,. U;6[ %-6,-'['\0.+,-

0;+,>-*+3- M*-*..,.'\09,- M:-`0;d,'?,.. $.0490;>'\09,-

SLIDE 21

$,-,9-*.'$3-+,@'''''''''''''$,-,9,..*-'$3-+,@

K3K:.*+03;'?360;> \0-0;>'-*+,'?360;> K-39*90.0B+0?'0;+,-;*.'436,.' 6,+,-40;0B+0?'0;+,-;*.'436,. ?3-+0?*.'6W;*40?B 0;K:+[3:+K:+'+-*;B\3-4*+03; K-39*90.0+W'6,;B0+W 434,;+B 4:.+0K.,'K,*`B ?34K,+0;>'436:.,B :;B:K,-J0B,6'.,*-;0;> B:K,-J0B,6'.,*-;0;>

7777

$,-,9-*.'$3-+,@ $,-,9-*.'$3-+,@

'

$,-,9,..:4 $,-,9,..:4

'

2*B*.'X*;>.0* 2*B*.'X*;>.0*

Neural circuit

'

Horizontal and reciprocal'

'

feedforward

'

loop

Synaptic plasticity

'

Hebbian, anti-Hebbian

'

Hetero-synaptic (LTD)

'

Hebbian and Dopamine

Learning algorithm

'

Un-supervised, statistical

'

Supervised learning

'

Reinforcement learning

Artificial NN

'

Associatron, Cognitron

'

Perceptron, MADALIN

'

Actor-critique architecture

Modularity

'

Areas, column

'

Microzone

'

Dorso-ventral channels

Function: old view

'

Everything

'

Movement coordination

'

Behavior selection

Function: new view

'

Everything

'

Everything

'

Everything

Computational principle

'

Learning statistical model

'

Learning of predictive and control model of dynamical systems

'

Learning of value funciton, reward function

' ' 7\0.+,-k';,0>]93-]336'X*:BB0*; ' ' 7]3-0L3;+*.k'60B+*;+'?30;?06,;?,

Input, output, and internal representation

'

Efficient representation based on statistics

• f the external-world data

'

Modeling of dynamics based

• n efficient representations

acquired in cerebral cortex

'

Modeling of value based on emotion and efficient cerebral representation

' ' ' ' ' ' '

Computation done

'

Compress to low-dimension

'

Learning cause-and-effect

'

Learning behavior-reward link

Essence

'

Stocahsticity

'

Regularity

'

Adaptive significance

Language analogue

'

Lexicon

'

Syntax

'

Semantics

"6*K+,6G'4360\0,6'*;6',@K*;6,6'\-34' "6*K+,6G'4360\0,6'*;6',@K*;6,6'\-34'A3W*' 3W*'NOPPPR OPPPR

Topics

• 1. Understanding brain by creating brain
• 2. Basics of cerebellum
• 3. Internal models are necessary
• 4. Feedback-error-learning model
• 5. Cerebellar STDP
• 6. Neurophysiology of ocular following

responses

• 7. Chaotic resonance in inferior olive
• 8. Human cerebellum and MOSAIC

Controversies of Inferior Olive Functions: Rhythmicity and Synchrony versus Learning Signals

• 1. Most intensive gap junctions (electrical

coupling) between IO neurons

• 2. Strong rhythmicity and synchrony under

anesthetized rodents with blockades of synaptic inputs to IO (Llinas)

• 3. No rhythmicity and little synchrony for awake

monkeys (Thach)

• 4. Leaning theory should explain gap junctions

and low firing rates

SLIDE 22

Schweighofer N, Doya K, H. Fukai, Chiron JV, Furukawa T, Kawato. M: Chaos may enhance information transmission in the inferior olive. Proc Natl Acad Sci USA., 101, 4655-4660 (2004).

Chaotic Resonance in Inferior Olive

De Zeeuw CI, Simpson JI, Hoogenraad CC, Galjart N, Koekkoek SKE, Ruigrok TJH: Microcircuitry and function of the inferior olive. Trends in Neurosciences., 21, 391-400 (1998).

Microcircuitry of Cerebello-IO Network

Biophysical model

• f IO neuron

Same initial condition Intermediate coupling Strong coupling

Schweighofer N, Doya K, Kawato M: Electrophysiological propersties of infereor olive neurons: a compartmental model. Journal of Neurophysiology 82, 804-817 (1999).

Spiking of 3x3 Cells without Inputs

SLIDE 23

Largest Lyapunov Exponent Network Mutual Information per Spike

Conclusions of IO Chaotic Resonance

• 1. Chaotic resonance can enhance information

transmission with low firing frequency.

• 2. Chaos is more efficient than synaptic noise in

information transmission.

• 3. IO network model can explain decrease of

rhythmicity and synchrony with decrease in coupling under physiological synaptic inputs.

• 4. Partial synchrony of a small subset of IO cells

was reproduced.

Topics

• 1. Understanding brain by creating brain
• 2. Basics of cerebellum
• 3. Internal models are necessary
• 4. Feedback-error-learning model
• 5. Cerebellar STDP
• 6. Neurophysiology of ocular following

responses

• 7. Chaotic resonance in inferior olive
• 8. Human cerebellum and MOSAIC

p q x y

50--3- 53:B,'M*6 53:B, M-3d,?+3- !?-,,;

p q

<*;6'BK*?,

x y

!?-,,;'BK*?,

$:-B3- &*->,+ 120s

#3+*+03;*. +-*;B\3-4*+03; =;+,>-*. +-*;B\3-4*+03;

˙

x

˙

y = p q

&-*?`0; 0;>'& >'&*B`

X*;+-W'3\'+],'5#'!?*;;,-

x y = cos sin sin cos

• p

q

SLIDE 24

Imamizu H, Miyauchi S, Tamada T, Sasaki Y, Takino R, Puetz B, Yoshioka T, Kawato M: Human cerebellar activity reflecting an acquired internal model of a new tool. Nature 403 192-195(2000)

Behavioral and Imaging Data for Learning Sessions: All Subjects Averaged

$3--,.*+,6'10+]'K,-\3-4*;?,',--3- $3--,.*+,6'10+]'+],'-3+*+03;'*\+,-'.,*-;0;> 1]0.,'+],',--3-B'1,-,',e:*.0L,6

0.6 0.4 0.2 0.0 0.8 1.0 200 250 150 100 50 11 9 7 5 3 1

i

r i 200 250 150 100 50 0.6 0.4 0.2 0.0 0.8 1.0 11 9 7 5 3 1

Number of sessions

Error- equalized

Two types of activity were observed

OE'i--3-'B0>;*.B'+]*+'>:06,'.,*-;0;> UE''"?e:0-,6'0;+,-;*.'436,.B *;6 $3;d:;?+03;B'3\

5#'B0>;*.'0;?-,*B,'\-34'9*B,.0;,'NtR M,-\3-4*;?,',--3-'0;?-,*B, \-34'9*B,.0;,'NtR

i--3-[ ,e:*.0L,6 (:49,-'3\'B,BB03;B

5#'B0>;*.'0;?-,*B,'\-34'9*B,.0;,'NtR

(:49,-'3\'B,BB03;B i--3-[ ,e:*.0L,6

M,-\3-4*;?,',--3-'0;?-,*B, \-34'9*B,.0;,'NtR

#,BK3;B090.0+W'B0>;*.k'-,.*+0J,'>336;,BB'3\ M-,60?+03;'9W'\3-1*-6'436,.B Z,0>]+0;>'3\'43+3-'?344*;6B'\-34 =;J,-B,'436,.B X*+0;>'3\'\3-1*-6'436,.'.,*-;0;> X*+0;>'3\'0;J,-B,'436,.'.,*-;0;>

d i dt = i t

( ) d i

di uD t

( ) ui t ( )

( )

dwi dt = i t

( ) di

dwi xA t

( ) xi t ( )

( )

uE t

( )=

i t

( ) ui t ( )

i=1 n

• exp xA t

( ) xj t ( )

2

{ }

j=1 n

• exp xA t

( ) xi t ( )

2

{ }

i t

( ) =

NOR NHR NaR NUR

i

Switching Modules by Responsibility Signal (Goodness of Prediction)

NOR NaR NHR

• +
• +
• +
• +
• +
• +

u1 u2 un x

1

x2 xA u

D

x

n

x

D

uA !3\+'5*@

=;J,-B, 436,.'O

1 2 n

=;J,-B, 436,.'U =;J,-B, 436,.'; \3-1*-6 436,.'O \3-1*-6 436,.'; \3-1*-6 436,.'U

Wolpert D, Kawato M: Multiple paired forward and inverse models for motor control. Neural Networks 11, 1317-1329 (1998).

Multiple Internal Models

Imamizu H, Kuroda T, Miyauchi S, Yoshioka T, Kawato M: Modular organization of internal models of tools in the human cerebellum. Proc Natl Acad Sci USA., 100, 5461-5466 (2003).

X (mm) Y (mm) Right Left Anterior Posterior Z ( m m ) S u p e r i

• r

I n f e r i

• r

Superior Z (mm) Inferior

SLIDE 25

Major part of MOSAIC is in cerebellum Prefrontal cortex; gating and MOSAIC cerebellum

Imamizu H, Kuroda T, Yoshioka T, Kawato M: Functional magnetic resonance imaging examination of two modular architectures for switching multiple internal models. Journal of Neuroscience 24, 1173-1181(2004)