The BrainScaleS physical model machine From commissioning to real world problem solving 5 th Neuro Inspired Computa>onal Elements Workshop NICE 2017 Karlheinz Meier Ruprecht-Karls-Universität Heidelberg meierk@kip.uni-heidelberg.de @brainscales
Why brain inspired compu>ng ? future compu>ng based on understanding biological biological informa>on informa>on processing processing need model system to test ideas Two fundamentally different modeling approaches: • NUMERICAL MODEL (Turing) represents model parameters as binary numbers can be combined to • PHYSICAL MODEL (not Turing) form a hybrid represents model parameters as physical quan>>es system → voltage, current, charge (like the biological brain)
Digital • Discrete values of physical variables • Computa>on by Boolean algebra • One wire one bit of informa>on • Signal restored aPer gate Analog • Con>nuous values of physical variables • Computa>on by component physics • One wire many bits of informa>on • Signal not restored aPer stage Nature / mixed-signal • Local analogue computa>on • Binary communica>on by spikes • Signal restora>on
Modern Neuroscience : Access to mul>ple Scales in Space and Time - 1,000,000 000 100,000 100,000 000,0 0.0001 0.0001 10,000 10,000 0 001 0.001 1,000 000 00 0.00 0 01 0.01 1,00 1,00 100 100 0.0 0.1 0 1 0.1 .1 10 10 10 10 10 e. 1 1 1 ide 1,000 000 1,000 1, 000 2014 PET imaging o Brain EEG and MEG and MEG e Lobe 100 100 0 100 00 10 ll Map 7 orders of magnitude t TMS TMS VSD VSD 10 10 0 10 10 0 ima ing imaging ima imag imag imag ag ag ag ging ing ing ing ing g fMRI fM fM fMR MR RI RI s Brain imaging i im mag gi ng lesions Nucleus 2-DG 2 r Micro ti Micr Mi Microstimulation osti osti mulati mula l tion tion imaging im 1 1 1 1 1 Size (mm) e O t Opto Optogenetics Opto Opto Optoge p g ge ge ge ge ene eneti ene ene ene tics tics tics tics Layer - Siz 0.1 0.1 1 0.1 0.1 Light microscopy Ligh ht m micr ros scopy s Field potentials Field potentials c Neuron Single units Sing Sing ngle u le u le units nits nits its its 0 0.01 .01 1 0.01 0.01 0 01 0 01 1 1 y Dendrite , Patch clam Patch clamp Patch clam mp mp p p 0.0 0.001 001 1 0 00 0 00 0 00 0.00 0.00 0.001 .0 0 0 0 01 Calcium imaging Elec Electron microscopy Electron tron micr n os sco opy Synapse 0.0001 0.0001 0.00 0.0001 0 0 0 0 00 00 0 0 0 001 01 01 1 11 orders of magnitude 1,000 000 00 0.0001 01 1,000 0.001 0.001 01 0.01 0.01 1 0.1 0.1 0 1 .1 1 1 1 100 00 00 0 1 1 1 1 10 10 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1, 1 0 0 0 0 1988 0 0 0 1 1 1 0 0 0 0 0 1 1 0 0 0 1 1 1 Time (s) Time (s) Millisecond Second Second Minut Minute te te Hour Hour Hour r r Day Day Day Month Month Month Sejnowski et al, Nature Neuroscience, 2014 Figure 1
Memory Requirement Cellular Human Brain 100 PB Cellular Rodent Brain 100 TB Cellular Mesocircuit Plas>city O(1-10x) 1 TB Cellular Neocor>cal Column Learning O(10-100x) Development O(100-1000x) 10 GB Single Cellular Model 1 MB 1 Gigaflops 1 Petaflops 1 Exaflops 1 Teraflops ComputaHonal Complexity Subcellular detail and plas>city require advances in strong scaling !
Time Scales Nature Simula>on Causality Detec>on 10 -4 s 0.1 s 1 s 1000 s Synap>c Plas>city Day 1000 Days Learning Year 1000 Years Development 12 Orders of Magnitude > 1000 Evolu>on > Millenia Millenia > 15 Orders of Magnitude
Physical Model System Con>nuous Time Integra>ng Neural Cell Membrane (+ non-linearity) dV ( ) C m dt = − g leak V − E leak V(t) R = 1/ g leak g leak [S] C m [F] E leak C m Biology(*) 10 -8 10 -10 VLSI 1 0 -6 10 -13 (*) Brette/Gerstner, J. Neurophysiology, 2005 dV ∑ ∑ ( ) + ( ) ( ) c m dt = − g leak V − E l p k g k V − E x p l g l V − E i + k l p k,l (t) exponential onset and decay (PSP shape) 0 to g max (“weights”) g k,l effective membrane time-constant c m / g total is time-dependent „Time“ is imposed by internal physics, not by external control 7
10 Ra>onales for the Physical Model System Mixed-Signal (Local analog computa>on, binary spike communica>on) Ø Driven by architecture, not devices (180nm & 65nm CMOS) Ø High Neuron Input Count (>10.000) Ø Configurability (cell parameters, connec>ons) -> Universality Ø Scalability : ChipScale (10 5 ) -> WaferScale (10 8 ) -> Systems (>10 9 ) Ø Accelera>on x10.000, consistent >me constants (1 day compressed to 10 seconds) Ø Short-term und long-term Plas>city Ø Upgradability with unchanged system architecture Ø Hybrid Opera>on, closed loop experiments Ø Non-Expert User Access Ø Objec>ve : Exploit configurability and accelera>on - rapid explora>on of large parameter spaces - cover short and long >mescale circuit dynamics - perform compu>ng in the presence of spa>al and temporal noise
BrainScaleS neural network wafer 200.000 AdEx neurons 50 Million synapses X10.000 accelera>on
Mul>-Scale Circuit High Input Count Network Chips, 400 Structure on an 8 inch Instances on Wafer, CMOS Wafer (180nm) Length Scale 1 cm network rou>ng Plas>c Synapses, 50.000.000 Million Instances on Wafer, Length Scale 10 µm, vola>le, fast, 4-bit SRAM Weights AdEx Neurons, 200.000 Instances on Wafer, Length Scale 300 µm, NON-vola>le, slow, Analog Floa>ng Gate Parameter Storage Poisson Noise Generators
Physical Model, local analogue computing, binary continuous time communication Wafer-Scale Integration of 200.000 neurons and 50.000.000 synapses on a single 20 cm wafer Short term and long term plasticity, 10.000 faster than real-time Wafer-scale integraGon of analog neural networks , J. Schemmel, J, Fieres and K. Meier In : Proceedings of IJCNN (2008), IEEE Press, 431
x 20 : 2500 PCBs
Scaling up 500 n / 100k s 200k n / 50m s 4m n / 1b s Big machine in commissioning phase since March 30 th 2016 Part the Human Brain Project (HBP) plaqorm system
Configura>on Space 40 MB for a full Wafer
Configura>on Space 40 MB for a full Wafer
Challenge and Opportunity : Variability
Pyloric rhythm of the crustacean stomatogastric ganglion 20.000.000 model networks created with 17 random cell parameters, fixed connec>vity (Neuron) 400.000 networks found with „iden>cal (de-generate)“ >ming behaviour in measured biological range Sensi>vity of single parameters within „de-generate“ solu>ons Marder, Taylor Nature Neuroscience 14, Nr 2, 2011
Variability has to be at the right place ... Marder, Taylor Nature Neuroscience 14, Nr 2, 2011
Hardware-In-the-Loop What for ? - Calibra>on - Learning - Environment - Data Separated ? Millions of parameters - network topology - neuron sizes and parameters - synap>c strengths
Conven>onal Computer calibra>on, learning, virtual environment, data Configure, load Neuromorphic Machines Read
Calibra>on Make BrainScaleS like a digital simulator ? OR Put variabiity at the right place ! By hand ? – By self learning ! Sebas>an Schmit, Paul Müller
APer hardware in-the-loop calibra>on Sebas>an Schmit et al., accepted IJCNN 2017
Feed-forward, rate-based. 4-layer spiking network MNIST classifica>on on a physical model machine performance before and aPer hardware in-the-loop learning Sebas>an Schmit et al., accepted IJCNN 2017, ISCAS 2017
MNIST classifica>on on a physical model machine Neuronal firing ac>vity aPer hardware in-the-loop learning label input 2 x hidden Sebas>an Schmit et al., accepted JCNN 2017, ISCAS 2017
Time Scales Nature + Accelerated Simula>on Real->me Model 10 -4 s 0.1 s 10 -8 s Causality Detec>on Synap>c Plas>city 1 s 1000 s 10 -4 s Day 1000 Days 10 s Learning Year 1000 Years 3000 s Development 12 Orders of Magnitude > 1000 > Millenia > Months Evolu>on Millenia > 15 Orders of Magnitude
BrainScaleS-2 New key features 62 nm prototype chip in the lab Ø Improved parameter storage Ø Hybrid plas>city by on-chip processor : on-chip loops Input : >ming correla>ons, rates, § membrane poten>als, external signals Change : synap>c weights, network § topology, neuron parameters Ø Structured neurons NMDA plateau poten>als create non- • linear dendrites Calcium spikes for coincidence • detec>on between basal and distal Ø Evalua>on system by inputs mid-2018 Na spikes (ac>on poten>als) • communicate with other neurons Ø Full-size prototypes and wafer masks by mid-2020
Final Thoughts Ø After 10 years of development the BrainScaleS large scale physical hardware system is being commissioned and delivers first results Ø Fully non-Turing, physical model computing can solve established machine learning tasks Ø 2 nd generation physical model systems start to offer very advanced accelerated local learning capabilities and exploitation of dendritic computation Goal : Build a continuously learning cognitive machine
Recommend
More recommend
