Supervised Learning in Structured Spiking Neural Networks 12 - - PowerPoint PPT Presentation

Introduction Background Our Approach Results Results SpiNNaker References

Supervised Learning in Structured Spiking Neural Networks12 (including a detour to SpiNNaker)

Andr´ e Gr¨ uning, Eric Nichols, Brian Gardner and Ioana Sporea Department of Computer Science University of Surrey, Guildford, UK 1st December 2015

Introduction Background Our Approach Results Results SpiNNaker References

Introduction Background Our Approach Results Results SpiNNaker References

Introduction Background Our Approach Results Results SpiNNaker References

1

Introduction

2

Background

3

Our Approach

4

Results

5

Results

6

SpiNNaker

7

References

Introduction Background Our Approach Results Results SpiNNaker References

What are we doing?

What are we doing? Formulate a supervised learning rule for spiking neural networks that

can train spiking networks containing a hidden layer(s) of neurons, can map arbitrary spatio-temporal input into arbitrary such

• utput spike patterns, ie multiple spike trains.

Why worthwhile? Understand how spike-pattern based information processing takes place in the brain. A learning rule for spiking neural networks with technical potential. Find a rule that is to spiking networks what is backprop to rate neuron networks.

Introduction Background Our Approach Results Results SpiNNaker References

Scientific Background

Human Brain Project The SPIKEFRAME project: “Structures of Learning Algorithms for Spiking Neural Networks” carried out as part of SP4 “Theoretical Neuroscience” in Ramp Up Phase and SGA1 Where are we scientifically? computational neuroscience cognitive science artificial intelligence / machine learning

Introduction Background Our Approach Results Results SpiNNaker References

Introduction Background Our Approach Results Results SpiNNaker References

1

Introduction

2

Background

3

Our Approach

4

Results

5

Results

6

SpiNNaker

7

References

Introduction Background Our Approach Results Results SpiNNaker References

Spiking Neurons

(a)

(c)

• utput spike

input spikes

• utput spike

input spikes u

(b) Spiking neurons: real neurons communicate with each other via sequences of pulses – spikes. 1 Dendritic tree, axon and cell body of a neuron. 2 Top: Spikes arrive from other neurons and its membrane potential rises. Bottom: incoming spikes on various dendrites elicit timed spikes responses as the output. 3 response of the membrane potential to incoming spikes. If the threshold θ is crossed, the membrane potential is reset to a low value, and a spike fired. From Andr´ e Gr¨ uning and Sander Bohte. Spiking neural networks: Principles and challenges. In Proceedings of the 22nd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning – ESANN, Brugge, 2014. Invited Contribution.

Introduction Background Our Approach Results Results SpiNNaker References

Spiking Neurons

Spiking Information Processing The precise timing of spikes generated by neurons conveys meaningful information. Synaptic plasticity forms the basis of learning. Changes in synaptic strength depend on relative pre- and postsynaptic spike times, and on third signals. Challenge: to relate such localised plasticity changes to learning on the network level.

Introduction Background Our Approach Results Results SpiNNaker References

Learning for Spiking NN

General Learning Algorithms for Spiking NN? There is no general-purpose algorithm for spiking neural networks. Challenge: discontinuous nature of spiking events. Various supervised learning algorithms exist, each with its own limitations eg:

network topology, adaptability (e.g. reservoir computing), limited spike encoding (e.g. latency, or spike vs no spike). size of network

Introduction Background Our Approach Results Results SpiNNaker References

Some Learning Algorithms for Spiking NN

SpikeProp: S.M. Bohte, J.N. Kok, and H. La Poutr´

• e. Spike-prop: error-backpropagation in multi-layer

networks of spiking neurons. Neurocomputing, 48(1–4):17–37, 2002 ReSuMe Filip Ponulak and Andrzej Kasi´

• nski. Supervised learning in spiking neural networks with ReSuMe:

Sequence learning, classification and spike shifting. Neural Computation, 22:467–510, 2010 Tempotron Robert G¨ utig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions. Nature Neuroscience, 9(3), 2006. doi: 10.1038/nn1643 Chronotron R˘ azvan V Florian. The chronotron: A neuron that learns to fire temporally precise spike patterns. PLoS ONE, 7(8):e40233, 2012 SPAN A. Mohemmed, S. Schliebs, and N. Kasabov. SPAN: Spike pattern association neuron for learning spatio-temporal sequences.

• Int. J. Neural Systems, 2011
• R. Urbanczik and W. Senn. A gradient learning rule for the tempotron.

Neural Computation, 21:340–352, 2009 Johanni Brea, Walter Senn, and Jean-Pascal Pfister. Matching recall and storage in sequence learning with spiking neural networks. The Journal of Neuroscience, 33(23):9565–9575, 2013 Nicolas Fremaux, Henning Sprekeler, and Wulfram Gerstner. Functional requirements for reward-modulated spile-timing-dependent plasticity. The Journal of Neuroscience, 30(40):13326–13337, 10 2010 Ioana Sporea and Andr´ e Gr¨

• uning. Supervised learning in multilayer spiking neural networks.

Neural Computation, 25(2), 2013

Introduction Background Our Approach Results Results SpiNNaker References

Introduction Background Our Approach Results Results SpiNNaker References

1

Introduction

2

Background

3

Our Approach

4

Results

5

Results

6

SpiNNaker

7

References

Introduction Background Our Approach Results Results SpiNNaker References

Our Approach

MultilayerSpiker Generalise backpropagation to Spiking Neural Networks with hidden neurons. Use stochastic neuron model

to connect smooth quantities (derivative exists) with discrete spike trains (no derivative)

Introduction Background Our Approach Results Results SpiNNaker References

Neuron model

Membrane potential uo(t) :=

• h

woh t Yh(t′)ǫ(t−t′)dt′+ t Zo(t′)κ(t−t′)dt′ , (1)

• postsynaptic neurons, h presynaptic neuron

uo membrane potential of o. woh strength of synaptic connection from h to o. Yh(t) =

th<t δ(t − th) spike train of neuron h where th are

the firing times of h Zo(t) =

to<t δ(t − to) spike train of neuron o where to are

the firing times of o.

Introduction Background Our Approach Results Results SpiNNaker References

Neuron model

Spike response kernel ǫ and reset kernel κ ǫ(s) = ǫ0 [e−s/τm−e−s/τs] Θ(s) and κ(s) = κ0e−s/τm Θ(s) , (2) spike response kernel ǫ0 = 4mV, reset kernel κ0 = −15mV, membrane time constant τm = 10ms, the synaptic rise time τs = 5ms Heaviside step function Θ(s).

Introduction Background Our Approach Results Results SpiNNaker References

Neuron model

Stochastic Intensity (instantaneous firing rate) and Spikes ρ(t) = ρ[u(t)] = ρ0 exp u(t) − ϑ ∆u

• ,

(3) firing rate at threshold ρ0 = 0.01ms−1. “threshold” ϑ = 15mV. smoothness of the threshold ∆uo = 0.2mV (output layer) or ∆uh = 2mV (hidden layer) Spikes are generated by a point process taking stochastic intensity ρo(t).

Ie in a small time interval [t, t + δt) a spike is generated with probability ρo(t)δt.

Introduction Background Our Approach Results Results SpiNNaker References

Backpropagation

Objective (“Error”) function P(zref

• |x) = exp

log (ρo(t)) Z ref

• (t) − ρo(t)dt
• ,

(4) where Z ref

• (t) =

f δ(t − ˜

tf

• ) is the target output spike train for

input x.a

• aJ. P. Pfister, T. Toyoizumi, K. Aihara, and W. Gerstner. Optimal spike-timing dependent plasticity for precise action potential firing

in supervised learning. Neural Computation, 18(6):1309–1339, 2006

Backprop approach ∆woh = ηo ∂ log P(zref|x) ∂woh (5)

Introduction Background Our Approach Results Results SpiNNaker References

Backprop approach

. . . and some ten slides later Lots of derivatives, indices, probabilities. Derivatives only possible due to smoothness of probability function. Relatively freely switching between expected values and their best estimates to be had when you only have single cast.

Introduction Background Our Approach Results Results SpiNNaker References

Backprop Weight Update

Backpropagated Error Signal δo(t) := 1 ∆uo

• Z ref
• (t) − ρo(t)
• ,

(6) Hidden-to-Output Weights ∆woh = ηo T δo(t) (Yh ∗ ǫ)(t) dt . (7) Input-to-Hidden Weights ∆whi = ηh ∆uh

• woh

T δo(t)([Yh(Xi ∗ ǫ)] ∗ ǫ)(t)dt . (8)

a

aBrian Gardner, Ioana Sporea, and Andr´

e Gr¨

• uning. Learning spatio-temporally encoded pattern transformations in structured spiking

neural networks. Neural Computation, 27(12):2548–2586, 12 2015. doi: 10.1162/NECO a 00790. Preprint available at http://arxiv.org/abs/1503.09129

Introduction Background Our Approach Results Results SpiNNaker References

Introduction Background Our Approach Results Results SpiNNaker References

1

Introduction

2

Background

3

Our Approach

4

Results

5

Results

6

SpiNNaker

7

References

Introduction Background Our Approach Results Results SpiNNaker References

Task

Task Purpose: explore the properties of the new learning algorithm. Map an input (given as a set of spike trains) to an output (given again as a set of spike trains). Simulation details:

• Brian Gardner, Ioana Sporea, and Andr´

e Gr¨

• uning. Learning

spatio-temporally encoded pattern transformations in structured spiking neural networks. Neural Computation, 27(12):2548–2586, 12 2015. doi: 10.1162/NECO a 00790. Preprint available at http://arxiv.org/abs/1503.09129.

Introduction Background Our Approach Results Results SpiNNaker References

Network Setup

20 40 60 80 100 Input spike pattern Xi 200 400 600 800 1000 Episodes 100 200 300 400 500 200 400 600 800 1000 Time (ms) Episodes 200 400 600 800 1000 1 2 3 4 Episodes Distance

E D A B C

Input Hidden Output Output neuron Hidden neuron

Left: spike rasters of input, hidden and output layers (with targets). Right top: network structure, bottom: van-Rossum distance.

Introduction Background Our Approach Results Results SpiNNaker References

Network in Action

Xi T (Xi ∗ ǫ) uh − ϑ T ([Yh(Xi ∗ ǫ)] ∗ ǫ) uo − ϑ T ∆w hi

Left: Input spike train Xi (top) and its evoked post synaptic potential Xi ∗ ǫ (bottom). Middle: Fluctuations of a hidden neuron membrane potential uh relative to a firing threshold ϑ, in response to inputs from input layer (top). The potential dependent factor of the back propagated error from hidden to input layer [Yh(Xi ∗ ∗ǫ)] bottom left: corresponding PSP (according to kernel Xi ∗ ǫ). Right: membrane potential of an output neuron uo, in response to hidden layer activity. Target indicated by dotted lines (top). Weight changes of input-to-hidden weight due to learning rule.

Introduction Background Our Approach Results Results SpiNNaker References

Experiments

8 16 24 32 40 20 40 60 80 100 Performance (%) Input patterns 8 16 24 32 40 5000 10000 15000 Episodes Input patterns

Free whi Fixed whi Single layer

Dependence of the performance on the number of input patterns and network setup. Each input pattern mapped to a unique target output of single output neuron and spike. Left: performance as a function of the number of input patterns. Right: Number of episodes to convergence in learning. Blue curves: hidden weights whi updated according to learning algorithm, red curves: fixed random weights (plus homoeostasis), green: single layer.

Introduction Background Our Approach Results Results SpiNNaker References

Experiments

2 4 6 8 10 1 2 3 4 5 nh / no Number of output spikes

no = 10

0.5 1 1.5 2 2.5 3 20 40 60 80 100 Performance (%) nh / no

no = 10 no = 20 no = 30

A B Dependence of the performance on the ratio of hidden to output neurons, and the number of target output spikes. p = 50. A unique target output spike pattern for each output neuron. (Left) Performance as a function of the ratio of hidden to output neurons. (Right) Minimum ratio of hidden to output neurons required to achieve 90% performance.

Introduction Background Our Approach Results Results SpiNNaker References

Introduction Background Our Approach Results Results SpiNNaker References

1

Introduction

2

Background

3

Our Approach

4

Results

5

Results

6

SpiNNaker

7

References

Introduction Background Our Approach Results Results SpiNNaker References

Results

Results Compared to other learning algorithms for spiking neuron networks, we can learn more input-output mappings: 20 classes or 200 individual patterns here vs 3-4 more timed output spikes: up to 10 individually timed spikes here vs 3-5 with multiple outputs: up to 30 here vs 1 Apply it! MultilayerSpiker opens up the use of spiking neural networks for technical/cognitive modelling tasks. Spiking networks are biologically plausible. Explore how computations can be done with neural networks. Next step in the Human Brain Project: Implementation on SpiNNaker, and other neuromorphic hardware.

Introduction Background Our Approach Results Results SpiNNaker References

Introduction Background Our Approach Results Results SpiNNaker References

1

Introduction

2

Background

3

Our Approach

4

Results

5

Results

6

SpiNNaker

7

References

Introduction Background Our Approach Results Results SpiNNaker References

Steps

Steps now: explore/utilise/abuse/adapt SpiNNaker middleware to deliver third signals to neurons near future: convince SpiNNaker people to adapt the middleware for third signals. far future: run truly learning spiking NN simulation on SpiNNaker. Credits to Jamie Knight for his support, and also to other SpiNNaker people!

Introduction Background Our Approach Results Results SpiNNaker References

Rule for Neurons with Targets

Rule for Neurons with direct error feedback

∆W =

• PSP(tT − tP),

if t = tT −PSP(tO − tP), if t = tO. (9) Rules of this type appear in many supervised learning schemes Pfister ReSume, Chronotron etc pp cp our equations (6) (7) t: times of target output T, presynaptic P and output spikes O with PSP(s) =

• e

−s τm − e −s τs ,

if s > 0 0,

• therwise

(10) where τm and τs are neuron and synaptic time constants, cp (2).

Introduction Background Our Approach Results Results SpiNNaker References

Target Spikes on SpiNNaker

Graphical illustration of the learning rule’s temporal parameters: tP tO tT On Spinnaker and in Nature Transform target output times tT into presynaptic spikes across a target synapse (nature: eg weak distal synapse): −PSP(tO − tP) PSP(tT−tP) tP tO tT

Introduction Background Our Approach Results Results SpiNNaker References

SpiNNaker Implementation

Changes to SpiNNaker “middleware”: Introduce third type of synapse (exhibitory, inhibitory plus target) Target presynaptic spikes bypass input buffers, proceed to postsynaptic buffer direct (playing the system) Trace put on output and target times to update synapses appropriately (definite abuse of the system) Diagram (next slide) courtesy of Jamie Knight: (showing

• riginal (blue), modified (green) and new (purple) part of the

Spinnaker “middleware”):

Introduction Background Our Approach Results Results SpiNNaker References

SpiNNaker Implementation Diagram

pre spikes target spikes STDP weights delay buffers excitatory inhibitory target weights input buffers input currents neurons

• utput spikes

postsynaptic event buffer

Introduction Background Our Approach Results Results SpiNNaker References

First Results on SpiNNaker

Some parameters Task: learn to reproduce the output spiking pattern experiment performed with a learning time of 1 second repeated 10 times. 1,000 Poisson spike trains at rate of 10Hz were put into 1 neuron. neuron learns to match precise target spike times.

Introduction Background Our Approach Results Results SpiNNaker References

First Results on SpiNNaker

Introduction Background Our Approach Results Results SpiNNaker References

Next Steps

Next Steps more neurons more patterns teaching neurons downstream of those with a direct error signal (“Backprop”) streamline SpiNNaker implementation

Introduction Background Our Approach Results Results SpiNNaker References

Introduction Background Our Approach Results Results SpiNNaker References

1

Introduction

2

Background

3

Our Approach

4

Results

5

Results

6

SpiNNaker

7

References

Introduction Background Our Approach Results Results SpiNNaker References

Links

Introduction Background Our Approach Results Results SpiNNaker References

References

S.M. Bohte, J.N. Kok, and H. La Poutr´ e. Spike-prop: error-backpropagation in multi-layer networks of spiking neurons. Neurocomputing, 48(1–4):17–37, 2002. Johanni Brea, Walter Senn, and Jean-Pascal Pfister. Matching recall and storage in sequence learning with spiking neural networks. The Journal of Neuroscience, 33(23):9565–9575, 2013. R˘ azvan V Florian. The chronotron: A neuron that learns to fire temporally precise spike patterns. PLoS ONE, 7(8):e40233, 2012. Nicolas Fremaux, Henning Sprekeler, and Wulfram Gerstner. Functional requirements for reward-modulated spile-timing-dependent plasticity.