A supervised learning approach based on STDP and polychronization in - - PowerPoint PPT Presentation

A supervised learning approach based on STDP and polychronization in spiking neuron networks

Hélène Paugam-Moisy1, Régis Martinez1 and Samy Bengio2

1LIRIS - CNRS - Université Lumière Lyon 2

Lyon, France http://liris.cnrs.fr

2IDIAP Research Institute

Martigny, Switzerland http://www.idiap.ch Samy is now at Google

ESANN 2007 - April, 27

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 2 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 3 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Motivation

In Spiking Neuron Networks (SNNs), information processing is based on the times of spike emissions. SNNs are a very powerful new generation of artificial neural networks but efficient learning in SNNs is not straightforward. A current track is to simulate the synaptic plasticity, as can be observed by neurobiologists [Bi and Poo,1998] but this method lacks supervised control of learning.

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 4 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Theoretical fundations

Theoretically, the use of delays increases the learning capacity of SNNs... [Maass, 1997] [Schmitt, 1999] ... but delays are rarely used in SNN models Recent advances in neural networks (ESN [Jaeger, 2001], LSM [Maass et al, 2002]) give interesting results The concept of polychronization emphasizes the importance of delays for explaining neural activity [Izhikevich, 2006]

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 5 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 6 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Problematics

A better computational power is a good point, but what about the learning algorithm ? How to take advantage of the computational power of delays ? We take advantage of polychronous groups activations to monitor activity in the network We define a supervised1 learning mechanism to control the computational power of a SNN Polychronization will help us monitor and understand the network activity.

1simplest way for us to show that polychronization can actually be a reliable

information coding

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 7 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 8 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

The model

Maintains biological plausibility within the internal network Neuron model : Spike Response Model (SRM0) [Gerstner 1997] Inspired from LSM/ESN architectures :

• input layer of spiking neurons
• recurrent randomly connected internal network
• output layer which supports a supervised learning rule

. . .

class 2 class 1

2 output cells K input cells Internal network M internal cells

input connections internal connections

• utput connections

with adaptable delay

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 9 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

The model

. . .

class 2 class 1

M internal cells

input connections internal connections with adaptable delay

K input cells Internal network 2 output cells

• utput connections

Input layer (stimulation layer) : 10 neurons Input injection

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 10 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

The model

. . .

class 1

2 output cells K input cells Internal network M internal cells

input connections internal connections

• utput connections

with adaptable delay

class 2

Internal Network : 100 neurons, 80% excitatory, 20% inhibitory Random recurrent topology Connection delays fixed (but randomly chosen) between 1 and 20 ms

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 11 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

The model

. . .

class 2 class 1

2 output cells K input cells Internal network M internal cells

• utput connections

with adaptable delay input connections internal connections

Output layer : 2 neurons : one for each target class recieves a connection from each internal neuron

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 12 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

The model . . .

class 2 class 1

2 output cells K input cells Internal network M internal cells

input connections internal connections

• utput connections

with adaptable delay

Tested on a classification task Two input patterns : Target neuron must fire before non-target neuron

20 ms Time [ms] 20 ms Time [ms] Input neurons Input neurons Stimulation pattern 1 Stimulation pattern 2

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 13 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 14 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

A two scale learning algorithm

. . .

class 2 class 1

2 output cells K input cells Internal network M internal cells

input connections internal connections

• utput connections

with adaptable delay 1

Unsupervised learning : Spike Time Dependent Plasticity (STDP) within the internal network (ms time scale) [Kempter et al., 1999]

2

Supervised mechanism : delay adaptation on output connections (at each input presentation) based on a margin criterion [Vapnik, 95]

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 15 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

• 1. Unsupervised learning algorithm

Unsupervised learning : Spike Time Dependent Plasticity (STDP) within the internal network (ms time scale) Temporal hebbian rule, suitable for SNNs At the synaptic level (local mechanism) Depending on activity going through the synapse Causality based on spike emissions order

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 16 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

• 2. Supervised learning algorithm

. . .

class 2 class 1

2 output cells K input cells Internal network M internal cells

input connections internal connections

• utput connections

with adaptable delay After the presentation of a given input pattern p, If target/non-target spikes order is OK AND If margin between target/non-target spikes > ǫ Then : pattern is well classified Otherwise,

• for target neuron :

decrement the delay (−1ms)

• for non-target neuron :

increment the delay (+1ms)

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 17 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 18 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Simulation protocol

Initial noisy stimulation : noise presented during 300 ms Learning phase : alternated presentation of two patterns Generalization phase : alternated presentation of the two noisy patterns NB : One presentation every 100 ms

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 19 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Initialization phase

20 40 60 80 100 120 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 Neuron ID Time [ms] Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 20 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Learning phase observation

Decreasing internal activity (STDP) Activity pattern different from an input to the other Margin evolution

20 40 60 80 100 120 8500 8600 8700 8800 8900 9000 9100 Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 21 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Generalization performance

Error rate with noise 4 : 4% Error rate with noise 8 : 19% Hard to discriminate by human

20 40 60 80 100 120 18900 19000 19100 19200 19300 19400 19500 19600 Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 22 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 23 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Polychronization [Izhikevich, 2006]

Definition : neuron interactions characterized by spike times following a precise temporal pattern, depending on delays.

Example :

N2 15ms 8ms N1 N3

Time [ms]

8 ms 15 ms

N3 N2 N1

If N1 emits a spike at t, and N3 at t + 7, then N2 emits a spike at t + 15. A set of such interacting neurons is called a polychronous group.

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 24 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Scanning for supported polychronous groups

Structure

Polychronous groups are supported by the topology. connections between neurons delays of the connections A given topology = a particular set of supported polychronous groups Each neuron can be involved in several polychronous groups To find all supported polychronous groups, we use the same algorithm as [Izhikevich 2006].

Dynamics

set of supported polychronous groups = set of activated polychronous groups

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 25 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 26 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Polychronous groups activations

20 40 60 80 100 60 65 70 75 80 85 90 95 100

# polychronous groups

activations in response to class 1 activations in response to class 2 20 40 60 80 100 60 65 70 75 80 85 90 95 100

Percentage of activation # polychronous groups

activations in response to class 1 activations in response to class 2

Figure: Activation ratio from 2000 to 5000 ms, and then from 8000 to 11000 ms.

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 27 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1

Motivations

2

Problematics

3

Network architecture

4

Learning mechanisms

5

Results (1)

6

Polychronization

7

Results (2)

8

Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 28 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Conclusion

Algorithm easy to implement The learning seems to work on a classification task Easily explained by polychronization Activity easily monitored with polychronous groups Internal network is no longer a black-box contrary to ESN and LSM

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 29 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Perspectives

Topology Dynamics Polychronous groups STDP

Complex network analysis : Are polychronous groups the (or a part of the) link between topology and dynamics How far ?

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 30 / 31

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Thank you for listening. Questions !

A supervised learning approach based on STDP and polychronization in spiking neuron networks – Hélène Paugam-Moisy, Régis Martinez and Samy Bengio

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 31 / 31

Appendix

Plan

9

Appendix Work in progress Reservoir Computing perspectives Groupes polychrones sur 100 neurones Modèle SRM0 Modèle SRM1 Forme d’un PPS Réseau expérimental Sensibilité à un motif spécifique Fenêtre STDP Eurich Fenêtre STDP classique Fenêtre STDP Meunier Stabilité du classifieur Codage temporel Architecture Activation des groupes polychrones Activité neuronale PG detection

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 32 / 31

9

Appendix Work in progress Reservoir Computing perspectives Groupes polychrones sur 100 neurones Modèle SRM0 Modèle SRM1 Forme d’un PPS Réseau expérimental Sensibilité à un motif spécifique Fenêtre STDP Eurich Fenêtre STDP classique Fenêtre STDP Meunier Stabilité du classifieur Codage temporel Architecture Activation des groupes polychrones Activité neuronale PG detection The model proposed Original problem Difference with synfire chain Network activity

Work in progress

Use larger inputs : encouraging tests with USPS dataset 2 versus 7 : 96% success on train set, 93% on test set 3 versus 8 : 89% success on train set, 86% on test set Switch to more than two classes Extend model with persistant activity

retour

Appendix

Reservoir Computing perspectives : Open questions

Might there be links with reservoir computing. Indeed, some theoretical properties exists : point-wise separation, universal approximation, echo state properties... But still difficulties to investigate what’s going on in the reservoir (refering to special session) Polychronous groups can be a reliable way to analyse dynamics of a spiking neuron reservoir to find optimal topologies (structures)

retour Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 35 / 31

Groupes polychrones sur 100 neurones

20 40 60 80 100 10 20 30 40 50 60 70 80 Temps 20 40 60 80 100 10 20 30 40 50 60 70 80 Neurones Temps

[48] 18,24,80 (0,11,11) ==> 37 (16) — [49] 19,31,43 (3,0,5) ==> 6 (6)

20 40 60 80 100 10 20 30 40 50 60 70 80 Temps 20 40 60 80 100 10 20 30 40 50 60 70 80 Neurones Temps

[50] 19,55,76 (0,11,13) ==> 70 (16) — [51] 21,52,76 (7,7,0) ==> 11 (12)

retour

Modèle SRM0 (used)

uj(t) = η(t − tf

j )

• A : refractory periode

+

• i

wij ǫ(t − tf

i − dij)

• B : excitatory potential

retour

Modèle SRM1

uj(t) = η(t − tf

j )

• A : refractory periode

+

• i

wij

• f

ǫ(t − tf

i − dij)

• B : excitatory potential

retour

Forme d’un PPS

0.2 0.4 0.6 0.8 1

• 1

1 2 3 4 5 Upsp [mV] Temps [ms] exp(-x/Tau) retour

Réseau expérimental

retour

Sensibilité à un motif spécifique

1 2 3 4 5

d = 21 d = 15 d = 8 d = 1 t+6 t+13 t+20 t+21 21 15 8 8 1 temps [ms] Neurones du sac arrivée au neurone de sortie 1 t Neurons 1 2 3 4 5 Neurone de sortie 1 d = 8

retour

Fenêtre STDP Eurich

retour

Fenêtre STDP classique

1.0 −1.0 20 ms

tpost − tpre [ms]

−20 ms décalage temporel de la synapse (tpost − tpre) augmentation du poids

Delta W retour

Fenêtre STDP Meunier

Si ∆W ≤ 0, le poids est augmenté : wij ← wij + α ∗ (wij − wmin) ∗ ∆W Si ∆W ≥ 0, le poids est diminué : wij ← wij + α ∗ (wmax − wij) ∗ ∆W

1.0 POTENTIATION DEPRESSION −0.5 t W 10ms 20ms 100ms 1.0 t W 20ms −20ms −0.25 DEPRESSION DEPRESSION POTENTIATION

+infini −infini

retour

Stabilité du classifieur

5 10 15 20 25 30 100000 200000 300000 400000 Stabilite des reponses Temps Diagramme de Stabilite des reponses

5 10 15 20 25 100000 200000 300000 400000 Stabilite des reponses Temps Diagramme de Stabilite des reponses

retour

Codage temporel

c d b a a b c d t

t−1 Neurones Temps

Codage temporel

Composantes du vecteur

Codage en intensité

Vecteur numérique Intensité Vague de spikes dans un intervalle de codage temporel

retour

Architecture

. . .

class 2 class 1

2 output cells K input cells Internal network M internal cells

input connections internal connections

• utput connections

with adaptable delay

retour

Activation des groupes polychrones

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000 14000 15000 16000 17000 18000 19000 20000 Polychronous groups t [ms]

retour

Activité neuronale

retour

PG detection

To find all supported polychronous groups, we use the same algorithm as [Izhikevitch 2006]. It consists in scanning for spike time combination of all groups possible of 3 neurons (i.e. combinatorial quiestions), so that the spikes would trigger the firing of one or more impacted neurons, taking axonal delays into account. Il est possible de procéder de même en cherchant plus de déclencheurs, mais la complexité est accrue: O(np), avec p nombre de déclencheurs.

retour

The model proposed

. . .

class 2 class 1

M internal cells

input connections internal connections with adaptable delay

K input cells Internal network 2 output cells

• utput connections

Input layer (stimulation layer) : 10 neurons Outgoing connection probability : 0.1 Delay to central assembly : 0 ms

retour

The model proposed

. . .

class 1

2 output cells K input cells Internal network M internal cells

input connections internal connections

• utput connections

with adaptable delay

class 2

Central assembly : 100 neurons, 80% excitators, 20% inhibitors Random topology Reccurent connection probability : 0.3 Recurrent connections delay from 1 to 20 ms Spike Time Dependent Plasticity (STDP)

retour

The model proposed

. . .

class 2 class 1

2 output cells K input cells Internal network M internal cells

• utput connections

with adaptable delay input connections internal connections

Output layer : 2 neurons : one for each target class Incoming connection probability : 1 (central assembly completely projected) Adaptable delays of input connections (all initialized to 10 ms)

retour

Initial work

Originally : problem for learning binary patterns Spike responses : all or nothing Solution : allow diversity in axonal delays

100 200 300 400 500 20 40 Neurones Temps Diagramme de Pulses

retour

Difference with synfire chain

in Synfire Chains and Catastrophic Interference – J. Sougné and R. French (2001) : when an initial neuron, A, fired, a second neuron, B, would fire 151ms later, followed by a third neuron, C, that would fire 289ms later with a precision across trials

• f 1 ms

in Polychronization : computation with spikes – E. Izhikevich (2006) : Synfire chains describe pools of neurons firing synchronously, not polychronously. Synfire activity relies

• n synaptic connections having equal delays or no

delays at all. Though easy to implement, networks without delays are finite-dimensional and do not have rich dynamics to support persistent polychronous spiking.

retour

Initialization phase

20 40 60 80 100 120 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 Neuron ID Time [ms]

retour

Learning phase observation

Decreasing internal activity (STDP) Activaty pattern different from an input to the other Margin evolution

20 40 60 80 100 120 2300 2400 2500 2600 2700 2800 2900 3000 3100 3200 8500 8600 8700 8800 8900 9000 9100

retour

Generalization performance

Error rate with noise 4 : 4% Error rate with noise 8 : 19%

20 40 60 80 100 120 18500 18600 18700 18800 18900 19000 19100 19200 18900 19000 19100 19200 19300 19400 19500 19600

retour