Spiking Neural Networks Advanced Seminar Computer Engineering - - PowerPoint PPT Presentation

▶
spiking neural networks
SMART_READER_LITE
LIVE PREVIEW

Spiking Neural Networks Advanced Seminar Computer Engineering - - PowerPoint PPT Presentation

Dec 07, 2022 1.21k likes •1.59k views

Spiking Neural Networks Advanced Seminar Computer Engineering Eugen Rusakov Spiking Neural Networks Content Introduction & Motivation Human Brain Project Basics and Background Simulators Conclusion

slide-1
SLIDE 1

Spiking Neural Networks

Advanced Seminar Computer Engineering Eugen Rusakov

slide-2
SLIDE 2

Spiking Neural Networks

  • Content
  • Introduction & Motivation
  • Human Brain Project
  • Basics and Background
  • Simulators
  • Conclusion

http://www.digitaltrends.com/computing/google-deepmind-artificial-intelligence/

2 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-3
SLIDE 3

Introduction & Motivation

Spiking Neural Networks

3 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-4
SLIDE 4

Introduction

  • Artificial Intelligence (AI) is

a research area from the neuro-informatics

  • A interdisciplinary field, in

which a number of sciences and professions converge

  • Artificial Neural Networks

(ANNs) are sub-area of AI, inspired by the neuro sciences

Neuro Computer Science Artificial Intelligence Artificial Neural Networks Spiking Neural Networks

4 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-5
SLIDE 5

Introduction

Techniques Logical Deduction Planing Searching Optimization Approximation

5 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-6
SLIDE 6

Introduction

  • Searching
  • Search for a specified solution of a given problem
  • Planing
  • Plan and develop action sequences out of a problem decription which

can be executed by agents a achieve a goal

  • Optimization
  • Tasks often brings out optimization problems, which are attemped to

solve by mathimatical programming

  • Logical Deduction
  • Creating knowledge presentations for automized logic deduction

(evidence systems or logical programming)

  • Approximation
  • Deduce general rules from a given data size

6 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-7
SLIDE 7

Introduction

  • First Generation
  • Introduced by Warren McCulloch and Walter Pitts in

1943

  • Logical and arithmetical function
  • Activation function was a Step-Function
  • Simple logic functions

(a and b / a or b)

  • Generate binary values

http://www.webpages.ttu.edu/dleverin/neural_network/neural_networks.html

7 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-8
SLIDE 8

Introduction

  • Second Generation
  • Perceptron-Model introduced by Frank Rosenblatt in

1958

  • Activation functions are typically sigmoid or hyperbolic
  • Including new topologies
  • Further layer
  • More complex structures

http://de.wikipedia.org/wiki/Perzeptron

8 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-9
SLIDE 9

Introduction

  • Third Generation
  • Modulation of spike frequencies and timings
  • Increasing amount of information transmitted per time unit
  • Considering neurons as independent nodes instead as

logic gates

  • Not firing at each propagation cycle
  • Synchronous or asynchronous information processing

http://lis2.epfl.ch/CompletedResearchProjects/EvolutionOfAdaptiveSpikingCircuits/

9 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-10
SLIDE 10

Motivation

  • Develop more realistic neural networks
  • Test and prove hypothesis of biological neural circuits
  • Better learn behaviour
  • SNNs are high potential models for problems without or little explicit

knowledge

  • A virtual insect seeking food without the prior knowledge of the

environment

10 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-11
SLIDE 11

Human Brain Project

Spiking Neural Networks

11 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-12
SLIDE 12

Human Brain Project

  • EU Flagship Initiative with nearly 500 researchers of 80

institutes from 20 countries. Dimensioned for 10 years with nearly 1.20 billion euros project budget.

  • A collaboration to realise a new ICT-accelerated vision for

brain research and its applications.

  • A approach of a concerted international effort to integrate

this data in a unified picture of the brain as a single multi- level system.

https://www.humanbrainproject.eu/de

12 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-13
SLIDE 13

Human Brain Project

  • Research Areas
  • Neuroscience
  • Achieve a unified, multi-level understanding of the human brain
  • Knowledge about healthy and diseased brain from genes to behaviour
  • Computing
  • Develop novel neuromorphic and –robotic technologies
  • Develop brain simulation, robot and autonomous systems control
  • Medicine
  • Develop biologically grounded map of neurological and psychiatric

diseases based on clinical data

  • Understand the causes of brain diseases and develop new treatment

13 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-14
SLIDE 14

Human Brain Project

  • Vision and Expectations
  • The goal of the Human Brain Project is to translate these prospects

into reality, catalysing a global collaborative effort to integrate neuroscience data from around the world, to understand the human brain and ist diseases, and ultimately to emulate its computational capabilities.

https://www.humanbrainproject.eu/de

14 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-15
SLIDE 15

Basics and Background

Spiking Neural Networks

15 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-16
SLIDE 16

Basics and Background

  • Artificial Neural Networks
  • A model and abstraction of

information processing

  • Not a replication of biological

neural networks

  • Consists of neurons connected

among themselves by synapses

  • Partitioned in three layers
  • Input, hidden and output layers
  • Different topologies

http://en.wikipedia.org/wiki/User:Mariam_Hovhannisyan

16 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-17
SLIDE 17

Basics and Background

  • Topologies

Recurrent Layer Single Layer Multi Layer

http://de.wikipedia.org/wiki/K%C3%BCnstliches_neuronales_Netz

17 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-18
SLIDE 18

Basics and Background

  • Artificial Neurons
  • One or more Inputs
  • Each input can carry a

different value

  • One or more Outputs
  • Each output carry the

same value

  • Activation function with a

threshold

http://de.wikipedia.org/wiki/K%C3%BCnstliches_neuronales_Netz

18 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-19
SLIDE 19

Activation functions

  • Activation functions
  • This function gives the signals passing through the neuron a weight

and decide if a signal can pass or not.

http://imgarcade.com/1/sigmoid-activation-function/

19 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-20
SLIDE 20

Basics and Background

  • Synapses
  • Connections between neurons, transmitting the information
  • Synapses have weights, which get multiplied with the signal passing

through

2 4 10 2 4

  • 2

12

  • 1

3

20 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-21
SLIDE 21

Basics and Background

  • Example of signal passing

2.0 1.0

  • 3.0

2.0 1.0

  • 2.0
  • 1.0
  • 1.0
  • 1.0

4.0

10.0

2.5 4.0

  • 2.0

12.0

  • 2.0

3.0 0.1 1.0 0.5 2.0 5.0 2.0 0.9

21 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-22
SLIDE 22

Basics and Background

  • Learn methods
  • Supervised
  • A set of example pairs are given and the aim is to find a correct

function

  • Unsupervised
  • Some data is given and the cost function to be minimized
  • Try to create a solution without knowing the goal values
  • Reinforcement
  • Data are usually not given, but generated by an agent’s interaction

with the environment

22 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-23
SLIDE 23

Basics and Background

  • Learning Behavior
  • Learning with neuron and synapses plasticity
  • Develop new connections
  • Delete existing connections
  • Modify weights of connections
  • Modify threshold values of neurons
  • Modify activation functions
  • Initiate new neurons
  • Eliminate existing neurons

23 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-24
SLIDE 24

Basics and Background

  • Example for learning behavior

2.0 1.0

  • 3.0

2.0 1.0

  • 2.0
  • 1.0
  • 1.0
  • 1.0

4.0

10.0

2.5 4.0

  • 2.0

12.0

  • 2.0

3.0 0.1 1.0 0.5 2.0 5.0 2.0 0.9 Expected output value: 1.0

24 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-25
SLIDE 25

Basics and Background

  • Example for learning behavior

2.0 1.0 7.0 2.0 1.0 6.0 1.0 3.0 1.0 4.0

10.0

7.0 4.0

  • 2.0

12.0

  • 2.0

3.0 1.0 1.0 5.0 2.0 5.0 2.0 1.0 Expected output value: 1.0

25 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-26
SLIDE 26

Basics and Background

  • Spiking Neural

Networks

  • Increasing the

information density due to spike modulation

  • Several different

modulations for various brain areas

introduction to spiking neural networks: information processing, learning and applications (Filip Ponulak, Andrzej Kansinski)

26 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-27
SLIDE 27

Basics and Background

27 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

Deep Machine Learning on GPUs, Daniel Schlegel, Advanced Seminar

slide-28
SLIDE 28

Simulators

Spiking Neural Networks

28 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-29
SLIDE 29

Simulators

  • Brian Simulator
  • High flexible simulator for rapidly

developing new models

  • Written in the programming

language Python

  • Easy and intuitive syntax,

attractive for teaching computational neuroscience

  • Especially valuable for working
  • n non-standard neuron

models

  • Disadvantage in performance

due to interpreter language

CUBA network, using fixed 80 synapses per neuron, varying the number of neurons N

Goodman D and Brette R (2008) Brian: a simulator for spiking neural networks in

  • Python. Front. Neuroinform. doi:10.3389/neuro.11.005.2008

29 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-30
SLIDE 30

Simulators

  • Neural Simulation Tool – NEST
  • Build to simulate large networks
  • Written object-oriented in C++
  • Consists of three main components
  • Nodes: neurons, devices are handled as nodes
  • Events: Spike-, Voltage- and Current-Events
  • Connections: Channels which exchange events

30 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-31
SLIDE 31

Simulators

  • Run-time of NEST for a large network

Network of 12500 neurons (80% excitatory / 20% inhibitory) Each neuron receiving 1250 inputs Total number of synapses 15.6 millions

NEST by example: an introduction to the neural simulation tool NEST (Marc-Oliver Gewaltig and Abigail Morrison and Hans Ekkehard Plesser)

31 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-32
SLIDE 32

Simulators

  • Comparison between CPU and GPU cluster

32 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering GPU: NVIDIA Tesla C1060 cluster of 64 nodes Infiniband communication backend CPU: Cluster of 128 nodes, Intel XEON E5520 2.27GHz Infiniband communication backend Master with 48 GB and Slaves with 12 GB memory

Kirill Minkovich, Corey M. Thibeault, 2014: HRLSim A High Performance Spiking Neural Network Simulator for GPGPU Clusters

slide-33
SLIDE 33

Conclusion

Spiking Neural Networks

33 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-34
SLIDE 34

Conclusion

  • Spiking Neural Networks are a high potential model for

realistic neural network behavior.

  • Modelling higher intelligence due to more complex neural

networks with high performance computer systems like Cluster or GPU computing.

  • A neural network model with a short life due to rapidly

advances in neurosciences.

  • Assuredly there will be further generations of neural networks!

34 Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering

slide-35
SLIDE 35

Spiking Neural Networks

Questions?

Eugen Rusakov, Spiking Neural Networks, Advanced Seminar Computer Engineering 35