Spiking Temporal Processing Unit Michael R. Smith 1 , Aaron Hill 1 , - - PowerPoint PPT Presentation

β–Ά
spiking temporal processing unit
SMART_READER_LITE
LIVE PREVIEW

Spiking Temporal Processing Unit Michael R. Smith 1 , Aaron Hill 1 , - - PowerPoint PPT Presentation

Aug 27, 2023 252 likes β€’380 views

Photos placed in horizontal position with even amount of white space between photos and header An Efficient Implementation of a Liquid State Machine on the Spiking Temporal Processing Unit Michael R. Smith 1 , Aaron Hill 1 , Kristofer D. Carlson

slide-1
SLIDE 1

Photos placed in horizontal position with even amount of white space between photos and header

Sandia National Laboratories is a multi-mission laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. SAND NO. 2016-10652 C

1

Michael R. Smith1, Aaron Hill1, Kristofer D. Carlson1, Craig M. Vineyard1, Jonathon Donaldson1, David R. Follett2, Pamela L. Follett2,3, John H. Naegle1, Conrad D. James1, James B. Aimone1

1Sandia National Laboratories, 2Lewis Rhodes Labs, 3Tufts University

An Efficient Implementation of a Liquid State Machine on the Spiking Temporal Processing Unit

slide-2
SLIDE 2

Spiking Temporal Processing Unit

2

slide-3
SLIDE 3

Synaptic Response Functions

3

  • LIF
  • Static

πœ€ 𝑒 βˆ’ π‘’π‘—π‘˜ βˆ’ 𝑒𝑗

  • First-order response

1 πœπ‘› π‘“βˆ’

π‘’βˆ’π‘’π‘—π‘˜βˆ’π‘’π‘— πœπ‘‘

βˆ™ 𝐼 𝑒 βˆ’ π‘’π‘—π‘˜ βˆ’ 𝑒𝑗

  • Second-order response

1 𝜐1

𝑑 βˆ’ 𝜐2 𝑑 (𝑓 βˆ’π‘’βˆ’π‘’π‘—π‘˜βˆ’π‘’π‘— 𝜐1

𝑑

βˆ’π‘“

βˆ’π‘’βˆ’π‘’π‘—π‘˜βˆ’π‘’π‘— 𝜐2

𝑑

) βˆ™ 𝐼 𝑒 βˆ’ π‘’π‘—π‘˜ βˆ’ 𝑒𝑗

𝑀𝑛

π‘œ = 𝑀𝑛 π‘œβˆ’1 βˆ’ 𝑀𝑛 π‘œβˆ’1

πœπ‘› + π‘₯π‘—π‘›π‘˜ βˆ™ 𝑑(𝑒 βˆ’ π‘’π‘—π‘˜ βˆ’ 𝑒𝑗)

π‘˜ 𝑗

slide-4
SLIDE 4

Synaptic Response Functions in the STPU

4

Input Neuron 𝒆𝒋 Efficacy 1 4 3 3 1 7 1 5 1 7 2 7 2 6 1 5 2 6 2 6 2 6 3 5 2 6 4 4 2 6 5 3 … … … …

slide-5
SLIDE 5

Neuromorphic Comparison

Platform STPU TrueNorth SpiNNaker Interconnect: 3D mesh multicast 2D mesh unicast 2D mesh mutlicast Neuron Model: Basic LIF Programmable LIF Programmable Synapse Model: Programmable Binary Programmable

5

  • The STPU 3D mesh is enabled due to the temporal buffer
  • The synapse model in the STPU is implemented via the temporal buffer
slide-6
SLIDE 6

Liquid State Machine

  • Input (spike trains)
  • Maps input streams to output streams
  • Liquid (or microcircuit)
  • A recurrent neural network of spiking

neurons (leaky integrate and fire)

  • Acts a preprocessor (temporal)
  • State
  • Measure the state of the liquid at any

given time 𝑒

  • Readout neurons
  • Plastic synapses
  • By assumption, has no temporal

integration capability of its own

6

NatschlΓ€ger, T., β€œThe Liquid State Machine Framework.” Neural Micro Circuits, http://www.lsm.tugraz.at/learning/framework.html. Accessed 26 September 2016

slide-7
SLIDE 7

Living on the Edge of Chaos

  • Fading memory
  • Feedback loops and synaptic

properties

  • Do not want to evolve to a

steady state

7

Zhang, Y., Li, P., Jin, Y. & Choe, Y. (2015). A Digital Liquid State Machine With Biologically Inspired Learning and Its Application to Speech Recognition.. IEEE Trans. Neural Netw. Learning Syst., 26, 2635-2649.

slide-8
SLIDE 8

Effects of Synaptic Response Function

8

Synaptic Response Train Sep Train Rate Test Sep Test Rate SVM Dirac Delta 0.129 0.931 0.139 0.931 0.650 First-Order 0.251 0.845 0.277 0.845 0.797 Second-Order 0.263 0.261 0.290 0.255 0.868 First-Order 30 0.352 0.689 0.389 0.688 0.811 First-Order 40 0.293 0.314 0.337 0.314 0.817 First-Order 50 0.129 0.138 0.134 0.138 0.725

Red indicates the best values for default parameters Blue indicates values that improved over second-order

slide-9
SLIDE 9

Liquid State Machine Results

9

Encoding Scheme 20 15 10 5.5 3 Current Injection TrainSep 0.263 0.409 .0378 0.334 0.324 SpikeRate 0.261 0.580 0.750 0.843 0.873 SVM acc 0.868 0.905 0.894 0.873 0.868 Bit Encoding TrainSep 0.271 0.310 0.338 0.350 0.353 SpikeRate 0.434 0.497 0.544 0.592 0.634 SVM acc 0.741 0.741 0.735 0.755 0.764 Rate Encoding TrainSep 0.164 0.364 0.622 0.197 0.047 SpikeRate 0.146 0.199 0.594 0.952 0.985 SVM acc 0.747 0.733 0.643 0.601 0.548

Red indicates highest training separation and SVM classification accuracy for each encoding scheme

slide-10
SLIDE 10

Liquid State Machine Results (cont)

10

Linear Model 3 X 3 X 15 πœΎπ’Œ = πŸπŸ” 5 X 5 X 5 πœΎπ’Œ = 𝟐𝟐 4 X 5 X 10 πœΎπ’Œ = πŸπŸ” 2 X 2 X 20 πœΎπ’Œ = 𝟐𝟏 Linear SVM 0.906 0.900 0.900 0.914 LDA 0.921 0.922 0.922 0.946 Ridge Regress 0.745 0.717 0.717 0.897 Logistic Regress 0.431 0.254 0.254 0.815

Red indicates highest classification accuracy

slide-11
SLIDE 11

Photos placed in horizontal position with even amount of white space between photos and header

Sandia National Laboratories is a multi-mission laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. SAND NO. 2016-10652 C

Thank you for your time

11