Langevin type dynamics for continuous and discrete systems Lukas - - PowerPoint PPT Presentation

Langevin type dynamics for continuous and discrete systems

Lukas Kades

Cold Quantum Coffee

• Heidelberg University -

17 April 2018

Structure

◮ Motivation ◮ Langevin dynamics for

the 2D Ising model

◮ Demystification ◮ Results ◮ Applications

http://www.kdnuggets.com Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 2

Motivation

Langevin Dynamics

Langevin equation: ∂ ∂τ φx(τ) = − δS δφx(τ) + ηx(τ) with Gaussian noise: ηx(τ), ηx′(τ ′)η = 2δ(x − x ′)δ(τ − τ ′) ηx(τ)η = 0 ⇒ Aim: Application of this formalism

• n

neuromorphic hardware

http://mathsissmart.tumblr.com http://www.fair-center.eu Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 3

Motivation

Human Brain Project/BrainScaleS - Introduction

◮ Neuromorphic

computing system

◮ 1.6 million neurons ◮ 0.4 billion dynamic

synapses

◮ 10000 times faster than

their biological archetypes

Petrovici M.A., PhD Thesis, 2016 Electronic Vision(s) Group, Heidelberg University Electronic Vision(s) Group, Heidelberg University Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 4

Motivation

Human Brain Project/BrainScaleS - Stochastic interference

Ornstein-Uhlenbeck process: dueff(t) dt = Θ [µ − ueff(t)] + ση(t) with: Θ = 1 τsyn , µ = uleak+

• syni
• spks

κ(t, ts,i) Free membrane potential ueff(t):

http://www.kdnuggets.com and Petrovici M.A., PhD Thesis, 2016 Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 5

Motivation

Comparison

Langevin dynamics: ∂ ∂τ φx(τ) = − δS δφx(τ)+ηx(τ)

◮ Continuous system ◮ Gaussian noise

contribution

◮ Coupled system

⇔

BrainScaleS: dueff(t) dt = Θ [µ − ueff(t)]+ση(t)

◮ Effective two-state

system

◮ Gaussian noise

contribution

◮ Coupled system

Langevin equation for a discrete two-state system?

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 6

How could a Langevin equation look like for the Ising model?

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 7

Langevin Dynamics for the 2D Ising Model

2D Ising Model

◮ N2 states si ∈ {−1, 1} = {↓, ↑}

• n a square lattice

◮ Hamiltonian:

H = −J

• i,j

sisj − h

• i

si

◮ Second order phase transition

⇒ Can be mapped easily onto a Boltzmann machine with si ∈ {0, 1}: H = −1 2

• ij

Wijsisj −

• i

bisi

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 8

Langevin Dynamics for the 2D Ising Model

Heuristic Approach

Langevin equation: ∂ ∂τ φx(τ) = − δS δφx(τ) + ηx(τ)

⇓

Discrete Langevin equation (φx := φx(τ), φ′

x := φx(τ + ǫ)):

φ′

x = φx − ǫ δS

δφx + √ǫηx ,

⇓

Identifications S = βH, φx = sx: s′

i = si − ǫβ ∂H

∂si + √ǫηi

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 9

Langevin Dynamics for the 2D Ising Model

Heuristic Approach

Hamiltonian with si ∈ {−1, 1} = {↓, ↑}: H = −J

• i,j

sisj − h

• i

si Langevin equation: s′

i = sign

• si − ǫβ ∂H

∂si + √ǫηi

• Lukas Kades

Langevin type dynamics for continuous and discrete systems 17 April 2018 10

Langevin Dynamics for the 2D Ising Model

Numerical Results

Langevin equation: s′

i = sign

• si − ǫβ ∂H

∂si + √ǫηi

• ⇓

Improved Langevin equation with adapted noise ˜ ηi, ˜ η′

j = δ(j − i)δ(t′ − t):

s′

i = sign

• si − ǫ β

λ(ǫ) ∂H ∂si + √ǫ˜ ηi

• 0.2

0.4 0.6 0.8 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 |Magentization| β Standard MC ǫ = 0.2 ǫ = 0.4 ǫ = 0.6 ǫ = 0.8 ǫ = 1.0 0.2 0.4 0.6 0.8 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 |Magentization| β Standard MC ǫ = 0.2 ǫ = 0.4 ǫ = 0.6 ǫ = 0.8 ǫ = 1.0

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 11

• I. Is this a Langevin equation for the Ising model?

s′

i = sign

• si − ǫ β

λ(ǫ) ∂H ∂si + √ǫ˜ ηi

• II. Why does this work?
• III. Is this a MCMC algorithm with the Boltzmann

distribution P(s) ∝ exp(−βH(s)) as equilibrium distribution?

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 12

To III.: Markov Chain Monte Carlo Algorithm?

Markov Property

Update rule:

s′

i = sign

• si − ǫ β

λ(ǫ) ∂H ∂si + √ǫ˜ ηi

• Transition probabilities (Φ(x) =

x

−∞ dt 1 √ 2π exp(−t2/2)):

W (↓→↑) = W (↓ | ↑) = Φ

• − 1

√ǫ − √ǫβ λ(ǫ) ∂H ∂si

• W (↑→↓) = W (↑ | ↓) = Φ
• − 1

√ǫ + √ǫβ λ(ǫ) ∂H ∂si

• ⇒ Markov property is fulfilled.

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 13

To III.: Markov Chain Monte Carlo Algorithm?

Ergodicity and Detailed Balance

◮ Ergodicity ⇒ Yes! ◮ Detailed balance equation:

W (↑→↓)P(↑) = W (↓→↑)P(↓) ⇒ W (↓→↑) W (↑→↓)

!

= P(↑) P(↓) = exp [−β(E(↑) − E(↓))]

⇒ Detailed balance equation seems to be satisfied.

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 14

To II.: Why does this work?

◮ Limit between the cumulative Gaussian distribution and

the exponential function: Φ

• − 1

√ǫ − √ǫ λ(ǫ)x

• ∝ exp(−x)

◮ Symmetry properties of the Ising model

(H = −J

• i,j sisj − h
• i si):

E(↑) = −E(↓) = ∂H ∂si Detailed balance equation:

⇒ W (↓→↑) W (↑→↓) = Φ

• − 1

√ǫ − √ǫβ λ(ǫ) ∂H ∂si

• Φ
• − 1

√ǫ + √ǫβ λ(ǫ) ∂H ∂si

= exp [−β(E(↑) − E(↓))]

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 15

To I.: Is this is a Langevin equation for the Ising model?

s′

i = sign

• si − ǫ β

λ(ǫ) ∂H ∂si + √ǫ˜ ηi

• ⇒ No!

⇒ It is a Langevin like MCMC algorithm with Gaussian noise input. Did we learn something?

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 16

Result I: Generalised Relations

Cumulative Gaussian Distribution

0.01 0.1 1 10 −6 −4 −2 2 4 6 4 8 12 −6 −4 −2 2 4 6 nǫ(x) x ǫ = 0.3 ǫ = 0.7 ǫ = 1.0 ǫ = 2.0 exp(x) nǫ(x) x

lim

ǫ→0

Φ

• − 1

√ǫ + √ǫ x λǫ

• Φ(− 1

√ǫ)

= exp(x)+O(ǫx2) , with: λǫ = √ǫϕ

• − 1

√ǫ

• Φ
• − 1

√ǫ

• Lukas Kades

Langevin type dynamics for continuous and discrete systems 17 April 2018 17

Result I: Generalised Relations

Derivatives of the Cumulative Gaussian Distribution

0.01 0.1 1 10 −6 −4 −2 2 4 6 4 8 12 −6 −4 −2 2 4 6 nǫ(x) x m = 1, ǫ = 0.2 m = 3, ǫ = 0.2 m = 10, ǫ = 0.2 m = 30, ǫ = 0.2 exp(x) nǫ(x) x

lim

ǫ→0 ∂m ∂tm Φ

• − 1

√ǫ + √ǫt

• t=−

Hem(−1/√ǫ) √ǫHem−1(−1/√ǫ) x

∂m ∂tm Φ(− 1 √ǫ + √ǫt)

• t=0

= exp(x) + O(ǫx2)

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 18

Result II: MCMC Algorithm based on Gaussian Noise

Generalised update rule: s′

i = si + (ν − si)Θ

• −1 −

ǫβ 2λ(ǫ)∆E(ν, si) + √ǫηT

i

• with a proposal state ν.

For the Ising model this is equivalent to: ⇔ s′

i = sign

• si − ǫ β

λ(ǫ) ∂H ∂si + √ǫ˜ ηi

• Lukas Kades

Langevin type dynamics for continuous and discrete systems 17 April 2018 19

Application I: Numerical Results for Other Models

q-Potts model: Hp = −Jp

• i,j

δsi,sj , with si ∈ {1, 2, . . . , q}. Clock model: Hc = −Jc

• i,j

cos (θi − θj) , with θi = 2πn

q ) and n ∈

{1, 2, . . . , q}.

1 2 3 4 5 6 7 8 9 0.8 0.9 1 1.1 1.2 1.3 1.4 Specific Heat β Standard MC ǫ = 0.2 ǫ = 0.5 ǫ = 1.0 ǫ = 1.5 ǫ = 2.0 0.5 1 1.5 2 2.5 3 3.5 0.6 0.7 0.8 0.9 1 1.1 1.2 Specific Heat β Standard MC ǫ = 0.2 ǫ = 0.5 ǫ = 1.0 ǫ = 1.5 ǫ = 2.0

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 20

Application II: Langevin Machine

Scheme: Identifications:

◮ W ′ ii = 1 √ǫ ◮ b′ i = √ǫ λ(ǫ)bi ◮ W ′ ij = √ǫ λ(ǫ)Wij

Implicit update rule: s′

i = sign(ui + ˜

ηi) Activation function:

pi(↑) = 1 1 + exp(2Hi(↑))

⇒ Alternative implementation of the Boltzmann machine with a different update dynamic and self interaction

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 21

Does this help for a computation on the neuromorphic hardware of the BrainScaleS project? ⇒ No! Still different dynamics.

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 22

Application III: Modified Ornstein-Uhlenbeck Process

Ornstein-Uhlenbeck process (free membrane potential): dsi dt = θ(µ − si) + ση(t) Resulting activation function for θ = 1 and σ = 1: p(↑) = p(si ≥ 0) =

∞

p(si)dsi = Φ(µ) Desired activation function: p(↑) = p(si ≥ 0) = 1 1 + exp(−2µ)

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 23

Application III: Modified Ornstein-Uhlenbeck Process

Modified Ornstein-Uhlenbeck process (free membrane potential): dsi dt = θ

√

¯ ǫ λ(¯ ǫ)µ − si + sign(si) √ ¯ ǫ

• + ση(t)

Resulting activation function for θ = 1 and σ = 1: lim

¯ ǫ→0 p(↑) = lim ¯ ǫ→0 p(si ≥ 0) =

1 1 + exp(−2µ)

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 24

Application III: Modified Ornstein-Uhlenbeck Process

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −4 −3 −2 −1 1 2 3 4 −0.04 −0.02 0.02 0.04 −2−1.5 −1−0.5 0 0.5 1 1.5 2 p(↑) µ Cumulative Distribution Logistic Distribution Ornstein-Uhlenbeck (OU) Langevin Modified OU, ¯ ǫ = 0.008 Modified OU, ¯ ǫ = 0.010 Modified OU, ¯ ǫ = 0.020

• abs. dev. to log. dist.

µ

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 25

Conclusion

Results:

◮ Limit between the cumulative Gaussian distribution and

the exponential function

◮ A Langevin like MCMC algorithm based on Gaussian noise

for discrete systems

◮ Langevin machine ◮ Modified Ornstein-Uhlenbeck process

Future work:

◮ Find useful applications (other neuromorphic systems) ◮ Investigate interactions and further aspects of the

neuromorphic hardware

Lukas Kades Langevin type dynamics for continuous and discrete systems 17 April 2018 26