[PPT] - Anomalous statistics of dynamical systems on networks Stefan PowerPoint Presentation

SLIDE 1

Anomalous statistics of dynamical systems on networks

Stefan Thurner

www.complex-systems.meduniwien.ac.at www.santafe.edu

trento jul 23 2012

SLIDE 2

with R. Hanel and M. Gell-Mann PNAS 108 (2011) 6390-6394 Europhys Lett 93 (2011) 20006 Europhys Lett 96 (2011) 50003

trento jul 23 2012 1

SLIDE 3

Why are networks cool?

Tell you who interacts with whom
Same statistical system on different networks can behave totally different

trento jul 23 2012 2

SLIDE 4

How ?

Simple example: Ising spins on constant-connectency networks
Show: this is not of Boltzmann Gibbs type – give exact statistics

trento jul 23 2012 3

SLIDE 5

Why Statistics ?

Central concept: understanding macroscopic system behavior on the basis
f microscopic elements and interactions → entropy
Functional form of entropy: must encode information on interactions too!
Entropy relates number of states to an extensive quantity, plays funda-

mental role in the thermodynamical description

Hope: ’thermodynamical’ relations → phase diagrams, etc.

trento jul 23 2012 4

SLIDE 6

3 Ingredients

Entropy has scaling properties → what are entropies for non-ergodic

systems?

How does entropy grow with system size? → what n.e. system is realized?
Symmetry in thermodynamic systems → if broken:

entropy has no thermodynamic meaning → forget dream about handling system with TD

trento jul 23 2012 5

SLIDE 7

What is the entropy of strongly interacting systems?

trento jul 23 2012 6

SLIDE 8

Appendix 2, Theorem 2

C.E. Shannon, The Bell System Technical Journal 27, 379-423, 623-656, 1948.

trento jul 23 2012 7

SLIDE 9

Entropy

S[p] =

W

i=1

g(pi) pi ... probability for a particular (micro) state of the system,

i pi = 1

W ... number of states g ... some function. What does it look like?

trento jul 23 2012 8

SLIDE 10

The Shannon-Khinchin axioms

SK1: S depends continuously on p → g is continuous
SK2: entropy maximal for equi-distribution pi = 1/W → g is concave
SK3: S(p1, p2, · · · , pW) = S(p1, p2, · · · , pW, 0) → g(0) = 0
SK4: S(A + B) = S(A) + S(B|A)

Theorem: If SK1-SK4 hold, the only possibility is Boltzmann-Gibbs-Shannon entropy S[p] =

W

i=1

g(pi) with g(x) = −x ln x

trento jul 23 2012 9

SLIDE 11

Shannon-Khinchin axiom 4 is non-sense for NWs

→ SK4 violated for strongly interacting systems → nuke SK4

SK4 corresponds to weak interactions or Markovian processes

trento jul 23 2012 10

SLIDE 12

The Complex Systems axioms

SK1 holds
SK2 holds
SK3 holds
Sg = W

i g(pi) , W ≫ 1

Theorem: All systems for which these axioms hold (1) can be uniquely classified by 2 numbers, c and d (2) have the unique entropy Sc,d = e 1 − c + cd W

i=1

Γ (1 + d , 1 − c ln pi) − c e

e · · · Euler const

trento jul 23 2012 11

SLIDE 13

The argument: generic mathematical properties of g

Scaling transformation W → λW: how does entropy change ?

trento jul 23 2012 12

SLIDE 14

Mathematical property I: an unexpected scaling law !

lim

W →∞

Sg(Wλ) Sg(W) = ... = λ1−c Theorem 1: Define f(z) ≡ limx→0

g(zx) g(x) with (0 < z < 1).

Then for systems satisfying SK1, SK2, SK3: f(z) = zc, 0 < c ≤ 1

trento jul 23 2012 13

SLIDE 15

Theorem 1

Let g be a continuous, concave function on [0, 1] with g(0) = 0 and let f(z) = limx→0+ g(zx)/g(x) be continuous, then f is of the form f(z) = zc with c ∈ (0, 1].

Proof. Note that f(ab) = limx→0 g(abx)/g(x) =

limx→0(g(abx)/g(bx))(g(bx)/g(x)) = f(a)f(b). All pathological solutions are excluded by the requirement that f is continuous. So f(ab) = f(a)f(b) implies that f(z) = zc is the only possible solution of this equation. Further, since g(0) = 0, also limx→0 g(0x)/g(x) = 0, and it follows that f(0) = 0. This necessarily implies that c > 0. f(z) = zc also has to be concave since g(zx)/g(x) is concave in z for arbitrarily small, fixed x > 0. Therefore c ≤ 1.

trento jul 23 2012 14

SLIDE 16

Mathematical properties II: yet another one !!

lim

W →∞

S(W 1+a) S(W)W a(1−c) = ... = (1 + a)d Theorem 2: Define hc(a) ≡ ...

trento jul 23 2012 15

SLIDE 17

Theorem 2

Let g be like in Theorem 1 and let f(z) = zc then hc given in Eq. (8) is a constant of the form hc(a) = (1 + a)d for some constant d.

Proof. We determine hc(a) again by a similar trick as we have used for f.

hc(a) = limx→0

g(xa+1) xacg(x) = g

(xb)(a+1

b −1)+1

(xb)(a+1

b −1)cg(xb)

g(xb) x(b−1)cg(x)

= hc a+1

b

− 1

hc (b − 1)

, for some constant b. By a simple transformation of variables, a = bb′ − 1,

ne gets hc(bb′ − 1) = hc(b − 1)hc(b′ − 1). Setting H(x) = hc(x − 1) one

again gets H(bb′) = H(b)H(b′). So H(x) = xd for some constant d and consequently hc(a) is of the form (1 + a)d.

trento jul 23 2012 16

SLIDE 18

Summary

Strongly interacting systems → SK1-SK3 hold → limW →∞

Sg(W λ) Sg(W ) = λ1−c

0 ≤ c < 1 → limW →∞

S(W 1+a) S(W )W a(1−c) = (1 + a)d

d real Remarkable:

all systems are characterized by 2 exponents: (c, d) – universality class
Which S fulfills above? → Sc,d = W

i=1 re Γ (1 + d , 1 − c ln pi) − rc

Which distribution maximizes Sc,d→pc,d(x) = e

− d

1−c

Wk
B(1+x

r) 1 d

−Wk(B)
r =

1 1−c+cd, B = 1−c cd exp 1−c cd

, Γ(a, b) =

∞ b dt ta−1 exp(−t); Lambert-W : solution to x = W (x)eW (x) trento jul 23 2012 17

SLIDE 19

Holds very generically

for all non-ergodic systems
for all non-Markovian systems

(complex systems)

trento jul 23 2012 18

SLIDE 20

Examples

S1,1 =

i g1,1(pi) = − i pi ln pi + 1 (BG entropy)

Sq,0 =

i gq,0(pi) = 1−

i pq i

q−1

+ 1 (Tsallis entropy)

S1,d>0 =

i g1,d(pi) = e d

i Γ (1 + d , 1 − ln pi) − 1

d (AP entropy)

...

trento jul 23 2012 19

SLIDE 21

Classification of entropies: order in the zoo

entropy c d

SBG =

i pi ln(1/pi)

1 1

Sq<1 =

1− pq i q−1

(q < 1) c = q < 1

Sκ =

i pi(pκ i − p−κ i

)/(−2κ) (0 < κ ≤ 1) c = 1 − κ

Sq>1 =

1− pq i q−1

(q > 1) 1

Sb =

i(1 − e−bpi) + e−b − 1

(b > 0) 1

SE =

i pi(1 − e pi−1 pi )

1

Sη =

i Γ(η+1 η , − ln pi) − piΓ(η+1 η )

(η > 0) 1 d = 1/η

Sγ =

i pi ln1/γ(1/pi)

1 d = 1/γ

Sβ =

i pβ i ln(1/pi)

c = β 1 Sc,d =

i erΓ(d + 1, 1 − c ln pi) − cr

c d

trento jul 23 2012 20

SLIDE 22

Distribution functions of CS

p(1,1) → exponentials (Boltzmann distribution)
p(q,0) → power-laws (q-exponentials)
p(1,d>0) → stretched exponentials
p(c,d) all others → Lambert-W exponentials

NO OTHER POSSIBILITIES

trento jul 23 2012 21

SLIDE 23

q-exponentials Lambert-exponentials

10 10

5

10

−30

10

−20

10

−10

10 x p(x)

(b) d=0.025, r=0.9/(1−c) c=0.2 c=0.4 c=0.6 c=0.8

10 10

5

10

−20

10 x p(x)

(c) r=exp(−d/2)/(1−c)

(0.3,−4) (0.3,−2) (0.3, 2) (0.3, 4) (0.7,−4) (0.7,−2) (0.7, 2) (0.7, 4)

trento jul 23 2012 22

SLIDE 24

The world beyond Shannon

−1 1 2 1 violates K2 violates K2 Stretched exponentials − asymptotically stable (c,d)−entropy, d>0 Lambert W0 exponentials q−entropy, 0<q<1 compact support

f distr. function

BG−entropy violates K3 (1,0) (c,0) (0,0)

d c

(c,d)−entropy, d<0 Lambert W−1 exponentials

trento jul 23 2012 23

SLIDE 25

Scaling property opens door to ...

...bring order in the zoo of entropies through universality classes
...understand ubiquity of power laws (and extremely similar functions)
...understand where Tsallis entropy comes from
...understand statistical systems on networks

trento jul 23 2012 24

SLIDE 26

The requirement of extensivity

trento jul 23 2012 25

SLIDE 27

Needed for TD program to work: extensive entropies

System has N elements → W(N)... phasespace volume (system property) Extensive: S(WA+B) = S(WA) + S(WB) = · · · [use scaling property I] → Can proof: extensive is equivalent to W(N) = exp

d

1−cWk

µ(1 − c)N

1 d

c

= lim

N→∞ 1 − 1

N W ′(N) W(N) d = lim

N→∞ log W

1 N W W ′ + c − 1

Message: Growth of phasespace volume determines entropy and vice versa

trento jul 23 2012 26

SLIDE 28

Examples

W(N) = 2N → (c, d) = (1, 1) and system is BG
W(N) = N b → (c, d) = (1 − 1

b, 0) and system is Tsallis

W(N) = exp(λN γ) → (c, d) = (1, 1

γ)

...

Can explicitly verify statements in theory of binary processes and spin- systems on networks

trento jul 23 2012 27

SLIDE 29

What does this imply further ?

almost all systems are Boltzmann Gibbs type
to be non-BG: phasespace has to collapse to a set of measure zero
this means: bulk of statistically relevant degrees of freedom is frozen
only systems where dynamics is confined its surface can be non-BG

Hypothesize applications in:

Self Organized Critical systems, sandpiles ...
Spin systems with dense meta-structures, such as spin-domains, vortices,

instantons, caging, etc.

Anomalous diffusion (porous media)

trento jul 23 2012 28

SLIDE 30

2 Examples

trento jul 23 2012 29

SLIDE 31

Spin system on networks

each node i has 2 states: si = ±1 ; YES / NO (e.g. opinion)
each node i has initial (’kinetic’) energy ǫ (e.g. free will)
(anti) parallel spins add J+(−) to energy E; ∆J = J− − J+
total energy in the system: E = ǫN
spin-flip of node can occur if node has enough energy for it (microcanonic)

→ Can show entropy depends on network !!!

trento jul 23 2012 30

SLIDE 32

Phasespace volume

N nodes, L links, k = N/L, φ = N/N(N − 1)

n+ ... spins pointing up, µ cost for link

phase space volume: Ω =

N

n+

(MC partition function)
derive n+

E can be estimated by E = L [(n+(n+ − 1) + n−(n− − 1)) J+ + 2n+n−J−] N(N − 1) +µL ∼ 2φn+(N−n+)∆J and n+ = N 2

1 −
1 −

2ǫ k∆J

∼

ǫ 2φ∆J

trento jul 23 2012 31

SLIDE 33

Phasespace volume and NW growth

Example 1: NW grows such that connectivity k is constant as it grows

k = const. → n+ = aN with 0 < a < 1 constant Sterling’s approximation W = N

aN

∼ bN with b = a−a(1 − a)a−1 > 1

From before: c = 1 and d = 1 → entropy of the system is BG

Example 2: NW growth: join-a-club network

new node links to αN(t) random neighbors, α < 1 What is this ?

trento jul 23 2012 32

SLIDE 34

trento jul 23 2012 33

SLIDE 35

trento jul 23 2012 34

SLIDE 36

trento jul 23 2012 35

SLIDE 37

Phasespace volume and NW growth

Example 2: NW growth: join-a-club network

new node links to αN(t) random neighbors, α < 1 constant connectancy, φ = const. → k = φN and n+ ∼ ǫ/2φ∆J = const. W = N

n+

∼ (N/n+)n+exp(−n+) ∝ N n+

From before (c, d) = (1 −

1 n+, 0), meaning Tsallis q-entropy with q = c

Note that intermediate cases with k ∝ N γ with 0 < γ < 1, require

generalized entropies with c = 1 and d = 1/γ.

trento jul 23 2012 36

SLIDE 38

Bonus track: Super-diffusion: Accelerating random walks

10 20 30 40 50 −40 −20 20 40 N Xβ

free decisions

1 2 3 4 5 6 7 8 9

∆ N ∝ Nβ

β=0.5

2 4 6 8 10 x 10

5

−1 −0.5 0.5 1x 10

5

N xβ

(b)

β=0.5 β=0.6 β=0.7

up-down decision of walker is followed by [N β]+ steps in same direction
k(N) number of random decisions up to step N → k(N) ∼ N 1−β
number of all possible sequences W(N) ∼ 2N1−β → (c, d) = (1,

1 1−β)

note continuum limit of such processes is well defined !

trento jul 23 2012 37

SLIDE 39

Conclusions

Interactions on networks may violate Shannon-Khinchin axiom 4
Keep Shannon-Khinchin axioms 1-3, and S = g (CS in general)
Showed: macroscopic statistical systems can be uniquely classified in

terms of 2 scaling exponents (c, d) – analogy to critical exponents

Single entropy covers all systems: Sc,d = re

i Γ (1 + d , 1 − c ln pi)−rc

All known entropies of SK1-SK3 systems are special cases
Distribution functions of all systems are Lambert-W exponentials. There

are no other options

Phasespace growth uniquely determines entropy
Statistical systems on networks: examples

constant connectivity, k → Boltzmann-Gibbs constant connectancy φ → Tsallis entropy

trento jul 23 2012 38

SLIDE 40

trento jul 23 2012 39

SLIDE 41

trento jul 23 2012 40

SLIDE 42

A note on R´ enyi entropy

It is it not sooo relevant for CS. Why?

Relax Khinchin axiom 4:

S(A+B) = S(A)+S(B|A) → S(A+B) = S(A)+S(B) → R´ enyi entropy

SR =

1 α−1 ln i pα i violates our S = i g(pi)

But: our above argument also holds for R´ enyi-type entropies !!! S = G W

i=1

g(pi)

lim

W →∞

S(λW) S(W) = lim

R→∞

G

fg(z)

z G−1(R)

R

= [for G ≡ ln] = 1

trento jul 23 2012 41

SLIDE 43

The Lambert-W: a reminder

solves x = W(x)eW (x)
inverse of p ln p = [W(p)]−1
delayed differential equations ˙

x(t) = αx(t − τ) → x(t) = e

1 τ W (ατ)t trento jul 23 2012 42

SLIDE 44

Example: a physical system

equation of motion for particle i in system of N overdamped particles µ vi =

j=i
J(

ri − rj) + F( ri) + η( ri, t)

vi ... velocity of i th particle µ ... viscosity of medium F ... external force

J(

r) = G

|

r| λ

ˆ

r ... repulsive particle-particle interaction η ... uncorrelated thermal noise η = 0 and η2 = kT

µ

λ ... characteristic length of short range pairwise interaction

Shown with FP approach and simulation (Curado, Nobre, et al. PRL 2011)

low temperature: Tsallis system (c, d) = (q, 0)
high temperature limit → BG system (c, d) = (1, 1)

trento jul 23 2012 43