Statistics of non-equilibrium systems from the theory of - - PowerPoint PPT Presentation

▶
statistics of non equilibrium systems from the theory of
SMART_READER_LITE
LIVE PREVIEW

Statistics of non-equilibrium systems from the theory of - - PowerPoint PPT Presentation

Nov 19, 2022 228 likes •1.25k views

Statistics of non-equilibrium systems from the theory of sample-space-reducing processes Stefan Thurner www.complex-systems.meduniwien.ac.at www.santafe.edu torino mar 14 2019 with Bernat Corominas-Murtra , Rudolf Hanel BCM, RH, ST, PNAS 112

slide-1
SLIDE 1

Statistics of non-equilibrium systems from the theory of sample-space-reducing processes Stefan Thurner

www.complex-systems.meduniwien.ac.at www.santafe.edu

torino mar 14 2019

slide-2
SLIDE 2

with Bernat Corominas-Murtra, Rudolf Hanel BCM, RH, ST, PNAS 112 (2015) 5348-5353 BCM, RH, ST, New J Physics 18 (2016) 093010 BCM, RH, ST, J Roy Soc Interface 12 (2016) 20150330 ST, BCM, RH, Phys Rev E 96 (2017) 032124 BCM, RH, ST, Sci Rep (2017) 11223 BCM, RH, LZ, ST, Sci Rep 8 (2018) 10837 RH, ST, Entropy (2018)

torino mar 14 2019 1

slide-3
SLIDE 3

motivation I

torino mar 14 2019 2

slide-4
SLIDE 4

power laws are pests

torino mar 14 2019 3

slide-5
SLIDE 5
  • they are everywhere
  • its hard to control them
  • you never get rid of them

torino mar 14 2019 4

slide-6
SLIDE 6

there is more than power laws

torino mar 14 2019 5

slide-7
SLIDE 7

City size

MEJ Newman (2005)

multiplicative

torino mar 14 2019 6

slide-8
SLIDE 8

Rainfall

SOC

torino mar 14 2019 7

slide-9
SLIDE 9

Landslides

SOC

torino mar 14 2019 8

slide-10
SLIDE 10

Hurrican damages

secondary (multiplicative) ???

torino mar 14 2019 9

slide-11
SLIDE 11

Financial interbank loans

multiplicative / preferential

torino mar 14 2019 10

slide-12
SLIDE 12

Forrest fires in various regions

SOC

torino mar 14 2019 11

slide-13
SLIDE 13

Moon crater diameters

MEJ Newman (2005)

fragmentation

torino mar 14 2019 12

slide-14
SLIDE 14

Gamma rays from solar wind

MEJ Newman (2005)

torino mar 14 2019 13

slide-15
SLIDE 15

Movie sales

SOC

torino mar 14 2019 14

slide-16
SLIDE 16

Healthcare costs

multiplicative ???

torino mar 14 2019 15

slide-17
SLIDE 17

Particle physics

???

torino mar 14 2019 16

slide-18
SLIDE 18

Words in books

MEJ Newman (2005)

preferential / random / optimization

torino mar 14 2019 17

slide-19
SLIDE 19

Citations of scientific articles

MEJ Newman (2005)

preferential

torino mar 14 2019 18

slide-20
SLIDE 20

Website hits

MEJ Newman (2005)

preferential

torino mar 14 2019 19

slide-21
SLIDE 21

Book sales

MEJ Newman (2005)

preferential

torino mar 14 2019 20

slide-22
SLIDE 22

Telephone calls

MEJ Newman (2005)

preferential

torino mar 14 2019 21

slide-23
SLIDE 23

Earth quake magnitude

MEJ Newman (2005)

SOC

torino mar 14 2019 22

slide-24
SLIDE 24

Seismic events

SOC

torino mar 14 2019 23

slide-25
SLIDE 25

War intensity

MEJ Newman (2005)

???

torino mar 14 2019 24

slide-26
SLIDE 26

Killings in wars

???

torino mar 14 2019 25

slide-27
SLIDE 27

Size of war

???

torino mar 14 2019 26

slide-28
SLIDE 28

Wealth distribution

MEJ Newman (2005)

multiplicative

torino mar 14 2019 27

slide-29
SLIDE 29

Family names

MEJ Newman (2005)

multiplicative, ???

torino mar 14 2019 28

slide-30
SLIDE 30

More power laws ...

  • networks: literally thousands of scale-free networks
  • allometric scaling in biology
  • dynamics in cities
  • fragmentation processes
  • random walks
  • crackling noise
  • growth with random times of observation
  • blackouts
  • fossil record
  • bird sightings
  • terrorist attacks
  • fluvial discharge, contact processes
  • anomalous diffusion ...

torino mar 14 2019 29

slide-31
SLIDE 31

where do power laws come from?

torino mar 14 2019 30

slide-32
SLIDE 32

classic mechanisms to get power laws

  • criticality (at phase transitions)
  • self-organised criticality
  • multiplicative processes with constraints
  • preferential processes
  • generalized entropies for non-multinomial systems

torino mar 14 2019 31

slide-33
SLIDE 33

Motivation II

torino mar 14 2019 32

slide-34
SLIDE 34

complex systems are driven systems

torino mar 14 2019 33

slide-35
SLIDE 35

driven system = driving + relaxing process

torino mar 14 2019 34

slide-36
SLIDE 36

would be nice to have: stat theory for driven, non-equilibrium systems that explains the abundance of power laws and variations

torino mar 14 2019 35

slide-37
SLIDE 37

Many processes are history- or path-dependent

  • future events depend on history of past events
  • often: past events constrain possibilities for future

→ sample-space of processes reduces as they unfold

torino mar 14 2019 36

slide-38
SLIDE 38

Sample-Space Reducing Processes (SSR)

torino mar 14 2019 37

slide-39
SLIDE 39

Example: History-dependent SSR processes

9 13 7 5

1) 4) 2) 3) 5) 6)

3 1

torino mar 14 2019 38

slide-40
SLIDE 40

Phase spaces are nested

Ω1 ⊂ Ω2 ⊂ Ω3 · · · ⊂ ΩN

torino mar 14 2019 39

slide-41
SLIDE 41

Restart means

Ω1 = ΩN

torino mar 14 2019 40

slide-42
SLIDE 42

Sentence-formation is SSR

funny

wolf

word

runs funny howls bites house can rucksack room

light moon

night green eats

door

  • pen

blue

sun

wind

cloud buy

high

short grey

jacket computer algebra write strong letter table fill information article punch hold line brown trash tree deer rabbit

space fly mosquito bee window think red building hospital glass wine take sell plane envelope

word

runs funny howls bites house can rucksack room

light moon

night green eats

door

  • pen

blue

sun

wind

cloud buy

high

short grey

jacket computer algebra write strong letter table fill information article punch hold line brown trash tree deer rabbit

space fly mosquito bee window think red building hospital glass wine take sell plane envelope

wolf wolf

word

house can rucksack room

light moon

night green

door

  • pen

blue

sun

wind

cloud buy

high

short

jacket computer algebra write letter table fill information article punch hold line brown trash tree deer rabbit

space fly mosquito bee window think red building hospital glass wine take sell plane envelope

runs howls bites eats grey

strong

wolf

word

runs funny howls bites house can rucksack room

light

green eats

door

  • pen

blue

sun

wind

cloud buy

high

short grey

jacket computer algebra write strong letter table fill information article punch hold line brown trash tree deer rabbit

space fly mosquito bee window think red building hospital glass wine take sell plane envelope

moon

night

a) b) c) d)

torino mar 14 2019 41

slide-43
SLIDE 43

5 10 0.45 n 5 10 0.1 0.2 0.3 0.4 n p(n) x x

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9

p(i)

1 5 10 1 5 10

10

0.1 0.2 0.3 0.4

Site i Site i

torino mar 14 2019 42

slide-44
SLIDE 44

SSR lead to exact Zipf’s law!

p(i) = 1 i

p(i) is probability to visit site i

torino mar 14 2019 43

slide-45
SLIDE 45

Proof by induction

Let N = 2. There are two sequences φ: either φ directly generates a 1 with p = 1/2, or first generates 2 with p = 1/2, and then a 1 with certainty. Both sequences visit 1 but only

  • ne visits 2. As a consequence, P2(2) = 1/2 and P2(1) = 1.

Now suppose PN−1(i) = 1/i holds. Process starts with dice N, and probability to hit i in the first step is 1/N. Also, any other j, N ≥ j > i, is reached with probability 1/N. If we get j > i, we get i in the next step with probability Pj−1(i), which leads to a recursive scheme for i < N, PN(i) =

1 N

  • 1 +

i<j≤N Pj−1(i)

  • . Since by assumption Pj−1(i) =

1/i, with i < j ≤ N holds, some algebra yields PN(i) = 1/i.

torino mar 14 2019 44

slide-46
SLIDE 46

Proof

pi = N

j=1 p(i|j) pj

with p(i|j) =

1 j−1, for i < j

pi = p1 N +

N

  • j=i+1

pj j − 1 = p1 N + pi+1 i +

N

  • j=i+2

pj j − 1 pi+1 = p1 N +

N

  • j=i+2

pj j − 1 → pi+1 − pi = −pi+1

i , or pi+1−pi pi+1

= −1

i

Ansatz: pi = 1

i ✓

torino mar 14 2019 45

slide-47
SLIDE 47

true for all systems with shrinking sample space over time

torino mar 14 2019 46

slide-48
SLIDE 48

Note: that is a process that is slowly driven

torino mar 14 2019 47

slide-49
SLIDE 49

What if restart SSR before it is fully relaxed?

φ ... Sample Space Reducing process (SSR) φR ... random walk mix both processes Φ(λ) = λφ + (1 − λ)φR , λ ∈ [0, 1] Restarting or driving rate (1 − λ) →

p(i) = i−λ

torino mar 14 2019 48

slide-50
SLIDE 50

The role of driving – result is exact too

Clearly pi = N

j=1 p(i|j) pj holds, with

p(i|j) =     

λ j−1 + 1−λ N

for i < j (SSR)

1−λ N

for i ≥ j > 1 (RW)

1 N

for i ≥ j = 1 (restart) We get pi = 1−λ

N + 1 Np1 + N j=i+1 λ j−1 pj

→ recursive relation pi+1 − pi = −λ

i pi+1 pi p1

= i−1

j=1

  • 1 + λ

j

−1 = exp

  • − i−1

j=1 log

  • 1 + λ

j

  • ∼

exp

  • − i−1

j=1 λ j

  • ∼ exp (−λ log(i)) = i−λ

torino mar 14 2019 49

slide-51
SLIDE 51

true for all systems with driving rate 1 − λ (shrinking sample space with probability λ)

torino mar 14 2019 50

slide-52
SLIDE 52

History-dependent processes with fast driving

same convergence speed as CLT for iid processes

torino mar 14 2019 51

slide-53
SLIDE 53

SSR-based Zipf law is extremely robust

torino mar 14 2019 52

slide-54
SLIDE 54

prior probabilities are practically irrelevant!

torino mar 14 2019 53

slide-55
SLIDE 55

what is a driven process?

torino mar 14 2019 54

slide-56
SLIDE 56

driven system = driving + relaxation part

torino mar 14 2019 55

slide-57
SLIDE 57

relaxation = sample space reducing process

torino mar 14 2019 56

slide-58
SLIDE 58

details of relaxing process do NOT matter

torino mar 14 2019 57

slide-59
SLIDE 59

Zipf-law is extremely robust—accelerated SSR

torino mar 14 2019 58

slide-60
SLIDE 60

not all transitions p(i|j) must exist—diffusion on networks

torino mar 14 2019 59

slide-61
SLIDE 61

SSR and diffusion on networks

torino mar 14 2019 60

slide-62
SLIDE 62

SSR is a random walk on directed ordered NW

1 2 3 4 5

1 2 3 4 5 1/2 1/4 1/8

Node rank Node occupation probability

10 10

2

10 10

−2

10

−4

Path rank Path probability acyclic pexit=0.3

a) b)

  • 0.65
  • 1

pexit=0.3

Start Stop

1/5 1/4 1/3 1/2 (1-pexit)/5 1

pexit pexit=1⇒ φ pexit=0⇒ φ∞

note fully connected

torino mar 14 2019 61

slide-63
SLIDE 63

SSR = targeted random walk on networks

simple example Directed Acyclic Graph (no cycles)

torino mar 14 2019 62

slide-64
SLIDE 64

torino mar 14 2019 63

slide-65
SLIDE 65

torino mar 14 2019 64

slide-66
SLIDE 66

torino mar 14 2019 65

slide-67
SLIDE 67

Simple routing algorithm

  • take directed acyclic network fix it
  • pick start-node
  • perform a random walk from start-node to end-node (1)
  • repeat many times from other start-nodes
  • prediction visiting frequency of nodes follows Zipf law

torino mar 14 2019 66

slide-68
SLIDE 68

All diffusion processes on DAG are SSR

sample ER graph → direct it → pick start and end → diffuse

torino mar 14 2019 67

slide-69
SLIDE 69

Exponential NW HEP Co-authors

torino mar 14 2019 68

slide-70
SLIDE 70

prior probabilities are practically irrelevant!

torino mar 14 2019 69

slide-71
SLIDE 71

What happens if introduce weights on links?

ER Graph poisson weights power weights

torino mar 14 2019 70

slide-72
SLIDE 72

prior probabilities are practically irrelevant!

torino mar 14 2019 71

slide-73
SLIDE 73

What happens if we introduce cycles?

ER → direct it → change link to random direction with 1 − λ

“driving” λ = 0.8 λ = 0.5

torino mar 14 2019 72

slide-74
SLIDE 74

Zipf’s law is an immense attractor!

torino mar 14 2019 73

slide-75
SLIDE 75

Zipf’s law is an attractor

  • no matter what the network topology is → Zipf
  • no matter what the link weights are → Zipf
  • if have cycles → exponent is less than one

torino mar 14 2019 74

slide-76
SLIDE 76

all good search is SSR

torino mar 14 2019 75

slide-77
SLIDE 77

What is good search?

a search process is good if ...

  • ... at every step you eliminate more possibilities than you

actually sample

  • ... every step you take eliminates branches of possibilities

if eliminate fast enough → power law in visiting times if eliminate too little → sample entire space (exhaustive search) if no cycles: expect Zipf’s law

torino mar 14 2019 76

slide-78
SLIDE 78

clicking on web page is often result of search process

torino mar 14 2019 77

slide-79
SLIDE 79

adamic & hubermann 2002

torino mar 14 2019 78

slide-80
SLIDE 80

breslau et al 99

torino mar 14 2019 79

slide-81
SLIDE 81

what about exponents > 1?

torino mar 14 2019 80

slide-82
SLIDE 82

1 2 3 4 5 6 7 8 9 1011121314151617181920

Multiplication factor µ

→ p(i) = i−µ

torino mar 14 2019 81

slide-83
SLIDE 83

10 10

1

10

2

10

3

10

4

10

−10

10

−8

10

−6

10

−4

10

−2

10

State Relative Frequency

µ=1 µ=1.5 µ=2 µ=2.5

10 10

2

10

4

10

−6

10

−4

10

−2

10

Energy Relative Frequency

µ=2.5 µ=3.5 µ=4.5

torino mar 14 2019 82

slide-84
SLIDE 84

what if we introduce conservation laws?

torino mar 14 2019 83

slide-85
SLIDE 85

Conservation laws in SSR processes

assume you have duplication at every jump, µ = 2 if you are at i → duplicate → one jumps to j, the other to k conservation means: i = j + k. conservation means: f(i) = f(state1) + f(state2) + · · · + f(stateµ)

→ p(i) = i−2 for all µ

same result was found by E. Fermi for particle cascades

torino mar 14 2019 84

slide-86
SLIDE 86

10 10

1

10

2

10

3

10

4

10

−7

10

−6

10

−5

10

−4

10

−3

10

−2

10

−1

Energy (arbitrary units) Relative frequency

µ=2.5 µ=3.5 µ=4.5

−2

torino mar 14 2019 85

slide-87
SLIDE 87

complex systems are driven – always!

torino mar 14 2019 86

slide-88
SLIDE 88

Complex systems are driven non-equilibrium systems

  • only driven systems produce non-trivial structures
  • without driving: just ground state or equilibrium
  • every driven system: relaxing part + driving part
  • every relaxing part is a SSR

torino mar 14 2019 87

slide-89
SLIDE 89

torino mar 14 2019 88

slide-90
SLIDE 90

where do all the distributions come from?

torino mar 14 2019 89

slide-91
SLIDE 91

Assume that driving rate depends on state λ(i)

torino mar 14 2019 90

slide-92
SLIDE 92

→ λ(x) = −x d

dx log p(x)

that can be proved

torino mar 14 2019 91

slide-93
SLIDE 93

Proof

transition probabilities from state k to i are pSSR(i|k) = λ(k)

qi g(k−1) + (1 − λ(k))qi

if i < k (1 − λ(k))qi if i > k g(k) is the cdf of qi, g(k) =

i≤k qi. Observing that

pλ,q(i + 1) qi+1

  • 1 + λ(i + 1)qi+1

g(i)

  • = pλ,q(i)

qi we get pλ,q(i) = qi Zλ,q

  • 1<j≤i
  • 1 + λ(j)

qj g(j − 1) −1 ∼ q(i) Zλ,q e

−

j≤i λ(j) q(j) g(j−1)

Zλ,q is the normalisation constant. For uniform priors, taking logs and going to continuous variables gives the result, λ(x) = −x d

dx log pλ(x).

torino mar 14 2019 92

slide-94
SLIDE 94

the driving process determines distribution

torino mar 14 2019 93

slide-95
SLIDE 95

Special cases λ(x) = −x d

dx log p(x)

  • Zipf: slow driving (λ = 1)

→ p(x) = x−1

  • Power-law: constant driving λ(x) = α

→ p(x) = x−α

  • Exponential: λ(x) = βx

→ p(x) = e−β(x−1)

  • Power-law + cut-off: λ(x) = α+βx

→ p(x) = x−αe−βx

  • Gamma: λ(x) = 1 − α + βx

→ p(x) = xα−1e−βx

torino mar 14 2019 94

slide-96
SLIDE 96

Special cases λ(x) = −x d

dx log p(x)

  • Normal: λ(x) = 2βx2

→ p(x) = e−β

2(x−1)2

  • Stretched exp: λ(x) = αβ|x|α

→ p(x) = e−β

α(x−1)α

  • Log-normal: λ(x) = 1 − β

σ2 + log x σ2

→

1 xe−(log x−β)2

2σ2

  • Gompertz: λ(x) = (βeαx − 1)βx

→ p(x) = eβx−αeβx

  • Weibull: λ(x) = β−ααxα + α − 1 →p(x) =
  • x

β

α−1 e

−

  • x

β

α

  • Tsallis: λ(x) =

βx 1−βx(1−Q)

→ p(x) = (1−(1−Q)βx)

1 1−Q

torino mar 14 2019 95

slide-97
SLIDE 97

Driving determines statistics of driven systems

slow → Zipf’s law constant → power law extreme driving → prior distribution (uniform) driving state dependent → any distribution

torino mar 14 2019 96

slide-98
SLIDE 98

Examples that are of SSR–nature

  • self-organized critical systems
  • driven systems (with stationary distributions)
  • search
  • fragmentation
  • propagation of information in laguage: sentence formation
  • sequences of human behavior
  • games: go, chess, life ...
  • record statistics

torino mar 14 2019 97

slide-99
SLIDE 99
  • do we now have a statistical theory for statistics of driven,

dissipative, (stationary) non-equilibrium systems?

  • are SOC processes a sub-set of SSR processes?
  • is there a maximum configuration principle for arbitrarily

driven systems?

  • can we do thermodynamics with these systems?

torino mar 14 2019 98

slide-100
SLIDE 100

Conclusions

  • most complex systems are driven and show power laws
  • path-dependent processes of SSR-type abound in nature
  • SSR has extremely robust attractors – priors don’t matter
  • relaxation is usually SSR
  • details of driving + SSR → explain statistics

torino mar 14 2019 99

slide-101
SLIDE 101

the following are subsets of SSR

  • criticality ???
  • self-organised criticality ✓
  • multiplicative processes with constraints ✓
  • preferential processes ???
  • generalized entropies for non-multinomial systems ✓

torino mar 14 2019 100