SLIDE 1 Learning the dynamics
Thierry Mora
Laboratoire de physique statistique École normale supérieure, Paris & CNRS
ENS Paris A. Walczak Università Sapienza Rome
- A. Cavagna
- I. Giardina
- O. Pohl
- E. Silvestri
M. Viale Princeton University
Aberdeen University
Vision
Institute
IST Austria G. Tkacik
SLIDE 2
statistical mechanics as a tool to describe correlated systems
(any) interacting agents collective behaviour interacting spins spontaneous magnetization
SLIDE 3
statistical mechanics as a tool to describe correlated systems
(any) interacting agents collective behaviour interacting spins spontaneous magnetization
A A R R R N N D D C E E Q G
SLIDE 4
statistical mechanics as a tool to describe correlated systems
(any) interacting agents collective behaviour interacting spins spontaneous magnetization
SLIDE 5
statistical mechanics as a tool to describe correlated systems
(any) interacting agents collective behaviour interacting spins spontaneous magnetization
SLIDE 6 H = − X
ij
Jijsisj
two modeling approaches
model bottom-up phenomena
solve
C(r), φ
SLIDE 7 H = − X
ij
Jijsisj
two modeling approaches
model bottom-up phenomena
solve
C(r), φ
model top-down
solve
H = − X
ij
Jijsisj
C(r), φ
SLIDE 8 H = − X
ij
Jijsisj
two modeling approaches
model bottom-up phenomena
solve
C(r), φ
model top-down
solve
H = − X
ij
Jijsisj
C(r), φ
inverse problem
SLIDE 9
σ = (σ1, σ2, . . . , σN)
how to fit models to data: the maximum entropy approach
SLIDE 10
N agents / units described by a variable σ
σ = (σ1, σ2, . . . , σN)
how to fit models to data: the maximum entropy approach
SLIDE 11
N agents / units described by a variable σ Maximize the entropy
σ = (σ1, σ2, . . . , σN)
how to fit models to data: the maximum entropy approach
SLIDE 12
N agents / units described by a variable σ Maximize the entropy under the constraint that observables
have the same average as the data
σ = (σ1, σ2, . . . , σN)
O1, O2, . . .
Oa⇥model = Oa⇥data
Oa⇥
hσii, hσiσji is typically a moment, e.g.
how to fit models to data: the maximum entropy approach
SLIDE 13 N agents / units described by a variable σ Maximize the entropy under the constraint that observables
have the same average as the data
σ = (σ1, σ2, . . . , σN)
O1, O2, . . .
Oa⇥model = Oa⇥data
Oa⇥
hσii, hσiσji is typically a moment, e.g.
how to fit models to data: the maximum entropy approach
P(σ) = 1 Z exp "X
a
JaOa(σ) #
−E/kBT
⇢
P(σ) = 1 Z e
P
i hiσi+P ij σiσj
e.g.
SLIDE 14 N agents / units described by a variable σ Maximize the entropy under the constraint that observables
have the same average as the data
σ = (σ1, σ2, . . . , σN)
O1, O2, . . .
Oa⇥model = Oa⇥data
Oa⇥
hσii, hσiσji is typically a moment, e.g.
how to fit models to data: the maximum entropy approach
P(σ) = 1 Z exp "X
a
JaOa(σ) #
−E/kBT
⇢
P(σ) = 1 Z e
P
i hiσi+P ij σiσj
e.g.
(disordered) Ising model!
SLIDE 15 Given the functional form
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
SLIDE 16 Given the functional form
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
SLIDE 17 Given the functional form what parameters Ja explain the data best?
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
SLIDE 18 Given the functional form what parameters Ja explain the data best? Bayes rule
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
SLIDE 19 Given the functional form what parameters Ja explain the data best? Bayes rule
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
P({Ja}|data) = P(data|{Ja})P({Ja}) P(data)
SLIDE 20 Given the functional form what parameters Ja explain the data best? Bayes rule
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
P({Ja}|data) = P(data|{Ja})P({Ja}) P(data)
∝ " M Y
m=1
P(σm|{Ja}) # P({Ja})
SLIDE 21 Given the functional form what parameters Ja explain the data best? Bayes rule
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
P({Ja}|data) = P(data|{Ja})P({Ja}) P(data)
∝ " M Y
m=1
P(σm|{Ja}) # P({Ja})
SLIDE 22 Given the functional form what parameters Ja explain the data best? Bayes rule
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
P({Ja}|data) = P(data|{Ja})P({Ja}) P(data)
∝ " M Y
m=1
P(σm|{Ja}) # P({Ja})
flat prior
SLIDE 23 Given the functional form what parameters Ja explain the data best? Bayes rule
maximum likelihood formulation
P(σ) = 1 Z exp "X
a
JaOa(σ) #
P({Ja}|data) = P(data|{Ja})P({Ja}) P(data)
∝ " M Y
m=1
P(σm|{Ja}) # P({Ja})
flat prior
∂ log P ∂Ja = 0 ⇒ −M ∂ log Z ∂Ja + X
a M
X
m=1
Oa(σm) = 0
⇥Oa(σ)⇤model = ⇥Oa(σ)⇤data
satisfies the constaints (maximum likelihood)
SLIDE 24 maximum entropy in biology
collective activity of neural populations
Schneidman et al. Nature 2006 Shlens et al J. Neuroscience 2006 Tang et al J. Neuroscience 2008 Tkacik et al PLoS CP 2014
SLIDE 25 maximum entropy in biology
collective activity of neural populations
Schneidman et al. Nature 2006 Shlens et al J. Neuroscience 2006 Tang et al J. Neuroscience 2008 Tkacik et al PLoS CP 2014
co-variations in protein families ⇨ contact prediction
Weigt et al. PNAS 2009; Morcos et al. PNAS 2011 Marks et al PLoS ONE 2012; Sulkowska et al PNAS 2012
SLIDE 26 maximum entropy in biology
collective activity of neural populations
Schneidman et al. Nature 2006 Shlens et al J. Neuroscience 2006 Tang et al J. Neuroscience 2008 Tkacik et al PLoS CP 2014
co-variations in protein families ⇨ contact prediction
Weigt et al. PNAS 2009; Morcos et al. PNAS 2011 Marks et al PLoS ONE 2012; Sulkowska et al PNAS 2012
diversity of antibody repertoires in the immune system
Mora Walczak Callan Bialek PNAS 2010
SLIDE 27 maximum entropy in biology
collective activity of neural populations
Schneidman et al. Nature 2006 Shlens et al J. Neuroscience 2006 Tang et al J. Neuroscience 2008 Tkacik et al PLoS CP 2014
co-variations in protein families ⇨ contact prediction
Weigt et al. PNAS 2009; Morcos et al. PNAS 2011 Marks et al PLoS ONE 2012; Sulkowska et al PNAS 2012
diversity of antibody repertoires in the immune system
Mora Walczak Callan Bialek PNAS 2010
DNA motifs of transcription factor binding sites
Santolini Mora Hakim Plos ONE 2014
SLIDE 28 maximum entropy in biology
collective activity of neural populations
Schneidman et al. Nature 2006 Shlens et al J. Neuroscience 2006 Tang et al J. Neuroscience 2008 Tkacik et al PLoS CP 2014
co-variations in protein families ⇨ contact prediction
Weigt et al. PNAS 2009; Morcos et al. PNAS 2011 Marks et al PLoS ONE 2012; Sulkowska et al PNAS 2012
diversity of antibody repertoires in the immune system
Mora Walczak Callan Bialek PNAS 2010
DNA motifs of transcription factor binding sites
Santolini Mora Hakim Plos ONE 2014
collective behaviour of mice
Shemesh et al. eLife 2013
SLIDE 29 maximum entropy in biology
collective activity of neural populations
Schneidman et al. Nature 2006 Shlens et al J. Neuroscience 2006 Tang et al J. Neuroscience 2008 Tkacik et al PLoS CP 2014
co-variations in protein families ⇨ contact prediction
Weigt et al. PNAS 2009; Morcos et al. PNAS 2011 Marks et al PLoS ONE 2012; Sulkowska et al PNAS 2012
diversity of antibody repertoires in the immune system
Mora Walczak Callan Bialek PNAS 2010
DNA motifs of transcription factor binding sites
Santolini Mora Hakim Plos ONE 2014
collective behaviour of mice
Shemesh et al. eLife 2013
collective behaviour of bird flocks
Bialek et al. PNAS 2012; Cavagna et al. PRE 2014 Bialek et al. PNAS 2014
SLIDE 30
dynamical maximum entropy
maximum entropy gives a “steady-state” picture. what about the dynamics? ad hoc dynamics such as Glauber, Metropolis may be wrong
SLIDE 31 dynamical maximum entropy
maximum entropy gives a “steady-state” picture. what about the dynamics? (a) solution: maximum entropy over trajectories t t+1 t+2
P(σ1, . . . , σT )
hσt
iσt0 j i
constraints on cross-time correlations, e.g. ad hoc dynamics such as Glauber, Metropolis may be wrong
SLIDE 32 dynamical maximum entropy
maximum entropy gives a “steady-state” picture. what about the dynamics? (a) solution: maximum entropy over trajectories t t+1 t+2
P(σ1, . . . , σT )
hσt
iσt0 j i
constraints on cross-time correlations, e.g.
P(σ1, . . . , σT ) = 1 Z exp @ X
i,j,t,t0
Jt−t0
ij
σt
iσt0 j
1 A
⇨
⇢
−A “action”
ad hoc dynamics such as Glauber, Metropolis may be wrong
SLIDE 33 dynamical maximum entropy
maximum entropy gives a “steady-state” picture. what about the dynamics? (a) solution: maximum entropy over trajectories t t+1 t+2
P(σ1, . . . , σT )
hσt
iσt0 j i
constraints on cross-time correlations, e.g.
P(σ1, . . . , σT ) = 1 Z exp @ X
i,j,t,t0
Jt−t0
ij
σt
iσt0 j
1 A
⇨
⇢
−A “action”
ad hoc dynamics such as Glauber, Metropolis may be wrong not the same as:
P(σi,t|{σj,t0}t0<t) = 1 Z({σj,t0}t0<t) exp 2 4hiσi,t + X
j,t0<t
Jt−t0
ij
σi,tσj,t0 3 5
SLIDE 34
example 1: flocks of birds
SLIDE 35
example 1: flocks of birds
SLIDE 36 aligned collective motion
fluctuations around main
strong polarization domains
φ = 1 N r v
i
r v
i i
∑
~ 0.95
ξ
SLIDE 37 a maximum entropy model for birds
velocity of bird
vi/k vik
SLIDE 38 P( s1, . . . , sN) = 1 Z exp @X
ij
Jij si sj 1 A = 1 Z exp(−H)
Cij = h si sji
a maximum entropy model for birds
velocity of bird constrain correlation functions (Heisenberg model on lattice)
vi/k vik
SLIDE 39 P( s1, . . . , sN) = 1 Z exp @X
ij
Jij si sj 1 A = 1 Z exp(−H)
Cij = h si sji
a maximum entropy model for birds
velocity of bird constrain correlation functions (Heisenberg model on lattice)
vi/k vik
SLIDE 40 d⇤ si dt = −⇥H ⇥⇤ si + ⇤ i(t) =
N
X
j=1
Jij⇤ sj + ⇤ i(t) P( s1, . . . , sN) = 1 Z exp @X
ij
Jij si sj 1 A = 1 Z exp(−H)
Cij = h si sji
a maximum entropy model for birds
velocity of bird constrain correlation functions (Heisenberg model on lattice) derives from Langevin eqn, equilavent to “social” model, similar to Vicsek’s
vi/k vik
(does not mean that’s the only possible dynamics, or the true one) alignment noise
SLIDE 41 Jij = ⇢ J
if j is one i’s nc first neighbors
parametrization
then symmetrized
SLIDE 42 Jij = ⇢ J
if j is one i’s nc first neighbors
Equivalent to maximum entropy with constraint on
Cint = 1 N
N
X
i=1
1 nc X
j∈V (i)
h si sji
parametrization
single snapshot — spatial averaging instead of ensemble averaging then symmetrized
SLIDE 43 predicting correlation functions
long-range order from local interactions
perpendicular correlation
interaction range correlation
range
SLIDE 44 10 20 30 40
Distance r2 (m)
0.002 0.004
Correlation C4(r1,r2)
r1=0.5 Data Model
i j l k
r1 r2 r1
B
predicting correlation functions
long-range order from local interactions
perpendicular correlation
interaction range correlation
range
4-bird correlation function
SLIDE 45
interaction range
metric or topological ?
SLIDE 46 interaction range
metric or topological ?
rc r1
SLIDE 47 interaction range
metric or topological ?
rc r1 rc r1
SLIDE 48 interaction range
metric or topological ?
rc r1 nc = 6 rc r1
SLIDE 49 interaction range
metric or topological ?
rc r1 nc = 6 nc = 6 rc r1
SLIDE 50 interaction range
metric or topological ?
r1 nc -1/3 rc r1 nc = 6 nc = 6 rc r1
SLIDE 51 r1 nc
interaction range
metric or topological ?
r1 nc -1/3 nc ~ (rc / r1)3 rc r1 nc = 6 nc = 6 rc r1
SLIDE 52 0.6 0.8 1 1.2 1.4 1.6 1.8
sparseness r1(m)
0.5 1
Interaction range nc
E
answer:
interaction is topological not metric
Bialek et al PNAS 2012
SLIDE 53 20 40 60 80
Flock size (m)
20 40 60
Interaction range nc
D
0.6 0.8 1 1.2 1.4 1.6 1.8
sparseness r1(m)
0.5 1
Interaction range nc
E
answer:
nc ~ 21
interaction is topological not metric
does not depend on flock density flock size
Bialek et al PNAS 2012
SLIDE 54
we’ve assumed that neighborhoods are fixed but birds may exchange neighbors fast
dynamics (may) matter
SLIDE 55
we’ve assumed that neighborhoods are fixed but birds may exchange neighbors fast
dynamics (may) matter
SLIDE 56
we’ve assumed that neighborhoods are fixed but birds may exchange neighbors fast the effective number of interaction partners
could be larger than the instantaneous one.
dynamics (may) matter
SLIDE 57 constrain and “action”
A = −1 2 X
t
X
i6=j
⇣ J(1)
ij;tst ist j + J(2) ij;tst+1 i
st
j
⌘
P(s1, . . . , sT ) = 1 ˆ Z exp (−A)
hst
ist ji
hst
ist+1 j
i
dynamics (on bird orientations)
SLIDE 58 constrain and “action”
A = −1 2 X
t
X
i6=j
⇣ J(1)
ij;tst ist j + J(2) ij;tst+1 i
st
j
⌘ in spin-wave approximation, equivalent to “collective random walk”
P(s1, . . . , sT ) = 1 ˆ Z exp (−A)
hst
ist ji
hst
ist+1 j
i
dynamics (on bird orientations)
~ ⇡
A and M functions of J(1) and J(2)
SLIDE 59 constrain and “action”
A = −1 2 X
t
X
i6=j
⇣ J(1)
ij;tst ist j + J(2) ij;tst+1 i
st
j
⌘ in spin-wave approximation, equivalent to “collective random walk”
P(s1, . . . , sT ) = 1 ˆ Z exp (−A)
hst
ist ji
hst
ist+1 j
i
dynamics (on bird orientations)
~ ⇡
alignment strength temperature
Langevin equation
SLIDE 60 infering J, nc, and a third parameter, the “temperature” T and similar eq. for T
inferring out-of-equilibrium behavior
J nc = 1 δt Cint − Cs + Gs − Gint 2Cint − C0
int − Cs
SLIDE 61 infering J, nc, and a third parameter, the “temperature” T and similar eq. for T
P( s1, . . . , sN) = 1 Z exp @ J T X
ij
nij si sj 1 A
inferring out-of-equilibrium behavior
J nc = 1 δt Cint − Cs + Gs − Gint 2Cint − C0
int − Cs
if equilibrium – slowly evolving and symmetric nij – then
- ne recovers the same result as the Heisenberg model, with J ← J / T
SLIDE 62 0.2 0.4 0.6 0.8 1 5 10 15 20
dynamic static
µ n∗
c
10 3.6 3.8 20
n∗
c
simulation of 2D topological model with Voronoi neighbors µ is a parameter quantifying how fast birds change neighbors
test on simulated data
at large µ,
dynamical maximum entropy works, static maximum entropy doesn’t
dynamical inference
static inference
true value static inference
number of partners
Cavagna et al PRE 2014
SLIDE 63
the retina
multielectrode array recordings
SLIDE 64
the stimulus
SLIDE 65
the stimulus
SLIDE 66 σi = 0, 1
binary neurons
raster → binary variables N ~ 150 neurons
10 ms
Marre et al.
SLIDE 67 neuron activities are correlated
independenc
total number of spikes
Schneidman et al, Nature 2005
Ising model P2(σ) = 1
Z e
P
i hiσi+P ij Jijσiσj
P1(σ) = 1 Z e
P
i hiσi
SLIDE 68 neuron activities are correlated
independenc
total number of spikes
Schneidman et al, Nature 2005
goal: build the thermodynamics of this correlated system from data Ising model P2(σ) = 1
Z e
P
i hiσi+P ij Jijσiσj
P1(σ) = 1 Z e
P
i hiσi
SLIDE 69
evaluate by modelling or by frequency counting
P(σ1, . . . , σN)
building the density of states
SLIDE 70
evaluate by modelling or by frequency counting
P(σ1, . . . , σN)
building the density of states
P = 1 Z e−E/kBT
define “energy” through Boltzmann law
E = − log P
SLIDE 71
evaluate by modelling or by frequency counting
P(σ1, . . . , σN)
building the density of states
P = 1 Z e−E/kBT
define “energy” through Boltzmann law now consider the distribution of energies E
E = − log P C(E) = number of states with E(σ) < E
SLIDE 72 evaluate by modelling or by frequency counting
P(σ1, . . . , σN)
building the density of states
P = 1 Z e−E/kBT
define “energy” through Boltzmann law now consider the distribution of energies E
E = − log P C(E) = number of states with E(σ) < E
define a microcanonical entropy :
S(E) = log C(E)
Mora Bialek J. Stat. Phys. 2011
SLIDE 73 density of states
Tkacik Mora Marre Amodei Berry Bialek, arxiv 2014
just counting states Maximum entropy model (under natural movie stimulus)
SLIDE 74
Zipf’s law (interlude)
Zipf 1949
SLIDE 75
Zipf’s law (interlude)
Zipf 1949
Probability P (E in log scale) ~ cumulative distribution
SLIDE 76
what does S(E) = E mean?
SLIDE 77
what does S(E) = E mean?
what’s the probability of a given energy?
P(E) ≈ eS−E
how many states at E probability of a state at E constant!
SLIDE 78
what does S(E) = E mean?
what’s the probability of a given energy?
P(E) ≈ eS−E
how many states at E probability of a state at E constant! (note: only works
if exponent is = 1)
SLIDE 79
what does S(E) = E mean?
E scales with N its fluctuations scale with N heat capacity
what’s the probability of a given energy?
P(E) ≈ eS−E
how many states at E probability of a state at E constant!
in usual thermodynamics…
(note: only works
if exponent is = 1)
C = Var(E) ∼ N
C / N diverges at 2nd order transition critical point (e.g. 2D Ising model)
SLIDE 80
what does S(E) = E mean?
E scales with N its fluctuations scale with N heat capacity
what’s the probability of a given energy?
P(E) ≈ eS−E
how many states at E probability of a state at E constant!
in usual thermodynamics… link to information theory
E = − log P
“surprise” (Shannon 1948) equipartition theorem (valid for independent units):
almost all codewords we see have the same surprise ~ entropy basis for compression (note: only works
if exponent is = 1)
C = Var(E) ∼ N
C / N diverges at 2nd order transition critical point (e.g. 2D Ising model)
SLIDE 81
specific heat
let’s add a spurious temperature — one direction in parameter space (T = 1 corresponds to the real ensemble)
PT (σ) = 1 Z(T)e−E/T
C = VarT (E/T) = VarT (− log P)
SLIDE 82
specific heat
let’s add a spurious temperature — one direction in parameter space (T = 1 corresponds to the real ensemble)
PT (σ) = 1 Z(T)e−E/T
C = VarT (E/T) = VarT (− log P)
SLIDE 83 specific heat
Tkacik Mora Marre Amodei Berry Bialek, arxiv 2014
Neural network
let’s add a spurious temperature — one direction in parameter space (T = 1 corresponds to the real ensemble)
PT (σ) = 1 Z(T)e−E/T
C = VarT (E/T) = VarT (− log P)
SLIDE 84
dynamical criticality
SLIDE 85 dynamical approach
10 ms
σt
SLIDE 86 dynamical approach
proposal: consider statistics over trajectories t t+1 t+2
10 ms
σt
P(σ1, . . . , σL)
SLIDE 87 define
dynamical approach
proposal: consider statistics over trajectories t t+1 t+2
10 ms
σt
P(σ1, . . . , σL)
E = − log P({σi,t})
calculate specific heat c = 1 NLVar(E)
SLIDE 88 link to dynamical criticality:
branching process
pij P({σi,t}) = Y
t N
Y
i=1
pi(t)σi,t[1 − pi(t)]1−σi,t
pi(t) = 1 − Y
j
(1 − pij)σi,t−1 ω = 1 N X
ij
pij
branching parameter:
t
Beggs & Plenz 2003
SLIDE 89 link to dynamical criticality:
branching process
pij P({σi,t}) = Y
t N
Y
i=1
pi(t)σi,t[1 − pi(t)]1−σi,t
pi(t) = 1 − Y
j
(1 − pij)σi,t−1 ω = 1 N X
ij
pij
branching parameter:
t
Shew Yang Petermann Roy Plenz 2009 Beggs & Plenz 2003
SLIDE 90 model for multi-neuron spike trains
let’s do something simple
independent neutrons
total number of spikes Kt = X
i
σi,t is informative of collective behaviour sampling 2N states is hard enough; here 2NL states — we need models
Tkacik Marre Mora Amodei Berry Bialek, JSTAT 2013
SLIDE 91 maximum entropy model with
constrains on temporal correlations of K
model for multi-neuron spike trains
let’s do something simple
independent neutrons
total number of spikes Kt = X
i
σi,t is informative of collective behaviour
P(Kt, Kt0)
|t − t0| < v
⬄ all neurons behave the same sampling 2N states is hard enough; here 2NL states — we need models
Tkacik Marre Mora Amodei Berry Bialek, JSTAT 2013
SLIDE 92 maximum entropy model with
constrains on temporal correlations of K
model for multi-neuron spike trains
“Energy”
let’s do something simple
independent neutrons
total number of spikes Kt = X
i
σi,t is informative of collective behaviour
P(Kt, Kt0)
|t − t0| < v
⬄ all neurons behave the same sampling 2N states is hard enough; here 2NL states — we need models
Tkacik Marre Mora Amodei Berry Bialek, JSTAT 2013
E = − X
t
h(Kt) − X
t v
X
u=1
Ju(Kt, Kt+u)
SLIDE 93 solving the problem
can be solved by transfer matrices (aka forward backward algorithm, or belief propagation in 1D)
E = − X
t
h(Kt) − X
t v
X
u=1
Ju(Kt, Kt+u)
Xt = (Kt, Kt+1, . . . , Kt+v−1) define a “super-variable” P({Xt}) = 1 Z exp "X
t
H(Xt) + X
t
W(Xt, Xt+1) # now becomes a 1D model
SLIDE 94 model predicts avalanche dynamics
10 10
1
10
2
10
3
10
−5
10
−4
10
−3
10
−2
10
−1
Avalanche size (number of spikes) Probability b.
v = 0 v = 1 v = 2 v = 3 v = 4 v = 5
SLIDE 95 model predicts avalanche dynamics
10 10
1
10
2
10
3
10
−5
10
−4
10
−3
10
−2
10
−1
Avalanche size (number of spikes) Probability b.
v = 0 v = 1 v = 2 v = 3 v = 4 v = 5
NB: no power laws in avalanche statistics
SLIDE 96 thermodynamics of spike trains
0.9 0.95 1 1.05 1.1 5 10 15 20
1/β c(β)
b.
N = 5 N = 10 N = 20 N = 30 N = 40 N = 50 N = 61 N = 97 N = 185
C(T)/NL
T
i
L
N
t
Mora Deny Marre PRL 2015
0.9 1 1.1 5 10 15 20
Temperature 1/β
a.
v = 0 v = 1 v = 2 v = 3 v = 4
C(T)/NL
T
v = temporal range
E = − log P({σi,t}) PT (σ) = 1 Z(T)e−E/T
C = VarT (E/T)
SLIDE 97 thermodynamics of spike trains
0.9 0.95 1 1.05 1.1 5 10 15 20
1/β c(β)
b.
N = 5 N = 10 N = 20 N = 30 N = 40 N = 50 N = 61 N = 97 N = 185
C(T)/NL
T
i
L
N
t
Mora Deny Marre PRL 2015
0.9 1 1.1 5 10 15 20
Temperature 1/β
a.
v = 0 v = 1 v = 2 v = 3 v = 4
C(T)/NL
T
v = temporal range
E = − log P({σi,t}) PT (σ) = 1 Z(T)e−E/T
!
C = VarT (E/T)
static (salamander) dynamic (rat)
SLIDE 98 scaling with network size
T
100 200 5 10 15 20
Network size N Var(surprise)/NL
b.
v = 0 v = 1 v = 2 v = 3 v = 4 v = 5
50 100 150 200 1 1.05 1.1 1.15 1.2
Network size N Peak temperature 1/βc
v = 0 v = 1 v = 2 v = 3 v = 4 v = 5
NL
SLIDE 99 conclusions
stationary maximum entropy models may capture
emergent behaviour in biological data but dynamic framework may be necessary to get parameters right in neural systems, heat capacity = useful indicator of critical properties critical signature enhanced by dynamical approach
application to other biological contexts?
SLIDE 100
random flickering checkerboard
SLIDE 101
random flickering checkerboard
static (salamander) dynamic (rat)
