From Complex to Simple: Hierarchical Free-energy Landscape - - PowerPoint PPT Presentation

SLIDE 1

From Complex to Simple: Hierarchical Free-energy Landscape Renormalized in Deep Neural Networks

Hajime Yoshino

Cybermedia Center & Department of Physics, Osaka Univ.

Deep Learning and Physics 2019@YITP

H. Yoshino, arXiv1910.09918 submitted to SciPost Phys referee round deadline Nov. 26 th… comments are welcome!

SLIDE 2

[Q] Why deep neural networks works?

data size # of parameters e.g.

• ver fitting

Common sense

• 1. poor generalization
• 2. many local minimum

learning is difficult (glassy dynamics) Empirical observations

• n deep networks
• 1. generalization is not bad
• 2. learning is not too difficult

Why??? Why???

108 106 ⌧

• G. Carleo, et. al, arXiv:1903.10563v1

SLIDE 3

History: Spinglasses, Hopfield model, error correcting codes,....

• Edwards-Anderson model(1975)

Quenched random spin-spin interactions

H = −

• <i,j>

Jijsisj

A lot of energetic degeneracies due to frustration Ground state: disordered

Jij = 0 J2

ij = J2

？ ？

si = ±1 (i = 1, 2, . . . , N)

• Hopfield model (1982) : associative memory

Firing of neurons

si = ±1

Jij = 1 √ M

M

X

µ=1

ξµ

i ξµ j

Jij

synaptic weight

ξµ

i = ±1

embedded patterns Hebb rule

Jij < 0

Jij > 0

• error correcting code： a statistical inference problem

send receiver tries to reconstruct si

Jij

Sourlas (1989)

SLIDE 4

… and back to p-spins… but with spin components and without quenched disorder

H. Yoshino, SciPost Phys. 4 (6), 040 (2018)

S1 S2 S0

M → ∞

<latexit sha1_base64="f6fwXjV2lpbD35eA2wSGmDwXznU=">ACcXichVHNSiNBEP4yrj+b9Se6F2Uvg0ERhFCjosaT6MWL4M9GBSMyM3Zi42R6mOkEY8gL7APoYQ/uCoLPoYX8CDjyB7zMJe9mBlEhERtRq6qr6qr/rbifwZKSJ7hJGx4fOru6ej8lPvX39A6nBoa1IlUNX5FzlqXDHsSPhSV/ktNSe2AlCYZcT2w7R8vN+nZFhJFU/ldDcReyS76siBdWzOUXzXzWpl56Rd0dT+VpkyWrOysZb4MrAzFlkb1lTqCnkcQMFGSUI+NAce7AR8dqFBULA2B5qjIUcybguUEeSuWXuEtxhM3rEe5Gz3Tbqc96cGcXsY+5QzFQ8pQ4TY3RLv6hBN/Sb7un/q9Nq8ZSmip7p8UVwf7At+HNf+ySuw1Dp9Yb6rWKGA+VitZfRAjzXu4LX7l5KyxubAxVhunC/rD+n/SHV3zDfzKX/dyXWx8R5K/4PGdzdeDramMNZ2ZWp9JLy61P6MHXzCKCX7xOSxiBWvI8bkBTnGOH4mGMWKYxmir1Ui0OZ/xzIzJByYljxU=</latexit>

1999 2001 1987 1991 2008 From spin glass to structural glass

Kirkpatrick-Thirumalai Wolyness (1989) Franz-Parisi (1995), Monasson (1995), Mezard-Parisi (1999) Parisi-Zamponi (2010), Charbonneau-Kurchan-Parisi-Urbani-Zamponi (2014) Yoshino-Mezard (2010), Yoshino-Zamoponi (2014), Rainone-Urbani-Yoshinno-Zamponi (2015)

d → ∞

<latexit sha1_base64="6R4WA9A0JTWCWI9zsDIx8f7WTbo=">ACcXichVHNSiNBEP4y6290Na4XxcuQoAgLoUZF1z3JevHoz0YFI2Fm7GjZHqY6YTNBl/AB9DHvwBQfExvOwL7CGPIB6z4MWDlUlEliVaDV1VX9VX/XW3E3gy0kT1hPGhq7unt68/OTD4cWg4NfJpM1Ll0BU5V3kq3HbsSHjSFzktSe2g1DYJcTW87hcrO+VRFhJX/XVcDsVuy931ZlK6tGcrvmXmtzLz0i7paSGUou0jW4rxl/h9YWYotg7atqtQ18tiDgosyShDwoTn2YCPitQMLhICxXdQYCzmScV3gCEnmlrlLcIfN6CHv+5ztFGf8+bMKGb/4A7FTMVTjmBikv7QDTXoN93SPT1nFaLpzTVNk7La4ICsPHYxuP7JK7DUOXlvqtYo4kusVrL6IEa93Bb/MrP08bG1/XJ2hRd0gPrv6A63fEN/Mpf92pNrP9Ckr/g5Z3NzsHmTNazc6szWvrU/ow8TSGOaX3wBS1jBKnJ8boATnOE80TDGDdNIt1qNRJszin/M+PwMVCPLA=</latexit>

p-spin to RFOT more on replicas

Jin-Yoshino (2017), Jin-Urbani-Zamponi-Yoshino (2018)

• 10 µm

SLIDE 5

Glass order parameter with replicas ✏ab = @G[ ˆ Q] @Qab

• ˆ

Q= ˆ QSP

= 0 G[ ˆ Q] = F[ˆ ✏] + N X

ab

✏abQab

replica 1 replica 2

We are interested with lim

✏ab→0+ lim N→∞ Qab

crystal

H[✏] =

n

X

a=1

H(sa) − X

a,b

✏ab

N

X

i=1

Sa

i Sb i

replicas a=1,2,…n

symmetry breaking field

Qab = 0 Qab > 0 liquid glass

spontaneous breaking of ergodicity

Explicit RSB : Parisi-Virasoro (1989)

−F[ˆ ✏] = ln TrSe−H[ˆ

✏]

Qab ≡ 1 N

N

X

i=1

D Sa

i Sb i

E = − 1 N @F[ˆ ✏] @✏ab

Overlap

SLIDE 6

Replica symmetry braking and ultra-metricity

• verlap matrix
• G. Parisi (1979)

first found in the SK model for spinglass

Qab =

breaking of ergodicity & permutation symmetry Edwards-Anderson (EA) order parameter Self-overlap

qEA = lim

b→a Qab

<latexit sha1_base64="GZjFwLgB+JxI2TeI0dcstFSRoKc=">ACh3ichVFNTxRBEH0MoLCiLHgx8TJxg/G01LJmARMTwJh4ZIEFEpZMZoYGOsxMDzO9G3EyVw/+AQ6eJCHR4FX/gBf/gAd+gvEIiRcP1nwkxBilOul+/ape9etuJ/RkrInOB4zBoeEbN0dGK7fGbt8Zr05MrseqF7mi4ypPRZuOHQtPBqKjpfbEZhgJ23c8seEcPMvyG30RxVIFa/oFNu+vRfIXenamimrah5aSTfyzeL6dOuJ30rcyuVqadm0rsZ3UqtaoPt+iBs2YBWi1StBsmo065VFDGcuq+h5d7EDBRQ8+BAJoxh5sxDy20AhZG4bCXMRI5nBVJUWNvjKsEVNrMHPO/xbqtkA95nPeNc/ZIrFCsVd0lhYoq+0Qe6oK90Rt/p1z+7JXmXzM0Rr06hFaE1/ube6s9rVT6vGvtXqv+61tjFXO5WsvswZ7J7uIW+/+r4YvXJylTykE7oB/t/R+f0hW8Q9C/d07ZYeYtK8QVZXD3432B9pt5o1pvtx7WFpfIzRnAfD/CIX3wWC3iBZXT43Nf4iE/4bIwa0bLmCtKjYFScxd/hLH4G9Ulos=</latexit>

a b c

distance

• verlap

Rammal-Toulouse-Virassoro (1986) Toulouse-Dehaene-Changeaux (1986)

Q(a, b) = min(Q(a, c), Q(b, c))

ultra-metricity

SLIDE 7

“Disorder-free” vector p-spin models on tree

“gap” p = 3 c = 4 H = − X

⌅

V (r⌅) . r⌅ = δ − 1 √ M

M

X

µ=1

Sµ

1(⌅)Sµ 2(⌅) · · · Sµ p(⌅)

Locally tree like lattice with connectivity c

Hamiltonian

S1 S2 S0

N⌅ = Nc/p = NMα/p < N p/p!

# of factor nodes

c = αM

|Si

2| = M

i = 1, 2, . . . , N

Si = (S1

i , S2 i , . . . , SM i )

continuous or Ising Sµ

i = ±1

“Inter-mediate sparseness “: high connectivity but not “global coupling”

H. Yoshino, SciPost Phys. 4 (6), 040 (2018)

SLIDE 8

e l but he a

• s

standard discrete coloring

“Vectorial” constraint satisfaction problems

antiferromagnetic Potts model

H = X

i,j

δqi,qj

“continuous” version

θ

“repulsive” vectorial spin model

H = X

i,j

V ✓ δ − Si · Sj √ M ◆

V (r) = lim

✏→∞ ✏r2✓(−r)

SLIDE 9

α = c/M

2 4 6 8 10 12 14

• 0.4
• 0.2

0.2 0.4

δ

α

Liquid q = 0 glass

q = 0

connectivity

Jamming (SAT-UNSAT) continuous RSB: hierarchical clustering of solutions

H. Yoshino, SciPost Phys. 4 (6), 040 (2018)

M → ∞

<latexit sha1_base64="f6fwXjV2lpbD35eA2wSGmDwXznU=">ACcXichVHNSiNBEP4yrj+b9Se6F2Uvg0ERhFCjosaT6MWL4M9GBSMyM3Zi42R6mOkEY8gL7APoYQ/uCoLPoYX8CDjyB7zMJe9mBlEhERtRq6qr6qr/rbifwZKSJ7hJGx4fOru6ej8lPvX39A6nBoa1IlUNX5FzlqXDHsSPhSV/ktNSe2AlCYZcT2w7R8vN+nZFhJFU/ldDcReyS76siBdWzOUXzXzWpl56Rd0dT+VpkyWrOysZb4MrAzFlkb1lTqCnkcQMFGSUI+NAce7AR8dqFBULA2B5qjIUcybguUEeSuWXuEtxhM3rEe5Gz3Tbqc96cGcXsY+5QzFQ8pQ4TY3RLv6hBN/Sb7un/q9Nq8ZSmip7p8UVwf7At+HNf+ySuw1Dp9Yb6rWKGA+VitZfRAjzXu4LX7l5KyxubAxVhunC/rD+n/SHV3zDfzKX/dyXWx8R5K/4PGdzdeDramMNZ2ZWp9JLy61P6MHXzCKCX7xOSxiBWvI8bkBTnGOH4mGMWKYxmir1Ui0OZ/xzIzJByYljxU=</latexit>

x 1 − q(x)

b)

10-5 10-4 10-3 10-2 10-1 100 10-4 10-3 10-2 10-1 100 6.7 6.71 6.72 6.722 6.724 6.726 power law

1 − q(x) = x−κ

κ = 1.415726...

Same jamming criticality as hard spheres and perceptron (p=1) Franz-Parisi (2016), Franz-Parisi-Sevlev- Urbani-Zamponi (2017)

clustering = glass transition

SLIDE 10

“Design space” of a multilayer-perceptron network

input

• utput

hidden

S0,1

<latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit>

S0,2

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S0,3

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S1,1

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S1,2

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S2,1 S3,1 S0,N S1,N

Si,l = (S1

i,l, S2 i,l, . . . , SM i,l )

Sµ

i,l = ±1

<latexit sha1_base64="IQ1ZI7jiQ/Wy8+UMKWaYvXyVZfA=">ACvnichVFNS9xAGH5Mq9WtH9v2IvQSulgUlmWyFloKwmIvRTUdV3B6JkZ9fBJBOTyaIN+QP+gR56aqHQ0t/QUy/9Az149iS2Jwu9NB3s0FRUd+BmWe93eWbGDlwRKcYOh7Q7d4dH7o2OFe6PT0xOFR8XItkHDq84UhXhu2FXFX+LyhHL5ehBy7Nd3rR3XvXzR4PIyH9VbUf8E3P6vqiIxLEdUqNhPT7uj1tJWIspsuzNYHYCsx0vIZrqZl021LFZ1Tb9I5XdfN3d3YautnrOnF6YIZeLrRKpZYhWhXwVGDkrIY0kWP8NEGxIOYnjg8KEIu7AQ0diAYaAuE0kxIWERJbnSFEgbUxVnCosYndo7tJuI2d92vd7Rpl6jyokKSV1SaFjhv1kX9gp+8G+smP279puSdal72afVnug5UFr6mC6/vdWlUerwva56kbXCh28yNwKch9kTP8ezkDfe/vutP5yZSZ5yj6yE/L/gR2y73QDv/fH+bTMV96jQF9gXH7wq2CtWjHmK9XlZ6XaYv4Zo3iMJ5ilF3+OGl5jCQ069xuO8Au/tZrW0TxNDkq1oVzCBdC2/sPJVqrwQ=</latexit>

L N

state of neurons with respect to M different patterns

α = M N N, M → ∞

Shun’ichi Amari (1971) “Esemble of random perceptrons”

SLIDE 11

activation function

S1

<latexit sha1_base64="QoVpjNLS1LOP8I7/d20UgzJjY=">ACaHichVFNLwNBGH6vujiwPi0mgqTs1sSYiTcHGktSpBmt01mHS7u9ndNmHjD7g4Ik4kIuJnuPgDn4CjpW4OHh3u4nQ4J3MzDPvM87z8zojik8n7GnhNTW3tHZ1d2T7O3rH0jJg0Prnl1zDa4atm7G7rmcVNYXPWFb/INx+VaVTd5Sa8shfulOnc9YVtr/oHDt6vaniV2haH5RKnFcqAcleUMy7Eo0q1AiUEGcazY8g2sAMbBmqogsOCT9iEBo/aJhQwOMRtIyDOJSifY4jJElboyxOGRqxFRr3aLUZsxatw5pepDboFJO6S8o0suyR3bIGe2B37Jl9/ForiGqEXg5o1pta7pRTx6PF939VZp97H+p/vTsYxdzkVdB3p2ICW9hNPX1w9NGcb6QDSbZFXsl/5fsid3TDaz6m3G9ygsXSNIHKD+fuxWs53PKdC6/OpNZWIy/ohvjmMAUvfcsFrCMFah0rsAJznCeJFkaUQa6ZKiVgzjG8hTXwCVvKLhw=</latexit>

J1

<latexit sha1_base64="zr/nSDwjQTQFp4csWjiF3RnDh0c=">ACaHichVHLgRBFD3T3uMxjQViIzMZsZpUIyFWwkasZjBGwmTS3cqo6Fe6eyah4wdsLBErEhHxGTZ+wMInYEliY+F2TwsiuJWqunXuPbdO1dUcQ3g+Y/cJqam5pbWtvSPZ2dXdk5J7+1Y9u+bqvKjbhu2uarHDWHxoi98g685LldNzeAlbWc+jJfq3PWEba34uw4vm2rVEltCV32CiouVQNmvyBklxyIbYT+cj1AGseVt+RIb2IQNHTWY4LDgk29AhUdjHQoYHMLKCAhzyRNRnGMfSeLWKItThkroDq1VOq3HqEXnsKYXsXW6xaDpEnMEWXbHrtgzu2X7IG9/VoriGqEWnZp1xpc7lRSB4PLr/+yTNp9bH+y/tTsYwvTkVZB2p0ICV+hN/j1vaPn5ZmlbDKztkT6T9j9+yGXmDVX/SLAl86RfJrA353VsdzykRuvDCZmZ2LW9GOYaQxRv89hVksI8i3StwiGOcJB4lWRqQhqpUiLm9ObSel3RsqLfw=</latexit>

J2

<latexit sha1_base64="be9vAOb/HSLVDCUOLTsC6symlCs=">ACaHichVE9SwNBEH05v+NHohYqNiEhYhU2UVCsgjZipYmXBGId+cal1zujrtLQEP+gI2lipWCiPgzbPwDFv4EtVSwsXDuElERdZbdnX0zb/btjmrpwnEZuw9IXd09vX39A8HBoeGRUHh0LOeYdVvjsmbqpl1QFYfrwuCyK1ydFybKzV53m1uLF8w1uO8I0Nt09i5dqSsUQO0JTXILktXIz1SqHY8kE8y3CfjgfoRg6tm6GL7GFbZjQUEcNHAZc8nUocGgUkQSDRVgJTcJs8oQf52ghSNw6ZXHKUAit0lqhU7GDGnT2ajo+W6NbdJo2MSOIszt2xZ7ZLbtmD+zt1pNv4anZY92tc3lVjl0MJl9/ZdVo93F7ifrT80udrDoaxWk3fIR7xVam9/YP3rOLmXizRl2zp5I/xm7Zzf0AqPxol1s8Mwpgl8b8LuTSyWSc4nUxnwsvdxpRT+mEcUs/fcC0ljFOmS6V+AQxzgJPEphaUKaqdKgQ5nHN9Mir4DSMuLgA=</latexit>

S2

<latexit sha1_base64="msQsOKxDYusk+CeQfiXTU+cko=">ACaHichVFNLwNBGH6vqo+ujgLqIhTs1sSYhTw8WxVUWCNLtrWpPuV3a3TWrTP+DiDiRiIif4eIPOPgJOJK4OHi73UQvJOZeaZ93nmRnNMYTnM/YQkzo6u7p74r2Jv6BwaQ8NLzh2TVX50XdNmx3S1M9bgiLF3hG3zLcblqagbf1Korf3NOnc9YVvrfsPhu6ZasURZ6KpPVLFQCjLNkpxiaRbG5E+gRCFKHK2fIUd7MGjhpMcFjwCRtQ4VHbhgIGh7hdBMS5hES4z9FEgrQ1yuKUoRJbpbFCq+2ItWjdqumFap1OMai7pJzENLtn1+yF3bEb9sjef60VhDVaXho0a20td0rJw7HC278qk2Yf+5+qPz37KGMx9CrIuxMyrVvobX394PilsLQ2HcywC/ZM/s/ZA7ulG1j1V/0yz9fOkKAPUL4/90+wkUkrc+lMfj6VXY6+Io4JTGW3nsBWawihyKdK3CE5zGniRZGpXG26lSLNKM4EtIUx9Y84uI</latexit>

S0

<latexit sha1_base64="CpYN3TX7zie2aju2flZiK+fYnXI=">ACaHichVFNLwNBGH6vujxQFxaTQVp+YtEuLUcHFEVSXVNLtrWhPb3c3utglN/4CLI+JEIiJ+hos/4OAn4Eji4uDtdhOhwTuZmWeZ93npnRbEO6HtFjSOno7Oru6e0L9w8MDkWiwyNbrlV1dJHVLcNytjXVFY0RdaTniG2bUeoFc0QOW1/pbmfqwnHlZa56R3YolBRy6YsSV31mMpminVqFKNxSpIfsXaQCkAcQaxZ0WvsYBcWdFRgYAJj7EBFS63PFIg2MwVUGfOYST9fYEGwqytcpbgDJXZfR7LvMoHrMnrZk3XV+t8isHdYWUMCXqgG3qle7qlJ/r4tVbdr9H0csCz1tIKuxg5Gs+8/6uq8Oxh70v1p2cPJSz6XiV7t32meQu9pa8dnrxmljYS9Wm6pBf2f0GPdMc3MGtv+tW62DhHmD8g9fO528HWbDI1l5xdn4+nl4Ov6MUkpjD72ANFaxhiyfK3GMU5yFnpWoMqZMtFKVUKAZxbdQpj4BVPGLhg=</latexit>

Sc JN

<latexit sha1_base64="VrJZuhLJvL/qUjHzqDOU0SeTNg=">ACaHichVFNLwNRFD0d3/XRwQKxaTEqnktCbFq2IiFtGhJkGZmvNZL5ysz0yY0/QM2logViYj4GTb+gIWfgCWJjYU70xFEcF/e/ede89572r2rpwPcbuI1JLa1t7R2dXtLunty8m9w8UXKvqaDyvWbrlbKiKy3Vh8rwnPJ1v2A5XDFXn62plwY+v17jCstc8/Zsvm0oZVOUhKZ4BOWXivXlRlFOpJIsDj74XyEgta8mX2MIOLGiowgCHCY98HQpcGptIgcEmbBt1whzyRBDnaCBK3CplcpQCK3QWqbTZoiadPZrugFbo1t0mg4x4xhnd+yKPbNbds0e2NuvtepBDV/LHu1qk8vtYuxgePX1X5ZBu4fdT9afmj2UMBtoFaTdDhD/FVqTX9s/el6dWxmvT7Bz9kT6z9g9u6EXmLUX7SLHV04R/dqA351COpmaSqZz04nMfNiKToxiDJP03zPIYBFZ5OlegUMc4yTyKMnSkDTSTJUiIWcQ30waeweA54uc</latexit>

Perceptron （McCulloch-Pitts model)

S0(t + 1) = sgn 1 √ N

N

X

i=1

JiSi(t) !

<latexit sha1_base64="2O2+RHzQ/Co0/j53Yd1tJ6OqD0=">ACx3ichVHLahRBFD1pX3F8ZNSN4KZxiMwgDNVRUIRA0I26kCTjJIF0bLo7NZMi/bKqZkgsauHWH3DhSkFE/Aw3+QEX+QR3ESICxfe7mkQDeotqurec+5daoqKhKhNGP7U86x4ydOnpo+3Thz9tz5meaFiysqH8mY9+M8yeVaFCqeiIz3tdAJXyskD9Mo4avR9r0yvzrmUok8e6x3C76RhsNMDEQcaoKCZtQLDLNtfd3ruPGl6mrhpl1/YQPdNsfyDA2njW+eiq1eWStr0ZpYMS8Z59Q6Pqa7+hKhJF805qHlLO2V65t3fGlG7pTtBsS6rzD3qeLXTQm2LefMdfGwiR4wRUnBk0OQnCKForMDQ0HYBgxhkjxR5TksGsQdURWnipDQbVqHFK3XaEZx2VNV7JhOSWhKYrqYZ/Ye3bA9tgH9pn9+GsvU/UotezSHk24vAhmXlzuHf6XldKusfWL9U/NGgPcrQK0l5USHmLeMIfP3t50LuzPGusTfsC+l/zfbZR7pBNv4Wv13iy6/QoA/w/nzuo87KXNe70Z1butlauFt/xTSu4Cra9N63sID7WESfzt3DVxziu/PAyZ2xszMpdaZqziX8Zs7zn1BHsdo=</latexit>

fixed point patterns

µ = 1, 2, . . . , M

Statistical mechanics on the “ensemble of fixed points”

Elisabeth Gardner (1957-1988)

Gardner volume “Gap”

e−βV (h) = θ(h)

“Hardcore” constraint

Si(t) = Sµ

i

<latexit sha1_base64="wuyXAWKQJXyT91z0o8NeC5Bg81Y=">ACeXichVHLSgMxFD0dX7W+6mMhuFGLUl2UVAVFEQ3LvuwKvgoM2PU4LyYSQt16A/4Ay7cqCBS/Qw3/oALP0FcKrjQhbfTAVFRb0hycnLPzUmiOYbwJGMPEaWpuaW1Ldoe6+js6u6J9/ateXbJ1XlBtw3b3dBUjxvC4gUpME3HJerpmbwde1wub6/XuauJ2xrVYcvm2q+5bYE7oqiSrGB/JFX1STcmIhADv+lmqFuMJlmJBDP8E6RAkEbGjl9hC7uwoaMExwWJGEDKjxqm0iDwSFuGz5xLiER7HNUESNtibI4ZajEHtK4T6vNkLVoXa/pBWqdTjGou6Qcxhi7ZzX2zO7YDXtk7/W8oMadS8VmrWGljvFnuPB/Ou/KpNmiYNP1Z+eJfYwF3gV5N0JmPot9Ia+fHTynJ/Pjfnj7I9kf9z9sBu6QZW+UW/zPLcKWL0Aenvz/0TrE2l0tOpqexMYnEp/IohjCKJL3LBaxgwKdO4RzlDdeRNGVGSymQjVYmEmn58CWX6AxVki8=</latexit>

V = Z

N

Y

j=1

dJj √ 2π e−

Jj 2 2

M

Y

µ=1

e−βV (rµ)

<latexit sha1_base64="ot8kMOWPau9oxqoIrzVKG6sTgNo=">AC/HichZHNahRBEMdrRqNxjGaNF8HL4JIQwSw9G8EgBIJeRFDy4W4CmewyM1sbO5kve3oXY9M+gC+QgydFEQlefQEvoCHPIJ4jOAlgdTMDorGJD30dPW/6ldV3e2nIc8kY7uGebs0LnzwxesiyOXLo9Wrow1s6QnAmwESZiIFd/LMOQxNiSXIa6kAr3ID3HZ37yf+5f7KDKexE/kVoprkbce8y4PElSu7LdnLVcHkvbclORdNpqY9bRLfVYu13hBapjuxKfy6KOEtjR6iGFaK2Vmz0TUtXdlGtYUtNDYD/h7dUnRCav8u4Ua8o9KiEfZSe3ZwUrdyjb+p2pcpqrBj2UcMpjSqUYz6pfAXOpBAD2IACEGSXYIHmT0rYIDFLS1kCRJsjihR9Bg0Vsj6KQIjxSN+m/TrvVUo1pn+fMCjqgKiFNQaQN4+wb+8j2Fe2w76z/WNzqSJH3sWrf6AxbQ9+ura0q9TqYhWCU/UCf2LKELM0WvnHpPCyU/RTDg+y+295buLo6rCfaW/aD+37Bd9oVOEPd/Bu8XcPE1WPQAzr/XfdRo1mvOdK2+cLs6d698imG4Djdgku7DszBA5iHBtU9MKrGLWPKfGm+M3fMT4NQ0yiZq/DXMD8fAg87xfA=</latexit>

α = M N

<latexit sha1_base64="buqQMfIAvnJWoco3d5kEck62VG8=">ACeXichVHLSsNAFD2Nr1ofrXUjuKkWRVyUqQqKIBTduFHaqtgpUziVEPTJCRpUN/wB9wIQgKYsXPcOMPuPATxGUF7rwNg2IinoHZs6ce8+dMzOyqam2w9hjQOro7OruCfaG+voHBsORoWjeNqWInKoRnWtsxtoam6yDmqo4lt0xK8ImtiSy6vtPJbNWHZqFvOkem2K3wfV0tqQp3iCpGogWumQd8qVCyuOKu1d31ejESZwnmRewnSPogDj/SRqSBAvZgQEVFQjocAhr4LBp7CAJBpO4XbjEWYRULy9QR4i0VaoSVMGJLdO8T7sdn9Vp3+pe+pDqjBIaVCXOmKYA/shjXZPbtlT+z9126u16Xl5ohWua0VZjF8MrLx+q+qQquDg0/Vn64dlLDguVXJvekxrXsobX3t+LS5sZidcCfZJXsm/xfskd3RDfTai3KVEdkzhOgLkt8f/CfIzySs4mZzFw8tex/RhCjGMcUvfg8UlhFGjk69xDnuEYj8CaNSVPSdLtUCviaYXwJafYDwveSBA=</latexit>

rµ = Sµ

N

X

i=1

1 √ N JiSµ

i

<latexit sha1_base64="P4+YCNesE0u+NLr7tbSB2gpgvQ=">ACunichVHLThRBFD0Ijg+GVD4mbiBONqUo0mECMJkQ1xYBxgITGTndTA5XpF9XVE7BSP+APuHAlCTHGT3Dpxh9wdKlsTEjQtvPxKiRL2Vqjr31D3qry01BkirGTEWv0tjl8YkrjavXrt+YbN68tZ4luQx4L0jCRG76XsZDEfOeEirkm6nkXuSHfMfLBXnG0MuM5HEz9RhyrcjbzcWfRF4i32ZPtRPlZqHVdTUzleNkeRqsWCT/9Q4fekF2jbayfalIsI4ih+osraWfMfoJxRsTLcSF9htlmHlda6COwatFHbStJ8Cwc7SBAgRwSOGIpwCA8ZjS3YEiJ24YmThIS5TmHQYO0OUVxivCIHdC6S95WzcbkFzmzUh1QlZCmJGULM+wze8fO2Cf2n1lP/+aS5c5il4OafcrLU/dyZfT3R/VUW0K+ydq/7Zs0If82WvgnpPS6a4RVDphy9enXUfrs3ou+yInVL/b9gJ+0g3iIfg+NVvYaDfoA+8/nvgjWZzv2/c7s6oP24uP6KyZwG3dwj957DotYxgp6VPcDvuAbTq1Hlm8Ja1CFWiO1Zgq/maV+AVavrvM=</latexit>

N, M → ∞ with fixed α

<latexit sha1_base64="moARqhCvJm0Z/4cZNVEckuXMI/M=">ACi3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQuoqPjp+CrElOQrxGTmpZVUqiUZ5ZkKRlVqSmKjEJOYUZCSqxAsoG+gZgIECJsMQylBmgIKAfIEVDEMKQz5DMkMpQy5DKkMeQwlQHYOQyJDMRBGMxgyGDAUAMViGaqBYkVAViZYPpWhloELqLcUqCoVqCIRKJoNJNOBvGioaB6QDzKzGKy7AqgiH6gzH2hKLYMCg6rBVYOVBp8NThisNnhp8AenadVgU0CuqQTSRC9qQXx/F0Swd8J6soF0iUMGQhdeF1dwpDGYAF2bSbQ9QVgEZA/kiH6y6qmfw62ClKtVjNYZPAa6P6FBjcNDgN9kFf2JXlpYGrQbAYucBRYgoAxPMAxGWFGeobGesaBRsoOTtDI4GCQZlBi0ACGuDmDA4MHQwBDKNDeLoYNDsZdjHxMhkzWTHZQJQyMUL1CDOgACZXAIDlq8=</latexit>

SLIDE 12

Sµ

⌅(1)

Sµ

⌅(2)

Sµ

⌅

J1

⌅

J2

⌅

Sµ

⌅(N)

<latexit sha1_base64="8TJBS5cQywa2sByQTfTnXTF/7rU=">ACfnichVFNLwNBGH6vq+igtxEQ3hoKYl0Tg1XJykVEuiNLtryqb7ZT+a1KZx9wcnEgaEQk/wsUfcPATxLESFwdvt5sIgncyM8z7vPDMjmapiO4w9hYS29o7OrnB3pKe3r38gOjiUtw3XknlONlTD2pZEm6uKznO4qh827S4qEkq35LK839rQq3bMXQN52qyXc18UBXSosOkQVo6PZPa+gubWiV5BUS7bR65o8em1mVoxGmNx5sf4T5AIQAxBZIzoFQrYhwEZLjRw6HAIqxBhU9tBAgwmcbvwiLMIKf4+Rw0R0rqUxSlDJLZM4wGtdgJWp3Wzpu2rZTpFpW6RchyT7JFdswZ7YDfsmb3/WsvzazS9VGmWlpuFgdOR7Jv/6o0mh0cfqr+9OyghJTvVSHvps80byG39JXjs0Z2aWPSm2KX7IX8X7Andk830Cuvcn2db5wjQh+Q+P7cP0E+GU/Mx5PrC7H0cvAVYxhAtP03otIYxUZ5OjcE9RxizsBwpQwK8y1UoVQoBnGlxBSH/TUk20=</latexit>

Sµ

⌅ = sgn

1 √ N

N

X

i=1

Ji

⌅Sµ ⌅(i)

!

<latexit sha1_base64="8fA3cAJF8y34tZihZo9XmxWhHFk=">AC8nichVFLaxRBEK4ZX3F9ZNWL4mXIEtlclp5EUIRA0It4kCTrJoF0dpjp9G6anVe6exZj039A8CzoKYqI6L/wkj+Q36CeFzBiwdrZwfExEc13V31dX1VX3dHeSyUJuTIcU+dPnP23NT52oWLly5P169cXVNZIRnvsCzO5EYUKh6LlHe0DHfyCUPkyjm69Hgwfh8fcilEln6RO/lfCsJ+6noCRZqhIL6i3bX0KSwgaFRHLKB2i1Cye2ioTLxVD+1Ho15TzdpT4bM+NZQtSu1eWwtVUSGLHo2y6GHtX8qS71GMm3rXl0rGLXCGv/2Kwp5iyVor+j54J6g7RIad5Jx6+cBlS2nNXfA4VtyIBAQlwSEGjH0MICscm+EAgR2wLDGISPVGec7BQ26BWRwzQkQHuPYx2qzQFONxTVWyGXaJcUpkejBLDskHMiIH5CP5Qn78tZYpa4y17OEeTbg8D6afX29/y8rwV3Dzi/WPzVr6MHdUqtA7XmJjG/BJvzhs5ej9r3VWXOLvCFfUf8+OSKf8Qbp8Bt7t8JX0MNP8A/twnbX5lr/Qml+53Vi6X3FNyEGWjie9+BJXgIy9DBviPnhjPjNFztvnL3beTVNepONfgN3M/QSdqsPi</latexit>

JN

⌅

<latexit sha1_base64="PzUIaUxmlcfnXo6NBtE8K0QXcG4=">ACeXichVHNLgNRFP46/un9bOQ2KAhYtHcthJiJWzEQqoUCTWZGReTzp+ZaRMmfQEvYGDRKQ8ho0XsOgjiCWJBQtnpiOIlHNz7z3O+c797v3yJamOi5jtYjQ1NzS2tbeEe3s6u6JxXv71h2zZCs8r5iaW/KksM1eB5V3U1vmnZXNJljW/IxQU/vlHmtqOaxp7ZPGCLu0b6p6qSC5BYnxgSfS2ZU1Sis5hSbJ5ZcdbrojxRCrJAhtmv5zPUAKhZc34NbaxCxMKStDBYcAlX4MEh8YWUmCwCvAI8wmTw3iHBVEiVuiLE4ZEqFWvfptBWiBp39mk7AVugWjaZNzGMsQdWZc/snt2yR/besJYX1PC1HNEu17ncEmMng6uv/7J02l0cfLH+1OxiDzOBVpW0WwHiv0Kp8vHp8+rs7kxb5xdsifSf8Fq7I5eYJRflKsVnjtD9HsDGjvr6WQqk0yvTCXm5sNWtGMIo5ig/57GHBaRZ7uPcY5qriJvAkjwoQwWU8VIiGnHz9MyHwAzCSow=</latexit>

S0,1

<latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit>

S0,2

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S0,3

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S1,1

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

input

• utput

hidden

S1,2

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S2,1 S3,1 S0,N S1,N

Usual strategy of learning (1) define “loss function"

E =

N

X

i=1 M

X

µ=1

⇣ Sµ

L,i − (S∗)µ L,i

⌘2

e.g. desired output (2) try to minimize the loss function via back-propagation e.g. SDG (stochastic gradient descent)

Too much long-ranged, highly convoluted, non-linear interaction! …hard to analyze

Sµ

L,i(t + 1) = sgn

@ 1 √ N

N

X

j=1

JL,i,jsgn 1 √ N

N

X

k=1

JL−1,j,k · · · sgn 1 √ N

N

X

m=1

J1,l,mSµ

0,m(t)

!!1 A

<latexit sha1_base64="30hvt09FDcdy8cKr/afOeav6xU=">ADinicjVHLbtNAFL2ueZRQaAobJDYWoSgVJhoHJF6qVAELhBqG9JWqlvLnkxSJ34xM4ko/kB+AWrEBCLFiC0s2/ACLfgJiWSQ2LhxLBCtCL2Wfe89c8/xmZkgi0IhCdkxJsxDh48cnTxWOj514uR0ebUikj7nLImTaOUrwW+YFGYsKYMZcTWMs78OIjYatC7PVxfHTAuwjR5KLczthH7nSRsh9SXCHkzxvnGpnLjvbUfTvUVXnRmbPmlctjS3QSbkRa8uq2+Y+VY5WrnjEpXqgtSv6sae6847exNZyJXsczeKs5ZW93I5u6v1QaV646QuOXbX7uEobaVSWKWDisZjRB07smOtf2+fYFeVcy4PO1t7k1eukBrJw9pfOEVRgSIW0/IbcKEFKVDoQwMEpBYR+CDwGcdHCQIbYBCjGOVZivM9BQm4fpxhO+Ij28NvBbr1AE+yHmiJnU/xLhC9HpgWz5At5S3bJZ/KOfCU/6mlco2hl23MwYjLMm/6ZnGj/+yYswStv6wxnqW0IZrudcQvWc5MtwFHfEHT57vNm4sz6oL5BX5hv5fkh3yCXeQDL7T10ts+QWU8AKcvce9v1ip15zLtfrSlcrCreIqJuEsnIMqnvdVWIC7sAhNoMYz473xwfhoTpl187p5czQ6YRSc0/BXmHd+Ab3E+i4=</latexit>

SLIDE 13

Gardner volume generalized for a multi-layer network “Gap”

(c.f. ) internal representation (2-layer): R. Monasson and R. Zecchina (1995) Hamiltonian with “short-ranged” interactions trace over hidden variables S0,1

<latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit>

S0,2

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S0,3

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S1,1

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

input

• utput

hidden

S1,2

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S2,1 S3,1 S0,N S1,N

e−βV (h) = θ(h)

“Hardcore” constraint

V (S(0), S(L)) = eNMS(S(0),S(l)) = @

L−1

Y

l=1 N

Y

i=1

X

Sµ

l,i=±1

1 A @ Z Y

⌅ N

Y

j=1

dJj

⌅

√ 2π e−

(Jj ⌅)2 2

1 A e−βH H =

M

X

µ=1

X

⌅

V (rµ

⌅)

Gaussian approx.

• r modified model

ξµν: Gaussian with zero mean and variance 1

rµ

⌅ = N

X

i=1

1 √ N Ji

⌅

1 √ M

M

X

ν=1

ξµνSν

⌅(i)Sν ⌅

!

<latexit sha1_base64="LFsx4CZQdFw0Nk7Cs76hK6jsVEQ=">ADQnichVHLahRBFL3dvmL7yEQ3gpvGITLZDNVRMAiBYDYiRJKMkwRSma7UzMpl+prh4Si/oBf8CFKwUR8TN04Q+4yMIPUFcygpsvFPToE583Karbp17z6lTVWEe80IScmTZp06fOXtu6rxz4eKly9O1mSsbRVaKiLWjLM7EVhgULOYpa0suY7aVCxYkYcw2w/7yqL45YKLgWfpIHuZsJwl6Ke/yKJAI+bWPoqNoUmpf0TAOon6xXwaC6UXHpUWZ+IoverqjHmqXdkUQKU8rWuwLiYimkh1I40AJtqvVgwmNjuJaOzRmXdmYK8g28jTtDQbrPyqFsYlw84Dbqxhi9ZuqzPqnbDZ4HP6jwVNBe/tyTm/VidNYsI9mXhVUocqVrPaK6CwCxlEUEICDFKQmMcQIHfNnhAIEdsBxRiAjNu6gw0OMgtsYthR4BoH8cerYrNMX1SLMw7Ah3ifEXyHRhlnwgr8mQvCdvyCdy/FctZTRGXg5xDsdclvT61v+XleAsYe8n65+eJXRhwXjl6D03yOgU0Zg/ePx02Lq7PqtukhfkC/p/To7IWzxBOvgWvVxj68/AwQfwJq/7ZLIx3/RuNefXbteX7lVPMQX4QY08L7vwBLch1VoQ2QtW9wSVmG/sz/bX+3huNW2Ks5V+C3s4x+kr+ap</latexit>

Sµ

⌅(1)

Sµ

⌅(2)

Sµ

⌅

J1

⌅

J2

⌅

Sµ

⌅(N)

<latexit sha1_base64="8TJBS5cQywa2sByQTfTnXTF/7rU=">ACfnichVFNLwNBGH6vq+igtxEQ3hoKYl0Tg1XJykVEuiNLtryqb7ZT+a1KZx9wcnEgaEQk/wsUfcPATxLESFwdvt5sIgncyM8z7vPDMjmapiO4w9hYS29o7OrnB3pKe3r38gOjiUtw3XknlONlTD2pZEm6uKznO4qh827S4qEkq35LK839rQq3bMXQN52qyXc18UBXSosOkQVo6PZPa+gubWiV5BUS7bR65o8em1mVoxGmNx5sf4T5AIQAxBZIzoFQrYhwEZLjRw6HAIqxBhU9tBAgwmcbvwiLMIKf4+Rw0R0rqUxSlDJLZM4wGtdgJWp3Wzpu2rZTpFpW6RchyT7JFdswZ7YDfsmb3/WsvzazS9VGmWlpuFgdOR7Jv/6o0mh0cfqr+9OyghJTvVSHvps80byG39JXjs0Z2aWPSm2KX7IX8X7Andk830Cuvcn2db5wjQh+Q+P7cP0E+GU/Mx5PrC7H0cvAVYxhAtP03otIYxUZ5OjcE9RxizsBwpQwK8y1UoVQoBnGlxBSH/TUk20=</latexit>

Sµ

⌅ = sgn

1 √ N

N

X

i=1

Ji

⌅Sµ ⌅(i)

!

<latexit sha1_base64="8fA3cAJF8y34tZihZo9XmxWhHFk=">AC8nichVFLaxRBEK4ZX3F9ZNWL4mXIEtlclp5EUIRA0It4kCTrJoF0dpjp9G6anVe6exZj039A8CzoKYqI6L/wkj+Q36CeFzBiwdrZwfExEc13V31dX1VX3dHeSyUJuTIcU+dPnP23NT52oWLly5P169cXVNZIRnvsCzO5EYUKh6LlHe0DHfyCUPkyjm69Hgwfh8fcilEln6RO/lfCsJ+6noCRZqhIL6i3bX0KSwgaFRHLKB2i1Cye2ioTLxVD+1Ho15TzdpT4bM+NZQtSu1eWwtVUSGLHo2y6GHtX8qS71GMm3rXl0rGLXCGv/2Kwp5iyVor+j54J6g7RIad5Jx6+cBlS2nNXfA4VtyIBAQlwSEGjH0MICscm+EAgR2wLDGISPVGec7BQ26BWRwzQkQHuPYx2qzQFONxTVWyGXaJcUpkejBLDskHMiIH5CP5Qn78tZYpa4y17OEeTbg8D6afX29/y8rwV3Dzi/WPzVr6MHdUqtA7XmJjG/BJvzhs5ej9r3VWXOLvCFfUf8+OSKf8Qbp8Bt7t8JX0MNP8A/twnbX5lr/Qml+53Vi6X3FNyEGWjie9+BJXgIy9DBviPnhjPjNFztvnL3beTVNepONfgN3M/QSdqsPi</latexit>

JN

⌅

<latexit sha1_base64="PzUIaUxmlcfnXo6NBtE8K0QXcG4=">ACeXichVHNLgNRFP46/un9bOQ2KAhYtHcthJiJWzEQqoUCTWZGReTzp+ZaRMmfQEvYGDRKQ8ho0XsOgjiCWJBQtnpiOIlHNz7z3O+c797v3yJamOi5jtYjQ1NzS2tbeEe3s6u6JxXv71h2zZCs8r5iaW/KksM1eB5V3U1vmnZXNJljW/IxQU/vlHmtqOaxp7ZPGCLu0b6p6qSC5BYnxgSfS2ZU1Sis5hSbJ5ZcdbrojxRCrJAhtmv5zPUAKhZc34NbaxCxMKStDBYcAlX4MEh8YWUmCwCvAI8wmTw3iHBVEiVuiLE4ZEqFWvfptBWiBp39mk7AVugWjaZNzGMsQdWZc/snt2yR/besJYX1PC1HNEu17ncEmMng6uv/7J02l0cfLH+1OxiDzOBVpW0WwHiv0Kp8vHp8+rs7kxb5xdsifSf8Fq7I5eYJRflKsVnjtD9HsDGjvr6WQqk0yvTCXm5sNWtGMIo5ig/57GHBaRZ7uPcY5qriJvAkjwoQwWU8VIiGnHz9MyHwAzCSow=</latexit>

rµ

⌅ = N

X

i=1

1 √ N Ji

⌅Sµ ⌅(i)Sµ ⌅

<latexit sha1_base64="I3kOm1j0xPHvbkwIKL9Rj68l4U=">AC8nichVHNThRBEK4Z/MH1hwUvEi8bNhi8bHrQBGNCQvRCPBgXSBh2ElP04udnT+6ezZip1/AxLMJnoAY/QtvPgCHngE43FJuHiwdnYSI6BWp7urv6rvq+ruMIuE0oQcO+7IpctXro5eq1y/cfPWHV8Yk2luWS8xdIolRshVTwSCW9poSO+kUlO4zDi62H36SC+3uNSiTR5rvcyvhXTnUR0BKMaoaD6RraNH+c2MH4YUdZVuzmV3M7XfJXHgRHznm2bJet3JGXGs8ZXu1IjYH3NX+qivpF825pnZxTaRljbvEh8Rty3tQsjNqjWSYMUVjveKVTh9KW0+oH8GEbUmCQwcEtDoR0B4dgEDwhkiG2BQUyiJ4o4BwsV5OaYxTGDItrFdQdPmyWa4HmgqQo2wyoRTonMGkyTb+Qj6ZOv5BP5Tn7+VcsUGoNe9nAPh1yeBWOv7zRP/8uKcdfw4jfrnz1r6MCjoleBvWcFMrgFG/J7r972m49Xp809ckh+YP8H5Jh8wRskvRP2foWvoMKfoB39rnPO2uzDe9BY3blYX3hSfkVo3AXpmAG3sOFmARlqGFdfvOpDPl1F3t7rsH7tEw1XVKzm34w9zPvwCO18S8</latexit>

SLIDE 14

glass order parameters same random input same random

• utput

machine 1

J1

⌅,i

J2

⌅,i

Jn

⌅,i

machine 2 machine n

S1

⌅

S2

⌅

Sn

⌅

qab,⌅ = 1 M

M

X

µ=1

(Sµ

⌅)a(Sµ ⌅)b

Qab,⌅ = 1 N

N

X

i=1

Ja

⌅,iJb ⌅,i

Replicas: machines learning in parallel

SLIDE 15

Parisi's RSB ansatz

• verlap matrix

Probability distribution of

• verlap between replicas

P(Q) = dx(Q) dQ Qab = Q(x) n n n → 0 l 1 2 L

Extension to multi-layers

1

<latexit sha1_base64="3bzTuENv8W8Lb0MR5px024P0Uk=">ACZnichVFNLwNRFD0dX1VfRYTEptEQq8mtSqpWwsZSW0WCyMx4mJjOTGamjWr6B9gSCysSCfEzbPwBC/+AWJLYWLgzHUQE9yXv3XvuPfed+5qG7rEd1HpKbmlta2aHuso7Oruyfe27fkWmVHE0XNMixnRVcYeimKHq6Z4gV2xFKSTXEsro75+eXK8Jxdctc9Kq2WC8p26a+pWuKx1AutRFPpmQKLEFyhig7mWUnRD5SYS2YMUvsYZNWNBQRgkCJjz2DShwea0iBYLN2DpqjDns6UFeoI4Yc8tcJbhCYXSX92OVkPU5Njv6QbsPa6wmGlxlzoSGKU7uqJnuqVreqS3X7vVgi6+miqfaoMr7I2eg6HC67+sEp8edr5Yf6r2sIWpQK3O6u0A8efQGvzK/slzYTo/Whujc3pi/Wd0Tzc8gVl50S5yIn+KGH8ByVnf0p8P/tNZmpBTaTmdm0zOzIafEcUwRjDOL57BDOaxgCLfK3CIxHqRuaUAabJRKkZDTj28mJd4B41uKhg=</latexit> <latexit sha1_base64="JbTPy8hR9fKbM/TxJyClFpEMlU=">ACZnichVFNLwNRFD0dX1VfRYTEptEQq8mtSqpWwsZSW0WCyMx4mJjOTGamjWr6B9gSCysSCfEzbPwBC/+AWJLYWLgzHUQE9yXv3XvuPfed+5qG7rEd1HpKbmlta2aHuso7Oruyfe27fkWmVHE0XNMixnRVcYeimKHq6Z4gV2xFKSTXEsro75+eXK8Jxdctc9Kq2WC8p26a+pWuKx1CONuLJlEyBJUjOEGUns+yEyEcqidAWrPgl1rAJCxrKEHAhMe+AQUur1WkQLAZW0eNMYc9PcgL1BFjbpmrBFcojO7yvs3RaoiaHPs93YC9xUWMy3uUkcCo3RHV/RMt3RNj/T2a7da0MVXU+VTbXCFvdFzMFR4/ZdV4tPDzhfrT9UetjAVqNVZvR0g/hxag1/ZP3kuTOdHa2N0Tk+s/4zu6YnMCsv2kVO5E8R4y8gOetb+vPBfzpLE3IqLadzk8mZ2fAzohjGCMb5xTOYwTwWUOR7BQ5xhOPIg9QtDUiDjVIpEnL68c2kxDvhWYqF</latexit>

x

<latexit sha1_base64="NxeohubzUKfMFGqJK+8u4N+6W0=">ACZnichVFNLwNRFD0d3/VREhsGk3FanKrolg1bCwVRVIiM+OVienMZGbaoOkfYEsrEgkxM+w8Qcs/ANiSWJj4c50EBHcl7x37n3Hfue6pt6K5HdB+RGhqbmlta26LtHZ1d3bGe3mXKjuayGuWYTmrquIKQzdF3tM9Q6zajlBKqiFW1J1ZP79SEY6rW+aSt2eL9ZKyZepFXVM8hnK7G7FESqbA4iRnxidpIsNOiHykEght3opdYg2bsKChjBIETHjsG1Dg8iogBYLN2DqjDns6UFeoIYoc8tcJbhCYXSH9y2OCiFqcuz3dAP2LldYzLS4Sw1xJOmOruiZbumaHunt127VoIuvZo9Ptc4V9kb3weDi67+sEp8etr9Yf6r2UMRkoFZn9XaA+HNodX5l/+R5cXohWR2hc3pi/Wd0Tzc8gVl50S5yYuEUf4Ckqd8S38+E9neUxOpeV0bjyRnQk/oxVDGMYov3gGWcxhHnm+V+AQRziOPEhdUr80UC+VIiGnD9Mir8Dd9qK0A=</latexit>

SLIDE 16

Replicated Gardner volume

qab(0) = qab(L) = 1

Replicated free-energy

−βF(S0, SL)

visible

NM = ∂nV n(S0, SL)

visible

• n=0

NM = Sn[{ ˆ Q(l), ˆ q(l)}]

<latexit sha1_base64="WJyPO9UnhzkN9QKvTFjHaiLgv7k=">ADbnicjVFbaxNBFD6b9VLjpWkFEURcDJUapi0giIUioL4oNI0Jq1k0jC7TpKhs5fuTiJ1nD/gH/DBJwUfxJ8hgi8+tCfIL4IFXzxwbMXLFq8zLIz3xzvu+cmeNGUiSKkF2rZB86fOTo1LHy8RMnT01XZmY7STiOPd72QhnGy5LuBQBbyuhJN+IYs58V/J1d+tmer4+4XEiwuC+2ol4z2fDQAyExRS/coHOoiZpy9Tlyvm0BjUyt9q6apO3Bapq+JWfiJ75h5s6lp7DsTkQhMYoy+d9cs5y5U8oGqOzRisRJM9nVg9h07m7j9X1cai+FIOU/QYpkUOcqt1LBLtaYjpymqcn5hRxvp5iaXr9SJXWSDecgaBSgCsVYDSvgMJDCMGDMfjAIQCFWAKDBL8uNIBAhFwPNHIxIpGdczBQRu0YozhGMGS3cB7irluwAe5TzyRTe5hF4h+j0oE58pG8JnvkPXlDPpHvf/TSmUdayw6ubq7lUX/6dnWt3+qfFwVjPZVf61ZwQCuZbUKrD3KmPQWXq6fPH6217q+NqcvkZfkM9b/guySt3iDYPLVe9Xka8z9wA1j7Lb+ln+AN9XI+9iVlMgWeROYBuzpO+K74nNa/zeqoOgs1hvLNUXm1eqKzeKNk7BObgINezViB27AKbfCsJeuB5Vpe6Yt9xj5vX8hDS1ahOQ2/DLv2A6c4+w=</latexit>

V n (S0, SL) =

n

Y

a=1

Y

⌅

TrJa

⌅

! 0 @ Y

⌅\output

TrSa

⌅

1 A Y

µ,⌅,a

e−βV (rµ

⌅,a)

rµ

⌅,a = Sµ ⌅,a N

X

i=1

1 √ N Ji

⌅,aSµ ⌅(i),a

<latexit sha1_base64="6OTW3KmDBK8uRZWaCB27VLpQTc=">AC/HichVHNahRBEK4Z/5L1J6teArksLpEIcemJgiIEgl6CB0mybhLIZIeTm9sdv7S3bMYm84D5AVy8KQoIsGrL5BLXsBDHkE8RsglgrWzA6JrtJrurv6qvqvu8MsEkoTcuS4585fuHhpZLRy+crVa2PV6zeWVZpLxlsjVK5GlLFI5HwlhY64quZ5DQOI74Sdp/04ys9LpVIk+d6O+PrMd1MREcwqhEKqnuybfw4t4Hxw4iyrtrKqeT1M7WmdEfJXHgRGznm2bZ9bvSMqMZ42vtqRGwPqav9SFMlMQrXk6VKNthLV/bTAl7mC8FlTrpEKqw07XunUobSFtPoBfNiAFBjkEAOHBDT6EVBQONbAwIZYutgEJPoiSLOwUIFuTlmcygiHZx3cTWokmeO7XVAWbYZcIp0RmDSbJF/KRHJNDsk+ktMza5miRl/LNu7hgMuzYGx3vHnyX1aMu4YXv1j/1KyhAw8LrQK1ZwXSvwUb8Huv9o6bj5YmzW3ylnxD/W/IETnAGyS97+z9Il96DRX8AO/P5x52lmca3r3GzOL9+tzj8itGYAJuwRS+9wOYg3lYgBb2/eHUnWnrvjvnP3U+DVNcpOTfhN3M/wQX68gW</latexit>

quenched random input/output

Sn[{ˆ q(l)}, { ˆ Q(l)}] = α−1

L

X

l=1

Sbond

ent [ ˆ

Q(l)] +

L−1

X

l=1

Sspin

ent [ˆ

q(l)] −

L

X

l=1

e

1 2

P

ab qab(l−1)Qab(l)qab(l)∂ha(l)∂hb(l)

n

Y

a=1

e−βV (ha(l))

• ha(l)=0

<latexit sha1_base64="W151Rme7a1IYaMmHG/mvld9r9w=">ADz3ichVHLbhMxFL1peJTh0RY2SGwsqlYJkMjTVipFqlTBhgWLpqEPqU5HnqnTjOp5dMYpKsaILfwAC1YgIYT4DYs2bDoJyCWRbBghtnShVlDsa+9zje46vbT+VYa4o3S8NlU+dPnN2+Jxz/sLFSyOjY5dX8qSbBWI5SGSrfk8FzKMxbIKlRraSZ45Eux6m/f62v7osD5P4odpLRSviW3HYDgOukPLGSuEkaXo6NutMa9bhiuyYiqwyc+swb/TzljPuEw7fEPXENY3o08Leds6EfGNLc0CyLiJ/Em8azUMQKPdFAW4OWc3NA0vMoRHkaxsdEtosWYXESdyNfZIQxZ5KwTp7yQNzQ0FkSG2wCUegXzvjgXaNnjL9Re6bHTtWZM2tNgpYPeSqLOWZCrn0dAepHmMGKFtkMuSTSywW8XGYVK0VZ3gjXmC8XJSuVQXzWEsCzc6ijy5Mh1nhpvdJzWqQ1yHLgFGIciFpPRd8BgExIoAsRCIhBIZbAIcdvHVygkCLXAo1chi06wIMOKjtYpXACo7sNo5bmK0XbIx5zO36gB3kfhnqCQwQb/Q9/SAfqIf6Ff659e2nr0etnD2e9rReqNvLja/PlfVYSzgs6R6sSeFbThtu01xN5Ty/ROEfT1u49fHjTvLE3oSfqGfsP+X9N9+hFPEO9+D942xNIrcOwDzNkgfTA7U4A59+8DrEzV3en6VGNmfOFu8RTDcA2uQwXvexYW4D4swjIEpc+lH6XfQ1BulB+Vn5af9UuHSoXmCgxE+fkfl7MPyA=</latexit>

N, M → ∞

α = M N

SLIDE 17

1st Glass transition

bond

continuous transition to full RSB glass phase at 1 st & (L-1) th layer

spin

• ther layers remain in the liquid phase

spin

αg ' 2.03

<latexit sha1_base64="Uj5ZNLsPCbzxQewKFMPvBgp/e3I=">ACfnichVFNL0NBFD2er6qvYkNsGg2xUbclIVYNG0tUkag07z2jJt6X916bVCP2/oCFUkjIuFH2PgDFv0JYkliY+H29SWC4E5m5syZe+6cmdEcQ3o+Ub1FaW1r7+iMdEW7e3r7+mMDgxueXJ1kdNtw3a3NUThrREzpe+IbYcV6imZohN7WCpsb9ZFq4nbWvdrzhix1SLltyTuozVYiN5FXD2VcL1bxrxovHeU+a4jCeTtJMIZagJAUR/wlSIUgjBU7doU8dmFDRwkmBCz4jA2o8LhtIwWCw9wOqsy5jGSwL3CMKGtLnCU4Q2X2gMcir7ZD1uJ1o6YXqHU+xeDusjKOcXqka3qhB7qhJ3r/tVY1qNHwUuFZa2qFU+g/Hc6+/asyefax/6n607OPcwHXiV7dwKmcQu9qS8fnb1kF9bGqxN0Sc/s/4LqdM83sMqvem1VrJ0jyh+Q+v7cP8FGOpmaSaZXZxOZxfArIhjFGCb5veQwTJWkONzT1DLe4UKBPKlDLdTFVaQs0QvoQy/wETcJ/</latexit>

bond

1 − q

<latexit sha1_base64="TboprELbJ9ahjMRilxn2XzKXFo=">ACaHichVFNLwNBGH6vujiwPiIhrioplViY+TcHUVpEgze6a1qb7ZXfaoPEHJK4cnEgkxM9w8Qc/AQcSVwcvLvdEGnwTjLzPO+zvPzGiuafiCsceY1NTc0trW3hHv7OruSci9fWu+U/F0ntcd0/E2NXnpmHzvDCEyTdcj6uWZvJ1rbwU5Ner3PMNx14VBy7ftSbRQNXRVE5ZTJvYKcZCkWxkgjUCKQRBQrjnyNLezAgY4KLHDYEIRNqPBpbEIBg0vcNmrEeYSM9xhDhpK1TFqUIltkxziXabEWvTPujph+p9qnBI6VCXI4xgjD2wG/bK7tkte2Ifv3arhV0CNwe0anUtdwuJ48Hc+78qi1aB3W/Vn64FipgN3Rrk3g2Z4B56XV89PHvNzWfHauPskr2Q/wv2yO7oBnb1Tb/K8Ow54uEXzAWR/nrwRrA2lVLSqXRmOrmwGH1GO4Yxigl68RksYBkryNO5JZzgFGexZ0mWBqSheqkUizT9+BHS6Ccx3Ysa</latexit>

qEA

<latexit sha1_base64="2f1uU3s+zy0a025anBjCukW5Hv8=">ACb3ichVHLSiNRED1pdYxRJ1EXCoI0E5RxEypG83DlA2GWvqIyKqG7vWpjv+zuBDXkB1wLkRBQRjxM9z4Ay78BJmVODAbF1Z3mhmGYca6cG/VqTp1T92rOobu+USPMamlte1De7wj0dnV/TGZ6uld8eyq4myZhu2u6YqnjB0S5R93TfEmuMKxVQNsaruzQb51ZpwPd2lv1DR2yayo6lb+ua4jP0db9S3BNeW6UmlKUOhyZTJFydKxTw7BRqjfEnORqk0Ipu3U9+wgS3Y0FCFCQELPvsGFHi81pEFwWFsE3XGXPb0MC/QIK5Va4SXKEwusf7DkfrEWpxHPT0QvYBV9jMtLlLAzKG6YFu6IXu6Zae6PWf3ephl0DNIZ9qkyucSvJ4YOnuyTx+7v1n/Ve1jG8VQrc7qnRAJ5tCa/NrR6cvS5OJwfYSu6Dvrv6RHuMJrNoP7XpBLJ4hEX5BKbDcrwf/21kZy2RzmdzCeHpqJvqMOAbxCZ/5xQuYwhfMo8z3WjBOS5iz1K/NCTJzVIpFnH68IdJo2+CI58</latexit>

QEA

<latexit sha1_base64="qB7x19pK/fXjA/+CbqC+c7boU=">ACb3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKRQXGV8cU5Sq4OtbGCygb6JkbWpqZWygY6BkbG1mYGAMZ5gaGZmbmCoZ6BmCgzAFAfkCKxhiGFIY8hmSGUoZchlSGfIYSoDsHIZEhmIgjGYwZDBgKACKxTJUA8WKgKxMsHwqQy0DF1BvKVBVKlBFIlA0G0imA3nRUNE8IB9kZjFYdwVQRT5QZz7QlFoGBQZVg6sGKw0+G5wWG3w0uAPTtOqwaAXFMJpJMgelML4vm7JIK/E9SVC6RLGDIQuvC6uoQhjcEC7NpMoOsLwCIgfyRD9JdVTf8cbBWkWq1msMjgNdD9Cw1uGhwG+iCv7Evy0sDUoNkMXOAosAQBY3iAYzLCjPQMjfWMA02UHZygkcHBIM2gxKABDHFzBgcGD4YAhlCgvXkMUxjmMsxj/MgkziTHpABRysQI1SPMgAKYNAFsg45y</latexit>

qEA

<latexit sha1_base64="2f1uU3s+zy0a025anBjCukW5Hv8=">ACb3ichVHLSiNRED1pdYxRJ1EXCoI0E5RxEypG83DlA2GWvqIyKqG7vWpjv+zuBDXkB1wLkRBQRjxM9z4Ay78BJmVODAbF1Z3mhmGYca6cG/VqTp1T92rOobu+USPMamlte1De7wj0dnV/TGZ6uld8eyq4myZhu2u6YqnjB0S5R93TfEmuMKxVQNsaruzQb51ZpwPd2lv1DR2yayo6lb+ua4jP0db9S3BNeW6UmlKUOhyZTJFydKxTw7BRqjfEnORqk0Ipu3U9+wgS3Y0FCFCQELPvsGFHi81pEFwWFsE3XGXPb0MC/QIK5Va4SXKEwusf7DkfrEWpxHPT0QvYBV9jMtLlLAzKG6YFu6IXu6Zae6PWf3ephl0DNIZ9qkyucSvJ4YOnuyTx+7v1n/Ve1jG8VQrc7qnRAJ5tCa/NrR6cvS5OJwfYSu6Dvrv6RHuMJrNoP7XpBLJ4hEX5BKbDcrwf/21kZy2RzmdzCeHpqJvqMOAbxCZ/5xQuYwhfMo8z3WjBOS5iz1K/NCTJzVIpFnH68IdJo2+CI58</latexit>

QEA

<latexit sha1_base64="qB7x19pK/fXjA/+CbqC+c7boU=">ACb3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKRQXGV8cU5Sq4OtbGCygb6JkbWpqZWygY6BkbG1mYGAMZ5gaGZmbmCoZ6BmCgzAFAfkCKxhiGFIY8hmSGUoZchlSGfIYSoDsHIZEhmIgjGYwZDBgKACKxTJUA8WKgKxMsHwqQy0DF1BvKVBVKlBFIlA0G0imA3nRUNE8IB9kZjFYdwVQRT5QZz7QlFoGBQZVg6sGKw0+G5wWG3w0uAPTtOqwaAXFMJpJMgelML4vm7JIK/E9SVC6RLGDIQuvC6uoQhjcEC7NpMoOsLwCIgfyRD9JdVTf8cbBWkWq1msMjgNdD9Cw1uGhwG+iCv7Evy0sDUoNkMXOAosAQBY3iAYzLCjPQMjfWMA02UHZygkcHBIM2gxKABDHFzBgcGD4YAhlCgvXkMUxjmMsxj/MgkziTHpABRysQI1SPMgAKYNAFsg45y</latexit>

SLIDE 18

2nd Glass transition at 2nd & (L-2) th layer continuous transition to full RSB glass phase which also induce 2nd glass transitions at 1st and L-th layer

bond spin

αg(2) ' 15.38

<latexit sha1_base64="64V74Zhkt2+ro2udTFp+qOwDKes=">AChHichVFNLwNBGH6sr6qv4iJx0GgIiTSzrY4iHBxbEuRqDS7a9TEftndNmHTo4s/4OBEIiK98gtc/AEHP0EcSVwcvN1uIgjeycw8z7vPMjGrwvUYe2yRWtvaOzojXdHunt6+/tjA4LprVRyNFzRLt5xNVXG5Lkxe8ISn803b4Yqh6nxD3V9u7G9UueMKy1zDm2+bShlU+wKTfGIKsVGi4pu7ykl3y86Rrxcq02mpoquMPhBXM4k03OlWIlWRDxn0AOQJhZK3YFYrYgQUNFRjgMOER1qHApbYFGQw2cdvwiXMIiWCfo4YoaSuUxSlDIXafxjKtkLWpHWjphuoNTpFp+6QMo5x9sCu2Qu7Z3X2xN5/reUHNRpeDmlWm1pul/pPhlf/lUZNHvY+1T96dnDLuYCr4K82wHTuIXW1FePTl9W5/Pj/gS7YM/k/5w9sju6gVl91S5zPH+GKH2A/P25f4L1VFJOJ1O5mcTiUvgVEYxgDJP03rNYxAqyKNC5x6jBrdShzQtpaVM1VqCTVD+BLSwgefZRy</latexit>

1 − q

<latexit sha1_base64="TboprELbJ9ahjMRilxn2XzKXFo=">ACaHichVFNLwNBGH6vujiwPiIhrioplViY+TcHUVpEgze6a1qb7ZXfaoPEHJK4cnEgkxM9w8Qc/AQcSVwcvLvdEGnwTjLzPO+zvPzGiuafiCsceY1NTc0trW3hHv7OruSci9fWu+U/F0ntcd0/E2NXnpmHzvDCEyTdcj6uWZvJ1rbwU5Ner3PMNx14VBy7ftSbRQNXRVE5ZTJvYKcZCkWxkgjUCKQRBQrjnyNLezAgY4KLHDYEIRNqPBpbEIBg0vcNmrEeYSM9xhDhpK1TFqUIltkxziXabEWvTPujph+p9qnBI6VCXI4xgjD2wG/bK7tkte2Ifv3arhV0CNwe0anUtdwuJ48Hc+78qi1aB3W/Vn64FipgN3Rrk3g2Z4B56XV89PHvNzWfHauPskr2Q/wv2yO7oBnb1Tb/K8Ow54uEXzAWR/nrwRrA2lVLSqXRmOrmwGH1GO4Yxigl68RksYBkryNO5JZzgFGexZ0mWBqSheqkUizT9+BHS6Ccx3Ysa</latexit>

SLIDE 19

Growth of glass phase under larger constrains

qEA

l l

spin bond

1/α = 0.02, 0.01, 0.005, 0.001, 0.0005, 0.00025

<latexit sha1_base64="jPL8GvWQ/jap1qiqxUGT7mfcpY=">ACknichVHLSsNAFD2Nr1ofrY+F4KZYFBdab6pFEQfGxcufFUFlZLEaQ2mSUjSghZ/wK0LF64URMQvcKsbf8CFnyAuFdy48CYNiIp6h7lz7pl7tyZUW1Ddz2ix4hUV9/Q2BRtjrW0trXHEx2da65VdjSR0yzDcjZUxRWGboqcp3uG2LAdoZRUQ6yre3P+/npFOK5umavevi2S0rR1Au6pnhM5RPD8siWYti7yhSlKTPETvYdZQNfw2FAmWw+kQoQW/InkEOQmiLVuISW9iBQ1lCBgwmNsQIHLYxMyCDZz26gy5zDSg32BQ8RYW+YswRkKs3vsixthqzJsV/TDdQan2LwdFiZRD890BW90D1d0xO9/1qrGtTwe9nVa1phZ2PH/WsvP2rKvHqYfdT9WfPHgqYCHrVuXc7YPxbaDV95eDkZWVyub86QOf0zP2f0SPd8Q3Myqt2sSWTxHjD5C/P/dPsJZJy6PpzNJYano2/IoetGHQX7vcUxjHovI8bnHuMEt7qRuaVKakeZqVIk1HThi0kLHzQilCI=</latexit>

0.2 0.4 0.6 0.8 1 5 10 15 20 0.2 0.4 0.6 0.8 1 2 4 6 8 10 12 14 16 18 20

ξ(α) ∝ ln α

<latexit sha1_base64="13Jt2AvwoBcVr58dGy40xBWyYrw=">AChHichVHLgRBFD3ae7wG4mFjgkhkUmNR4iFCBtLr0GiZdLdChU13ZXumgkmljZ+wMKRERs+QIbP2DhE8SxMbCnZ5OBMGtVNWpU/fcOlXlKClCzdhjlVFdU1tX39CYaGpuaW1LtneshH4hcHnW9aUfrDl2yKXweFYLfmaCridyRfdXZny/urR6EwveW9b7iG3l72xNbwrU1Ublkj7UnBixbqh170LRU4Cvtm5b0zApn5pIplmZRmD9BJgYpxDHvJy9hYRM+XBSQB4cHTVjCRkhtHRkwKOI2UCIuICSifY5DJEhboCxOGTaxuzRu02o9Zj1al2uGkdqlUyT1gJQm+tgDu2Iv7J5dsyf2/mutUlSj7GWfZqei5SrXdty19PavKk+zxs6n6k/PGluYiLwK8q4ipnwLt6IvHpy8LE0u9pX62Tl7Jv9n7JHd0Q284qt7scAXT5GgD8h8f+6fYGU4nRlJDy+MpqZn4q9oQDd6MUDvPY5pzGEeWTr3CNe4wa1RZwZI8ZYJdWoijWd+BLG1AeHR5Tq</latexit>

“penetration depth”

QEA

<latexit sha1_base64="RPD8LwSR4z5gJHAWTCHdiCZKVOM=">ACb3ichVHLSsNAFD2Nr1pfVRcKghRLRTdlWgXFVUEl21tiWksRg3mRpMVa+gOuBReioCAofoYbf8CFnyCuRMGNC2/SgKiod2DmzLn3DkzI5mqYjuMPQSEtvaOzq5gd6int69/IDw4tGEbVUvmOdlQDasgiTZXFZ3nHMVRecG0uKhJKs9Le8tuPl/jlq0Y+rpTN3lZE3d0ZVuRYeozUylUbK0yMpisxKOsjzIvITJHwQhR9pI3yFErZgQEYVGjh0OIRViLBpFJEAg0lcGQ3iLEKl+doIkTaKlVxqhCJ3aN5h3ZFn9Vp7/a0PfU+VRikNKhLExHE2D27Zi/sjt2wR/b+a7eG18V1U6dVam5WRk4HF17+1el0epg91P1p2sH25j3Crk3vQY9x5yS187OH5ZW8jGpPsgj2R/3P2wG7pBnrtVb7M8OwJQvQFie8P/hNsJOJmXgyMxtNLfmfEcQYJjBFLz6HFaRo7O1XGEU5wFnoURYVyItEqFgK8ZxpcQpj8Aq2GN6g=</latexit>

SLIDE 20

More “terraces” under larger constraints

α = 4000

<latexit sha1_base64="ghbOtJ0+kI+w09oJPANmemhYCs=">ACbnichVHLSsNAFD2Nr1ofrQoiFgsla7KbRUQSi6canVPqCKJHG0oWkSkrSgxR9wLy4ERUFE/Aw3/oCLfoK4ERTcuPA2DYiKesNkzpy586ZO4qla45L1AxIHZ1d3T3B3lBf/8BgODI0nHfMmq2KnGrqpl1UZEfomiFyrubqomjZQq4quigoleXWfqEubEczjQ13xJbVXnP0HY1VXaZKm3KulWF2eJaDsSoyR5Ef0JUj6IwY9VM3KNTezAhIoaqhAw4DLWIcPhr4QUCBZzW2gwZzPSvH2BQ4RYW+MswRkysxX+7/Gq5LMGr1s1HU+t8ik6D5uVUcTpgW7ohe7plh7p/daDa9Gy8s+z0pbK6zt8NHY+tu/qirPLsqfqj89u9jFvOdVY+Wx7Ruob19YOTl/WFbLwxTZf0xP4vqEl3fAOj/qperYnsKUL8AKnv7f4J8ulkaiaZXpuNZb8pwhiHFNIcL/nkMEKVpHzOnaM5wHnqVRaUKabKdKAV8zgi8hJT4AWBKNSw=</latexit>

spin bond

SLIDE 21

Summary: depth dependent free-energy landscape

energy landscape become simpler and flatter at deeper layers

SLIDE 22

Fluctuating boundary

“1RSB" boundary “full RSB" boundary

qi(0) = ( r mi < xinput 1 mi > xinput

SLIDE 23

“full RSB” boundary

spin bond

α = 4000

<latexit sha1_base64="xAqxpDAKjT5MIgYeF0EANMJjL8=">ACcHichVHLSsNAFD2N7/quim48FEUcVFuVFAEQXTjsrW2FaqUJE7bYJqEJC1q8QfciwtBURAUP8ONP+DCTxB3Krhx4W0aEC3qHZg5c+49lzN3VNvQXY/oMS1tLa1d3R2hbt7ev6IwODGdeqOJpIa5ZhOZuq4gpDN0Xa0z1DbNqOUMqIbLq7mo9n60Kx9Utc8Pbt8V2WSmaekHXFI+p3JZi2CVlaY6I8pEYxcmP0WYgByCGIBJW5Bpb2IEFDRWUIWDCY2xAgcsrBxkEm7lt1JhzGOl+XuAQYdZWuEpwhcLsLu9FvuUC1uR7vafrq/e4wmKlxV0OMYoJeqAbeqV7uqUn+vi1W83vUnez6fa0Ao738UTb3/qyrz6aH0pfrTtYcCFny3Oru3fab+Dq2hrx6cvKYW1ydqk3RJz+z/gh7pjl9gVt+0q6RYP0WYv0D+OfBmkJmJy7PxmeRcbHkl+IxODGMcUzxeSxjDQmk/Ykd4wznoRcpKo1IY41SKRohvAtpOlPiACNzg=</latexit>

α = 50

<latexit sha1_base64="FBq/8SR3yJLinzqr1B/wPcwm3ec=">ACbHichVG7SgNBFD1ZXzG+4qMQRBCDYhVuoqIgmhj6SsaSUR210lcsi92N4EY/AFbCwu1UBARP8PGH7DIJ4hgE8HGwpvNgmhQ7zAzZ87c+fMjGLrmusRVUNS2tbe0e4M9LV3dPbF+0f2HatoqOKlGrplpNWZFfomilSnubpIm07QjYUXewohZX6/k5JOK5mVte2RZ7hpw3tZymyh5Tu1lZtw/lxVnaj8YoTn6MNYNEAGIYs2K3iKLA1hQUYQBARMeYx0yXG4ZJECwmdtDhTmHkebvCxwjwtoiZwnOkJkt8JjnVSZgTV7Xa7q+WuVTdO4OK8cwQU90RzV6pHt6po9fa1X8GnUvZ6VhlbY+30nw5v/6oMnj0cfqn+9Owh3nfq8bebZ+p30Jt6EtHZ7XNhY2JyiRd0wv7v6IqPfANzNKberMuNs4R4Q9I/HzuZrCdjCem48n1mdjScvAVYxgHFP83nNYwirWkOJzDZziApehV2lIGpFG6lSKNAM4ltIk59I7IzY</latexit>

SLIDE 24

Teacher student setting

Jteacher

ij

Out put of teacher

Jstudent

ij

Student is forced to reproduce teacher’s output Randomly quenched 1) Training random training data random test data

Jteacher

ij

Jstudent

ij

Compare Randomly quenched Now quenched 2) Test a statistical inference problem (SIP)

SLIDE 25

teacher+student machine (inference)

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 2 4 6 8 10 12 14 16 18 20

0.2 0.4 0.6 0.8 1 5 10 15 20

RS solution

R, Q

<latexit sha1_base64="owAZR8mkXjRZkQgK8+U2zvTMw=">ACaXichVG7SgNBFD1Z3/GRqE3QJhgUCwkTFRQr0cbSRPMADbK7jmbIZnfZnQ1+AMWthGsFATFz7DxByzyCZIygo2FdzcLokG9AzNnzr3nzpkZzTaEKxlrhpSe3r7+gcGh8PDI6FgkOj6Rc62qo/OsbhmWU9BUlxvC5FkpMELtsPVimbwvFbe9PL5GndcYZm78tTmxYp6bIojoavSozIL8fRBNMGSzI94N0gFIEgtq3oPfZxCAs6qiAw4QkbECFS2MPKTDYxBVRJ84hJPw8xznCpK1SFacKldgyzce02wtYk/ZeT9dXn1CFRUqLupwjln2wh5Ymz2zR/bKPn7tVve7eG5OadU6Wm4fRC5iO+/qiq0SpS+VH+6ljCqu9WkHvbZ7x76B197azR3lnLzNbn2C1rkf8b1mRPdAOz9qbfpXnmGmH6gtTPB+8GucVkaim5mF5OrG8EnzGIacxgnl58BevYwjaydG4Jl2jgKtRSxpWYMtUpVUKBZhLfQkl8AoJEixQ=</latexit>

r, q

<latexit sha1_base64="3HEYnsHWRMFn2phEp/1T0/6t39M=">ACaXichVFNLwNBGH6vqo+2nIRLo0NcZBmtiTEqeHiqKVIENldg4n9srtVNM/4OBaiROJhPgZLv6Ag58gPVbi4uDd7SaC4J1k5pnfZ93npnRHEN4PmPMamjs6u7J96b6OsfGEym0kPrnl12dV7SbcN2NzXV4aweMkXvsE3HZerpmbwDe1oKchvVLjrCdta86sO3zHVA0vsC131A8qdzhzvpmSWZWFkfgIlAjKiWLFTt9jGHmzoKMEhwWfsAEVHo0tKGBwiNtBjTiXkAjzHUkSFumKk4VKrFHNB/QbitiLdoHPb1QfUIVNilt6lJHBhPsid2xFntk9+yFvf/arRZ2CdxUadXaWu7sJs9GVt/+VZm0+j8VP3p2sc+5kO3gtw7IRPcQ2/rK6eN1upCcaI2ya5Zk/xfsWf2QDewKq/6TYEXL5GgL1C+P/hPsJ7LKjPZXGFWzi9GnxHGMYxRS8+hzyWsYISnXuIczRwEWtKaWlEGm2XSrFIM4wvIckfAzOLVA=</latexit>

1+s replica student machine input

• utput

random teacher machine

r, R

<latexit sha1_base64="gAZkjdb1BwUuoMmDyRStNaIBVEw=">ACaXichVG7SgNBFD1Z3/GRBvRJhgUCwmTKChWQRtLjSYKGsLuOjFLNjvL7iQYgz9gYRvBSkFQ/Awbf8DCTxDLCDYW3t0siAb1DsycOfeO2dmNs0XMnYc0jp6e3rHxgcCg+PjI5ForHxvCtqjs5zujCFs6epLjcNi+ekIU2+ZztcrWom39Uq615+t84d1xDWjmzYvFBVjyjZOiq9ChnIZ4tRhMsyfyId4NUABIYlNEb3GAQwjoqKEKDguSsAkVLo19pMBgE1dAkziHkOHnOU4RJm2NqjhVqMRWaD6i3X7AWrT3erq+pgqBCkFdTlFHLPsid2xNntk9+yFfzarel38dw0aNU6Wm4XI2eT2+/q0SpS/VH+6lihxXdrkHvbZ7x76B19/aTV3l7Nzjbn2DV7Jf9X7Jk90A2s+pt+s8WzlwjTF6R+Png3yKeTqcVkemspkVkLPmMQ05jBPL34MjLYwCZydG4Z52jhIvSqxJRJZapTqoQCzQS+hZL4BMTmizU=</latexit>

• verlap

l l

α = 0.01

<latexit sha1_base64="eTerCyt7ZDpu/wP9SPjlIKC8w8w=">ACcHichVFNSwJRFD1OX2ZfVhuhR9iRAt5Y0ERBFGblmpgUrMTE8dmi9mRsnEP9A+WgRFQVD0M9r0B1r0E6JdBm1adB0HoqS6D9497x7LufdJ1ua6riMPQWEru6e3r5gf2hgcGh4JDw6lnXMiq3wjGJqpr0rSw7XVINnXNXV+K5lc0mXNb4jH2y07neq3HZU09h2axYv6FLJUIuqIrlE5fKSZpWlVRZn4l4SsmLqU4g+iAKP5Jm+AZ57MOEgp0cBhwCWuQ4NDKQSDRVwBdeJsQqp3z9FAiLQVquJUIRF7QHuJTjmfNejc6ul46kOqMElpUpcGphBj+yWNdkDu2P7OPXbnWvS8tNjbLc1nJrb+Q4svX+r0qn7KL8pfrTtYsilj23Krm3PKb1DqWtrx6dNrdW0rH6LtiL+T/kj2xe3qBUX1TrlM8fYQfYH4c+CdIJuIiwvxRGoxurbuf0YQE5jBHE18CWvYRBIZb2InOMdF4FWICJPCdLtUCPiacXwLYf4TfeaNyQ=</latexit>

α = 0.02

<latexit sha1_base64="m2pOZmsFsUYbMrWxUlOFapurQtg=">ACcHichVHLSsNAFD2Nr1pfVTcFz6KIi7KtAqKIhuXLbVPqAWSeK0huZFkhZr6Q+4FxeCoiAofoYbf8CFnyDurODGhTdpQLSod2DumTP3XM7ckUxVsR3GngJCV3dPb1+wPzQwODQ8Eh4dy9pG1ZJ5RjZUw8pLos1VRecZR3FUnjctLmqSynNSZdO9z9W4ZSuGvuPUTV7UxLKulBRZdIgq7IqeSCusRhL7IWjlLyY6gRxH0ThR9I32AX+zAgowoNHDocwipE2LQKiIPBJK6IBnEWIcW752giRNoqVXGqEImt0F6mU8FndTq7PW1PfUgVBikN6tLEFGbZI7tlLfbA7tgz+/i1W8Pr4rqpU5baWm7ujRxHt/VWmUHRx8qf507aCEFc+tQu5Nj3HfIbf1taPT1vZqerYx67YC/m/ZE/snl6g197k6xRPnyFEXxD/OfBOkE3E4ouxRGopur7hf0YQE5jBPE18GevYQhIZb2InOMdF4FWICJPCdLtUCPiacXwLYeETf+iNyg=</latexit>

Bayes optimal, Nishimori condition Review: Zdeborova-Krzakala (2017) Symmetry breaking field: remanent bias in the liquid phase

O(ln(N)/N)

<latexit sha1_base64="fa2RXETstgtUqCDhusVSqSei5LY=">ACm3ichVHBShtRFD2Otmq0NepGKEIwWCKU8cZEYlwFsylF1MRGhSEmfEZByczw8wkqEN+wB8Q6spCocWfKHTu/CTyguFdx0TuTRBFpex+8d9+59x3nuqbeiuR3TVJ/UPHs+ODQcGRl98XIsOj6x5VpNRxMlzTIsZ0dVXGHopih5umeIHdsRSkM1xLZ6kA/q2y3huLplveObFtKHVT39M1xWOoFk37lbBJ2amrVT8pUxhvSF7MLCVpkZMO0l5PVAwzsTY3vzbXrkXjPWaM5AxRNp2N9ZixXimObmxY0c+oYBcWNDTRgIAJj3MDClweZSRBsBmrwmfM4UwP6wJtRFjbZJZghsLoAc913pW7qMn7oKcbqg+ZYbHS4i5txDBLP+kL3dB3uqBf9Puv3fywS+DmiFe1oxV2bexkavPuv6oGrx72H1T/dO1hD0uhW53d2yES3EPr6FvHpzeby8VZ/zV9pGv2f05X9I1vYLZutU8FUTxDhL+A5GwQqfsHf5psLcjJlJwqpO5le5nDOEVZpDgF8gh7fYQInP/YCv+IFLaVrKS+k1Q5V6utqJvEopNIfg96Zwg=</latexit>

SLIDE 26

Simulations of learning

SLIDE 27

random inputs/outputs

e−V (h) = e−✏h2θ(h) β = 1 kBT

<latexit sha1_base64="Pm1bLs+NvJhfEezLaXpbhuTJH3k=">ACunichVFNSxtBGH5cq9X4Fe1F6GUxROzBMBsLlIhpJce/UoUXA2z68QM2S93JwG7B/oH/DgqYVSn9Cj734Bzx49KgeI/TSg282C9JK23eYmXeZ53npmxAkdGirHLIW34ycjo07Hx3MTk1PRMfnauHvmd0BY123f8cNfikXCkJ2pKkfsBqHgruWIHav9tr+/0xVhJH1vW50EYt/lR5sSpsrghr5mjiIl01LK7Xl1ovEn3tATBFEnH9/TWQVxOElO1CWSbh4fd/ihnrLWzGbI7dhI4nYjNkNXrybSNfYCWhv4MbKkgCzW/fwXmDiEDxsduBDwoCh3wBFR24MBhoCwfcSEhZTJdF8gQY60HWIJYnBC2zQe0WovQz1a92tGqdqmUxzqISl1FNkF+8p67Jx9Y9fs19rxWmNvpcTmq2BVgSNmQ/zWz/q3JpVmg9qP7pWaGJV6lXSd6DFOnfwh7ou+9Pe1uvN4vxIvEbsn/R3bJftANvO6d/XlDbJ4hRx9g/Pncj5N6uWSslMobLwuVavYVY3iOBSzRe6+igndYR43O/Y4r3OBWe6NZmtTaA6o2lGme4bfQ1D0pJKq0</latexit>

soft-core potential Monte Carlo simulation

H = − X

⌅ M

X

µ=1

V

• rµ

⌅

• <latexit sha1_base64="vsSqFWBU3tPq4vKkCsQnjc58XU=">AlkXic3VrNbxtFB+wXz0S0hIXCxCUCuVyCkgClKkJo2ThqaNGydp2rqNdp31h7K2N7u2a8eEP4B/gAMnkDgzvwFXPgHOCucEAci8SFA+9mfXO164tBKXUT/bOvm9N+/NvHk7M2s38JtRt1D48Ykn3r6mWefe/6F3IsvfzKiZOnTu9EnV5Y9barHb8T7rpO5PnNtrfdbXZ9bzcIPafl+t5N9+Ay1t/se2HU7LS3usPAu9ty6u1mrVl1usDaO7lxZeHtStRr7Y0qru9UD6LDnhN6x3nBa/UW5o/vja4d7+Qrvlfrng3vIfNYg1fCZr3RPZfL5fZOzhTmCvTJjwvzemGiU+pc+r0q6zC9lmHVmPtZjH2qwLZ85LAK6w+ZgQXAu8tGwAuh1KR6jx2zHMj2AOUBwgHuAfzW4e6O4LbhHnVGJF2FVnz4hiCZ7OFHwpfFx4Uvi98U/i18GeqrhHpQFuGcHW5rBfsnfj0tfIfE6VacO2yRiKVIRHBnUOehfD1J/jXZTV2kfxqgp8BcdDjKm+rf/TZg/KHm7OjtwpfFn4DX78o/Fj4Drxt93+vfnXD2/w8w5LY6hBqa/BFjdP3ThqyDvoc4DZoBAfsvDYiaXIuUCsT0QK+T5ERgrb7GT03nbpvUcPWtIop+HQtgN2pFgW83y4utQ3IfQixkGD+gnj9jxuVd4F0ApAjzKdmC0cezr1JMYOdUMCyKQaYk2Qphx6UjePzk2m6GrQRbhLHUpVqOxRCBQ2N6QonkEPESirYhC3KNKf8e2PNAyK7IVtsauw/c2fDeohHz8XaH7LUIiPTzbHl6/YVzXKCa7UGpRuhCjCxQGWPqA/A8yXH7hMJYiShzBZDpH6fYyBtUhli4xpYgFtYpKnbYItuko3+Tdsedr9hbPiU8Sow4gv069OTwKHcGte5VOfSnVpTp5o6PW1aWt0+1e1THkI5uTWP6jC6MGv6FJ+y7BHVH1labAhJnX9A/APK0IHWmk91PlnpgkWqZItq8Vklc9vEbWvcAXEHYLPMDYgbAFduNSIu+lc3+qZLdeiDqr9G/D49D/gTW2nYdQntVWqxI3p0hFwr5I0zckfi91FJCbtGf3ZJUQq+MIkFteprlcQTIclvCF7zy0Uzq1qlmPW8dql2V/OlRNyS5ntZeFC2r4iZPR+LokeKxk9dov420qPqXHtST2WHt8hoUJ6Xqtx0RCREfeKXItriwW6tiFS8Dkut90RkdyhtXVdjAXOvxrJ4BMfV6VzUJoDvce01ojrUQLrsa4+rs8pOkpAiKmIp0Ue7lUtRSAVUdQ14BUxDVC5CXMDpCK2RHWJjLQCrmstbSCpCKWDFaugKkYq5oWm4CqYibhi1bQCpmS9NyHUhFXDdsWQJSMUualjUgFbFm2LIJpGI2NS3rQCpi3bDlIyAV85GB2QBSMRuGNctAKmZs2YRSEUsaohVIBWxathSBlIxZcOWq0Aq5qrW0jaQitg2WroBpGJuGJhdIBWza1hzC0jF3NKsuQ2kIm5b5uSmNeYTVo02h5pOa9FTktBrdFegqNUfQ0gjnThPk+7J9W7RWnMOGbRwBwCyYhDw/8lKd45Zsnaj7sKZtfAlKSsxDElA4MrAhkRGNb0gWRE39AxAJIRAwMxpF1ighgaiCMgGXFk2OEQyRhHiwqXSEa4GqKRMoZqHlTEGYe6QHJiJ4l9nB3P6Jrm3gh7ZnbFIGy3iaEGvrQN/KyJHQrEogokZ43IM4HckeLqdHWGDrle3NaS9TV94hE9nPspqtuK6ltmeNFdQoxm/EWGjKZAuPc2TdiviTCKnWRjRnKmMTy34icjI2nZgRZpxHUF7NmRoQR5SL1foDMnfro1pj3TyAZW2XAq2dAq27NiexZk34rsW5ANK7JhQ6syIEFObQihxZk3YqsW5HT9Fvd2m9NK9bMAgM60zJbQb6Z0W06lwzcjhVnrvA64xlh5gwerzaJeoaE2Yud8QyxS5jzpDOefXYJ2xctXq8akEWrciJfdiPsMTmpBObeNzxk2D98LipYq7TIQ5YEG+6kGJ+4B7iZckBiArgpZ0aFIf5Pq90FTJE4Oh9JpYp7aPhb5EM+T0DLMCyrmAmDeJG+yLEnzD/fLaAtvayT0TdI2jV/xc+bx8Sjd4jSv/t8ePtreHk8xe7lsMoe5FlylV8Q7q/jp1AfJnrAYTwrwdKMBevKwc79IPdUFyTy1F1AW4DXvkQ0B2Rb3ynlpRP7rM8ms08rJmHhs1iFj4p68CLuhIsP9Il4vw94e74qEWIR6fC+Ad3jNU3lTSG5I/3P2Pbf9Zscd12IM75ejSMmeY+lrsB92ofw06WaOEV16K2Wb6zvQ4FNtHJegtb3rPyUQp7R8ruJ+MxClimLPX6TLzjV/d1XGYH7qtQMk8FOhrC9lwvj2cwSiLvSGEfc+KOfQavIOzKW7GT3HM1I+o7f2lByZ3aZHqyB8G3+X1gmHMs/YWdFhqmN89A54XWaHldoist18n1E3B6tI3iGivdp6fYkejzBtemYbEs7xZq2iFhH2l9nW9O2KNrybLnPq39ealJcdMgb7Lbvk8IWSpZQWe3t0WrTnXvuyWtRNPkWtRL/PxHlZrJkVxlU7obGd2k6K/RucAVSkecS4FNJcyjzh+J8f8vo0kuLUPD8satpHY2tklKqR78aXNZS+Q+Ko/YmeuXR64Sl+xX2b/FNiJOESb7L08hOJSVpjlKozpGi26+WZLNGbnOEkGrLsatK74rKIn6HOeljkU/vQRmz08dsgZ/d752cMf5LZBZ2LszNvzN34ca7M5eWxP+MnmevszcgW82z9kleDqX2DbY8y37if3Mfpk9M/vB7KVZgX3yCSFzhimf2at/AYvumGc=</latexit>

Hamiltonian Gap with random boundaries Dynamical variables

Ji

⌅, Sµ ⌅

(i = 1, 2, . . . , N)(µ = 1, 2, . . . , M) (⌅ = 1, . . . , LN)

(i = 1, 2, . . . , N)

S0,i, SL,i

S0,1

<latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit><latexit sha1_base64="BubXqNOViuEiOYVJKEj5exHJo=">ACcnichVHLSsNAFD2N7/quhHdVEvFhciNCIor0Y1LtfYBbSlJnGowTWKSFjX0B/yBLtxYQVD8Df+gAs/QVwquHhbRoQLeodmDlz5p47Z+aqtqG7HtFTROrq7unt6x+IDg4Nj4zGxsYzrlV1NJHWLMNycqriCkM3RdrTPUPkbEcoFdUQWfVos3WerQnH1S1zu1RbGiHJh6WdcUj6miX1DL8VS95NOCXC/FErRIQcQ7gRyCBMLYtmI3KGAfFjRUYGACY+xAQUujzxkEGzmivCZcxjpwblAHVHWVjlLcIbC7BHPB7zLh6zJ+1ZN1CfcIbFSour1BFHkh7pl7pge7omT5+reYHVpuTnlV21phl0bPJ1Pv/6oqvHo4/FL96dpDGauBW53d2wHTeofW1tfOGq+ptd2kP0dX9ML+m/RE9/wCs/amXe+I3QtEuQXyzw/vBJmlRZnxznJifSNsRj+mMYt5/vEVrGML20jzvcdo4BLNyJs0Jc1IYekSKiZwLeQFj4B/C+O7g=</latexit>

S0,2

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S0,3

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S1,1

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S1,2

<latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit><latexit sha1_base64="(nul)">(nul)</latexit>

S2,1 S3,1 S0,N S1,N

rµ

⌅ = N

X

i=1

1 √ N Ji

⌅Sµ ⌅(i)Sµ ⌅

<latexit sha1_base64="I3kOm1j0xPHvbkwIKL9Rj68l4U=">AC8nichVHNThRBEK4Z/MH1hwUvEi8bNhi8bHrQBGNCQvRCPBgXSBh2ElP04udnT+6ezZip1/AxLMJnoAY/QtvPgCHngE43FJuHiwdnYSI6BWp7urv6rvq+ruMIuE0oQcO+7IpctXro5eq1y/cfPWHV8Yk2luWS8xdIolRshVTwSCW9poSO+kUlO4zDi62H36SC+3uNSiTR5rvcyvhXTnUR0BKMaoaD6RraNH+c2MH4YUdZVuzmV3M7XfJXHgRHznm2bJet3JGXGs8ZXu1IjYH3NX+qivpF825pnZxTaRljbvEh8Rty3tQsjNqjWSYMUVjveKVTh9KW0+oH8GEbUmCQwcEtDoR0B4dgEDwhkiG2BQUyiJ4o4BwsV5OaYxTGDItrFdQdPmyWa4HmgqQo2wyoRTonMGkyTb+Qj6ZOv5BP5Tn7+VcsUGoNe9nAPh1yeBWOv7zRP/8uKcdfw4jfrnz1r6MCjoleBvWcFMrgFG/J7r972m49Xp809ckh+YP8H5Jh8wRskvRP2foWvoMKfoB39rnPO2uzDe9BY3blYX3hSfkVo3AXpmAG3sOFmARlqGFdfvOpDPl1F3t7rsH7tEw1XVKzm34w9zPvwCO18S8</latexit>

SLIDE 28

t t

Relaxation of autocorrelation functions Cbond(t, ⌅) = 1 N

N

X

i=1

hJi

⌅(0)Ji ⌅(t)i

<latexit sha1_base64="c/nVoVvOj8RyKmvCdzJ4IimyBQ=">Alunic3VpLbyPHEW45dmzTj107MGDAF8Kygl1AEabBHEMC7AelKy1dpcrSlrtLtfCDV8QEPOaIbkqLpH+A/kENOCZCD4Z+RS/6AD74HuToAL74kKrqHk6/ZkgEydoxC+T0VH9VXdVdXdPdQy8KOknfcb5Zeu4Xz7/wyxdfern0yquvX7t+htvniThIG74x40wCONTz038oNPzj/udfuCfRrHvdr3Af+BdbGP9g6EfJ52wd9QfR/6TrtvqdZqdhtsH1tn10fbZpB53y17YO5/e6K/WvcBtXCSXAzf2b27Um7HbmKxPJ3en5Xoy6J5NOhvr0/hth64vVbgl2+DuCQCdZ3pDemld2/Wa7HJFYqlc6uLztrDn3Ks8K6Xlhm4lMN3jzLVZn5yxkDTZgXeazHutDOWAuS4Aes3XmsAh4T9gEeDGUOlTvsykrgewAUD4gXOBewG8L7h4Lbg/uUWdC0g1oJYBvDJltuJ87XzpfOf83fnK+YfzQ6uCelAW8Zw9bisH51d+Lt2vdzpbpw7bN2JlUgkcCdS57F8A3m+NdnTfY+dUBPyPioMcN3tbw6o/f1T4XJn82vmL80/w9c/ON87fwNve8F+Nv973D/9UYElqdQy1TfixsV7Jw/ZAn0ucNs0giO2qo1InpwH1C1EdIEfUGTEoO1pQc8tpm1x79GDrjTKeTi07YJdKZalvACuHvVNDL2IcdCmfsK4XSUu9wrvIiglgEfZEYbx75FPYmR0yiwIAGZrmgjhmXj+T9U2IrBbraZBHOUo9iNZlJRAKF7Y0pmifAQyTaijE/VTpP7GtDLTDKmyX7bO78H0E3tUQj7+7tL9ESGRnp1tz67fMK6bFJN9KHUpI/QhRjaojDH1B/A8y3HnhMJYShzRZDpf06xUTaoBrFwh21BLBxQVJywTXZIJRv9L2171v2GsRFQxqvDiG/Qb0BPApdya1rnUZ1Hd2pNi2pa9LTpanXnVHdOeQjl5NZ8qsPowqwZUHzKsldUf2VpsS0kdf4F8S8oQ0dawHVBWSlBxapkl2qxWeVzO0Rt6dxR8Qdgc0yNyJuBFy51YS46F/L6Js+1aEPqv4m8Yf0POBPZLWdtlGf1TaorkHckiKVCPsSTd+Y+IPcUBu1p7dkz1C7M0iQG5h+p2ZhEgyx0JX/DKRzOrO6Cag5zxOqXaU82XKnGrmu814UHNavukNH7uSp6rGr02EPiHys9psa1L/VYfnzHhIrpea3GRVtERtorci2uLTbo2oNIwe43HYoIjmktXVLjAXOvybJ4BMfV6VrUFoDvVNa6T1KIH1WNea1ZcUHVUgxNTF06IM96qWCpCKqGiIO0Aq4g4hyhLmBEjFnAhrMsw2kIrZ1lraBVIRu0ZLHwOpmI81LQ+AVMQDw5YjIBVzpGm5C6Qi7hq2bAGpmC1Nyz6Qitg3bDkEUjGHmpYDIBVxYNhyG0jF3DYw94BUzD3Dmh0gFbOjWbMJpCI2NcQekIrYM2ypAamYmHLJ0Aq5hOtpWMgFXFstHQfSMXcNzCnQCrm1LDmIZCKeahZ8whIRTyzMlDMa8xm3RptH3SsqpFTldB7dNegqNUfW0gjvTgvky7J9W7TWnMOGbTwFwCyYhLw/8tKd45Zsvaj6cK5tTAVKWsxDFVA4MrAhkRGdYMgWTE0NAxApIRIwMxpl1ihgbiCsgGXFl2OESyRhXiwqPSEZ4GqKdM4ZqHtlXEGYeGQDJiIEl9nB3P6Frj3gx7Zl7FIGy3iakGoLoW9l5ERoViUQ0SQ87kFG8DsRPN3OUNig69VtjWlvMxQe4dOZj7KarbiuHXYmzRXUaMZvQthkAaRHT/Os3bo4kyhpFiY0Z+qzUwt+IjKxth1ZkWZcJ9CeDRlbkJfUy3U6wiIn2+Nac8ispFVNl5INrbKDqzYgQU5tCKHFmTbimxbkCMrcmRBjq3IsQXZsiJbVuQi/day9lvHijWzwIjOtMxWkG9mdJvOLQN3YsWZK7xwNiPMnMHj1SbRKpAwezGczRC7hDlPwtns0vY5uCe1eM9C7JiRVYsuRfzGZ7QxHRqm547LrN1+N5StDRol4EoHzTYTzU4cQ9wN+GBxAh01cmKkCL9Pao/B02JODkcS6eJZWp7KvIhniehZgXVMwtwLxH3hRZkucf7pfRFt7WROibp20Rv9LnzM/Ho3yL87z6/bwp+3tdIHZy2WzOcy14Cq9Lt5ZpU+nIUgOhMV4UoCnG23QU4ad+/vU32QLFN7EWUBXvM7siEi29JeWZVG5Mc+kyw6rZyPScfmADIm7skrsBuqMNwv4nUb9vZ4VyHEJtTjewG8w2uZyodC8p40v8d2368fpPjrg9xteracRk7HUFXhA+xB+utQUp6guvdUKjPV9LCZVs7L0PqelZ9SyDNafjeRnlnIMjWx8+XSXf86r6Oy5zAfQNK5qlAqCFsz/XabAajJPKeSm8Yce+Dci6tJh/DXHpS0HM8I5ULeutMyZHFbfq0CsK38U9onXAJs/xzdkNkmOYsD90UXufp8YSmtNwi3yfEHdA6gmeodJ+Wb0+mxdcm475tvRyrOmJiHWl/XWxNT2LPbqWInue0tqflzoUN23yprjtp4SQpbIVdHF7R7TqVPe+R9JKNE+uS73Ez39UablmXhQ36ITOdmY3L/qbdA7QkOIR51JEc8mlzBP/vkhr08TKU7N8OKpn0ys0ZGqRr5bnxHQ+k7JI46n+uZR6cXvuJX2rfZPyUmEi7zpkgvP5GYpzVFqTpjima7Xp7JMr3ZGU6mociuDr0roko4neYkz4T+fRTKGN2+oxt8LP7s+vLxn+JzMLJrbX136zduv/b5Y+2xP+MXmLvsHchW62z37OP4OlcZcdgz7dLzy+9tvT6yocr3kpn5YJDn1sSMr9iymel/2+LVqXS</latexit>

Cspin(t, ⌅) = 1 M

M

X

µ=1

hSµ

⌅(0)Sµ ⌅(t)i

<latexit sha1_base64="CbseTsj5vdYGyZHi0M2r6lTo5Y=">AlwHic3VrNbiPHEW45ieMwib2EyBALoRlBbuAIw2CeIEGD9ULJs7S5XlLTaXa6FGWr4A81wRjMklxRNP4BfwAefEiCHI+RS14gB9zCXJ0gFx8cFV1D6f/ZkgEydqxCuTMVH1VXdVdXdPdlBcHvXTgOJ+vPSd737v5e+/8oPKD3/041dfu/X6G2dpNExa/mkrCqLk3HNTP+j1/dNBbxD453Hiu6EX+I+8q12UPxr5SdqL+ieDSew/C91Ov9futdwBsC5ufbx7MW0mYTWNe/3Z7cF60wvc1lV6PXQT/85Ws524renmbHpvVm2mwxCw4XBrc/YhMJqB2+8EfrUBTEkJZICZ3XbuFAgGd6rNhFQrlcrFrVnw6G/6vxmU79ZeKvHr3+xk9Zk12yiLXYkIXMZ302gPuAuSwFeso2mcNi4D1jU+AlcNcjuc9mrAK6Q0D5gHCBewXfHXh6Krh9eEabKWm3oJUAPgloVtma8zfnT84Xzl+dPzv/cL4stDUlG+jLBK4e1/Xji9c+Vnj3wu1QrgOWDfXKtFI4cmlyBL4BAviG7A2e4fi6kGcMXEw4hZva3Tz6ReN3x2vTX/h/MH5J8T6e+dz5y8QbX/0r9YfH/rHn5V4knmdgLQNH7S4fO8UITtgzwVul0ZwzNa1ESnS84DCUkQI/IAyIwFrz0t6bjlry0ePEYTSKBfh0LcrdqN4lvECuHrUNwn0IuZBl/oJ83aduDwqfIrhLgU86kYw2j2HepJzJxWiQcp6ISijQRmXDGS90+FrZXY6pJHOEs9ytV0rhELFLY3oWyeAg+R6CuiEPdNpf/EtyrQHquxfXbI7sPnCXwe0B3y8Xufnk8IifTifHtx/YZ53acHMBdSBVhADmyRfeYU7+FyPMad0kozJWUKlcMlf7blBtVgxqQC/fYDuTCEWXFGdtmx3Rno/+lby+63zA3Aqp4TRjxLfoO6E3gUm3NZB7JPHpSJR2SdOhtE2qyS5JdUh1CPbk1n2SYXVg1A8pPWfeG5DeWFrtCU+dfEf+KnSstRaQLCAvPfBI1QxJiu8qmdsnbl/jok7Bp9lbkzcGLhyqylxMb6O0TcDkmEMqv028Uf0PuBvZLWdriHPpS2StYhbUbRS4V+q2ZsQf1g4CsjN27NHckCIg3kGyC3vkWxvngGy3omIBa98NHPZEUmOCsbrnKTnWix14ta12BsigobV932ho/dzXfRY3eix8Q/VXpMzWtf6rHi/E4IldD7Ws2LrsiMrFdkKa4tujah0zB97jcdiQyOaK1dUeMBc6/NungGx9XpRtwtwF2Z7TWyOSogXKUdebyimKjDoSYpnhbVOFZtVIDUhE1DXEPSEXcI0RVwpwBqZgz4U2O2QVSMbtaS/tAKmLfaOk9IBXznmblEZCKeGT4cgKkYk40K/eBVMR9w5cdIBWzo1k5BFIRh4Yvx0Aq5lizcgSkIo4MX94HUjHvG5gHQCrmgeHNHpCK2dO82QZSEdsa4gBIRwYvjSAVEzD8OUDIBXzgdbSKZCKODVaegikYh4amHMgFXNuePMYSMU81rx5AqQinljm5LGY1hNQhptn6ysa5kTKqhD2ktwlGqvC8SRHjxXafekRrctjRnHbBuYayAZcW3EvyPlO8fsWPvxXMGcG5i6VJU4pm5gcEUgI2LDmxGQjBgZNsZAMmJsICa0S8wREwNxAyQjbgw/XCIZ42pZ4RHJCE9DdAvGUK0jhwrCrCNDIBkxtOQe7u6ndO0TL6E9c58yUI4bTxMyaxH0rYycCsuqBiLahMc9yBi+p4Kn+xkJH3S7uq8J7W1GIiJ8O/NRVqsVt7XHLqS5ghbN/E0Jmy6B9OhtnrfbFGcSFc3DlOZMc35qwU9Epta2YyvSzOsU2rMhEwvymnq5SWcYAfGLvTH9WUY3tuomS+kmVt2hFTu0IEdW5MiC7FqRXQtybEWOLciJFTmxIDtWZMeKXKbfOtZ+61mxZhUY05mW2QryzYpus7lj4M6sOHOF81nhFkzeL7aNDolGmYvRvMZYtcw50k0n312DdscPLBGfGB1qzImqX2Yj3DE5qETm2zc8dVtgmfu4qVFu0yEOWDBfupBiceAe4mPNAYg60meRFRpr9N8kuwlIqTw4l0mliltmeiHuJ5EnqGdUHF3AXM2xRNmSdF8eF+GX3hbU2FvUXWlokre898eyIq9rgoqv/vCL/Z0c6WmL1cN5/D3Aqu0pviN6vs7TQCzaHwGE8K8HSjC3aqsHN/h3pqAJpVai+mKsAlvyYfYvIt65V1aUS+7jPJstPKxZhsbI6gYuKevAa7oRrD/SJed2Fvj081QmyDH8XwCe8Vun+WGg+kEb6v+Pb19dvct4NIM/4ejXLmPx3LHUFHtA+hJ8utcUpqku/agXG+j4R2Nwq5+Vofc/KTynkGS3/NpGdWcg6DbHL9bJdvzqvo7rnMFzC+7MU4FIQ9je6435DEZN5D2XfmHEvQ/qubSafApz6VlJz/GKVC3prQulRpa36dMqCH+Nf0brhGuY5R+z26LCtOd16I6IusiOJyxl9x2KfUrcIa0jeIXK9mnF/uR2fMG12VjsS7/Am7IWFfaX5d707f4o1sp8+c5rf35XY/ypkvRlLf9nBCyVr6CLm/vhFad6t73RFqJFumF1Ev8/EfVliWLsrhFJ3S2M7tF2d+mc4CWlI84l2KaSy5VnmT+nx/y+jSV8tQ8P6xp1qdzb2SUapHvxvc0lL5D4qjLhZF5dHrhK3FlfZv/p8RUwuXRlNnlJxKLrGYo1WZC2Wy3ytZbjc/w8ktlPnVo9+KGyKL+BPWpI9EPf0Q7rE6fcS2+Nn9xa1V43+JzJuzuxubv9y4+/BXq+/uiP8zeoX9nL0F1WqT/Ya9C2/nOjsFf/6+Ul5c+Unaztr3bVo7ZpDX1oROm8y5W/t5isZKi0</latexit>

Cspin(t, l)

<latexit sha1_base64="fOqUfVnJWnBzBdHVnHlg0xYVdW0=">AlVnic3VpfbxtFEN8CpcX8aVpAQuLFIg1qpShyCoiCVKn546Rp09aNkzRtXSKfc/6jnO3rne3acMH4AvwAsg8VD4GLzwBZDoR0A8FokXhJiZ3fPtvztbCNpSj+zbm/3N7Mzu7Nzunh3fa4SdXO7RkRdefOnoy8eOv5J59bX3zgxdfLUdtjuBhV3q9L2sGOUw5dr9FytzqNjufu+IFbjqe9PZX8L6mz03CBvt1mZn4Lt3m+Vaq1FtVModYO1OvbW0OywFzWzoN1qHZzqz3tlMJrM7NZ2by9EnOyrM64VpJj6F9slTb7MS2NtVmFd1mQua7EOlD1WZiHQHTbPcswH3l02BF4ApQbVu+yQZUC2CygXEGXg7sNvDe7uCG4L7lFnSNIVaMWDbwCSWTaT+zn3MPc491Puh9yvuT8TdQ1JB9oygKvDZV1/98QX7xT/GCvVhGuH1WOpFIkQ7srkWQBfb4x/HVZl58mvBvjpEwc9rvC2egdfPi5+ujEzfD/3be438PWb3KPcj+Btq/d75bsb7sZXKZEVgdQW4Uvapy8d5KQNdBXBm6dRrDPZrURSZJzgJqpiCbwPYqMALTdT+m5ybRN7j160JRGOQmHtu2zA8WyiOfB1aG+CaAXMQ7q1E8Yt7PE5V7hnQ+lEPAo24bRxrGvU9i5FRSLAhBpinaCGDGJSN5/2TYTIquOlmEs9ShWA1HEr5AYXsDiuYh8BCJtiIKc8q/RPbskDLM9W2Bq7Bt/b8L1OJeTj7wrdbxIS6cnZ9uT6DeO6SjHZgVKTMkIHYuQClTGmPgHP4xy3RyiMlZAylw+Z/nmKjaxBRYiFq2wRYmGdomKbLbANKtnov7TtSfcbxoZHGa8EI36Bfj16EpQpt0Z1DtU5dKfW1KimRk+bpla3R3V7lIdQTm7NpTqMLsyaHsWnLHtA9QeWFutCUufvE3+fMrSvteZRnUdWOmCRKtmkWnxWydwWcVsat0/cPtgsc3i+sCVWw2Ji/7VjL7pUB36oOqvEr9HzwP+RFbqRv1cW2F6irEzShSobAv1PQNiN9NHAXkxu3ZPVklxOoAuSWl6lueRQBstym8AWvfDTjunWqWU8Yrx2q3dF8KRC3oPleFB4UrbavCBm9nwuixwpGj90i/pbSY2pcu1KPJcd3QKiAntdqXNRFZES9Itfi2uICXVsQKfgcl9tui0hu09q6JsYC51+VZPCJj6vSOSjNgd5DWmtE9SiB9VhXG9VnFB0FIMSUxNMiC/eqljyQishriKtAKuIqIbISZhtIxWwLa2LMEpCKWdJaWgFSEStGS5eAVMwlTctNIBVx07BlE0jFbGpargGpiGuGLYtAKmZR07IGpCLWDFs2gFTMhqZlHUhFrBu2XAZSMZcNzHUgFXPdsGYZSMUsa9YsAKmIBQ2xCqQiVg1bikAqpmjYcgVIxVzRWtoCUhFbRks3gFTMDQOzA6RidgxrbgGpmFuaNbeBVMRty5zcEPMas0mTRtslLbNa5DQV1BrtJThK1VcH4kgH7rO0e1K9W5DGjGMWDMw9IBlxz/B/UYp3jlm09uOgtkxMAUpK3FMwcDgikBG+IY1PSAZ0TN09IFkRN9ADGiXGCMGBuIASEYcGHaUiWRMWYsKh0hGOBqinjCGah5ZUxBmHukCyYiuJfZwdz+ka4t4Ae2ZWxSBst94mhBpa0Pfysih0KxKIKJKeNyD9OF3KHi6nW1hg65XtzWgvU1PeIRPZz7KarbiupbZrjRXUKMZvyFhwmQDj3N43ZL4kwio1kY0pwpjU4t+InI0Nq2b0WacR1CezZkYEHeo14u0RmGR/xka0x7JpH1rbLBRLKBVbZrxXYtyJ4V2bMg61Zk3YLsW5F9C3JgRQ4syJoVWbMiJ+m3mrXfGlasmQX6dKZltoJ8M6PbdC4auG0rzlzhtUczwswZPF5tErUCbMX26MZYpcw50l7NPvsErY5uGr1eNWCzFuReUvuxXyGJzQBndpG547TbB6+5xQtFdplIMoFDfZTDU7cA9xNOCDRB10lsqJNkX6a6vdAUyhODgfSaWKW2j4U+RDPk9AyzAsq5hxgTpM3aZYk+Yf7ZbSFtzU+sZpm8Sv6Dnz/HiUbHGSV/9vD59tbw8nmL1cNp7DXAu0kvinVX0dOqBZFdYjCcFeLpRBz1Z2Lmfp57qgGSW2vMpC/Caj8gGn2yLemVWGpGnfSaZdlo5HhONzTpkTNyT52E3lGe4X8TrEuzt8S5PiAWox/cCeIfXLJU3hOR1aT/HdueXr/JcdeBOPr1Shi4vdY6grco30IP12qilPUMr3V8oz1fSCwsVbOi9H6npWfUsgzWn43EZ1ZyDJFscdPlol2/Oq+jstsw30FSuapQFtD2J7rxdEMRknk3ZfeMOLeB+XKtJq8A3PpbkrP8YyUTemtXSVHprfp0ioI38bfpXCPZjln7MzIsNUR3norPA6SY8jNEXlGvk+JG6X1hE8Q0X7tGR7Yj2u4Np0jLelWBNS0RsWdpfp1vTstija0mz5z6t/XmpQXFTJ2/S275PCFkqXkGnt7dJq0517sprUST5JrUS/z8R5Wa8ZFcYVO6GxnduOiv0rnABUpHnEu+TSXypR5gtE/P+T1aSjFqXl+mNe0D0fWyChVI9+NL2sofYfEUXtjPXPo9MJV/Ir6Nv6nxFDCxd6k6eUnEuO0RihVZ0DRbNfLM1msNz7DiTWk2dWgd8VFEUX8DnPSA5FP4MyZqcH7AI/u9+dmjb+S2QWts/NzX8wd+7Gh9MXF8X/jI6zd9l7kK3m2cfsIjydC2wL7DlgX7OH7PvTv5z+a+bozDEOfeGIkHmTKZ+Zqb8B58eAeQ=</latexit>

Cbond(t, l)

<latexit sha1_base64="rpwICZ3lqiwREAFHzawHaEApys=">AlVnic3VpbxtFJ4CpcVcGm4SEi8WaRCVosgpIApSpebipKFp68ZJmrYukdZX5S1d7tru3bc8AP4AzwAkg8AD+DF/4AEv0JiEeQeEGIc87Meue2awtBKfjI3tkz3zlzsyZszOzdgKvFXULhYcnjyqZNPnzr9TO7Z5/4czMiy/tRn4vrLk7Nd/zwz2nGrleq+PudFtdz90LQrfadjz3pnO4gvU3+24YtfzOdncYuHfb1UanVW/Vql1g7c+8srI/qoTtvON3Do7f6s5753K53P7MbGhQJ/8uLCoF2aZ+JT8F196lVXYAfNZjfVYm7msw7pQ9liVRUB32CIrsAB4d9kIeCGUWlTvsmOWA9keoFxAVIF7CL8NuLsjuB24R50RSdegFQ+IUjm2Vzh8LXhV8K3xe+LfxU+D1V14h0oC1DuDpc1g32z3zyWvm3iVJtuHZM5HKkIjgrkqehfD1JvjXZXV2gfxqgZ8BcdDjGm+rf/TpL+UPtuZGbxa+LPwMvn5ReFj4Drzt9H+tfXD3fosw5LY6hBq6/BFjdP3ThqyAfqwG3SCA7YvDYiaXIOUDsT0Qa+R5ERgrb7GT03nbpvUcP2tIop+HQtkN2pFgW8zy4OtQ3IfQixkGT+gnjdp643Cu8C6AUAR5lfRhtHPsG9SRGTi3Dghk2qKNEGZcOpL3T47NZehqkU4Sx2K1WgsEQgUtjekaB4BD5FoK6IQ97jSX7EtD7TKimyNbBr8L0N3+tUQj7+rtH9NiGRHp1tj67fMK7rFJNdKLUpI3QhRi5SGWPqfA8yXEHhMJYiShzBZDp/0+xkTeoDLFwlS1DLGxSVOyJbZFJRv9k7Y96n7D2PAo41VgxC/Sr0dPgirl1rjOoTqH7tSaBtU06GnT1uoOqO6A8hDKya25VIfRhVnTo/iUZY+o/sjSYlNI6vxD4h9Shg601jyq8hKByxSJdtUi8qmdshbkfjDog7AJtlbkDcALhyqxFx0b+G0TdqkMfVP14vfpecCfyGo7TaM+qa1RXY24OUqEvZFmr4h8Xupo4DcpD27J+uEWB9HgNzyKtWtjiNAltsWvuCVj2ZSt0k1mynjtUe1e5ovJeKWN/LwoOy1fY1IaP3c0n0WMnosVvE31F6TI1rV+qx9PgOCRXS81qNi6aIjLhX5FpcW1ykawciBZ/jctu+iGSf1tYNMRY4/+okg098XJUuQGkB9B7TWiOuRwmsx7rGuD6n6CgBIaYinhZ5uFe1FIFURFDXAVSEVcJkZcwu0AqZldYk2BWgFTMitbSGpCKWDNaugykYi5rWm4CqYibhi3bQCpmW9NyDUhFXDNsWQZSMcualg0gFbFh2LIFpGK2NC2bQCpi07DlQyAV86GBuQ6kYq4b1qwCqZhVzZolIBWxpCHWgVTEumFLGUjFlA1brgCpmCtaSztAKmLHaOkGkIq5YWD2gFTMnmHNLSAVc0uz5jaQirhtmZNbYl5jNmnTaLukZV6LnLaC2qC9BEep+pAHOnAfZ52T6p3S9KYcySgbkHJCPuGf4vS/HOMcvWftxTMHsGpiRlJY4pGRhcEciIwLCmDyQj+oaOAZCMGBiIe0SE8TQBwByYgjw4qkYypalHhEMkIR0M0U8ZQzSMbCsLMIz0gGdGzxB7u7kd07RAvpD1zhyJQ9htPE2JtPvStjBwJzaoEIuqExz3IAH5Hgqfb6QsbdL26rSHtbfrCI3w681FWsxXtcr2pbmCGs34jQgbTYF06GmetFsRZxI5zcKI5kxlfGrBT0RG1rYDK9KM6wjasyFDC/Ie9XKFzjA84qdbY9ozjWxglQ2nkg2tsj0rtmdB9q3IvgXZtCKbFuTAihxYkEMrcmhBNqzIhU5Tb81rP3WsmLNLDCgMy2zFeSbGd2mc9nA7Vpx5grPH8IM2fweLVJNDIkzF70xzPELmHOE38+wStjm4bvV43YIsWpFS+7FfIYnNCGd2sbnjrNsEb7nFS012mUgygUN9lMNTtwD3E04IDEAXRWywqdIP0v1B6ApEieHQ+k0MU9tH4t8iOdJaBnmBRVzHjBnyZsS9L8w/0y2sLbGgl9k7RN41f8nPn/eJRucZpX/20PH29vj6eYvVw2mcNcC67SK+KdVfx06oNkT1iMJwV4utEPXnYuV+gnuqCZJ7aCygL8Jp3yYaAbIt7ZV4akX/7TDLrtHIyJh6bTciYuCcvwm6oyHC/iNcV2NvjXZEQS1CP7wXwDq95Km8JyevSP89tv17/SbHXRfijK9X4hJ3mOpK3CP9iH8dKkuTlGr9FbLM9b3ocAmWjkvQet7Vn5KIc9o+d1EfGYhy5TFHj9dJt7xq/s6LrML9zUomacCvoawPdfL4xmMksi7L71hxL0PylVpNXkH5tLdjJ7jGSmf0Vv7So7MbtOlVRC+jb9L64R7Ms/Zm+JDFMf56Fzwus0PY7QFJcb5PuIuD1aR/AMFe/T0u1J9LiCa9Mx2ZOijUdEbFVaX+dbU3HYo+uJcue+7T256UWxU2TvMlu+z4hZKlkBZ3d3jatOtW97a0Ek2Ta1Mv8fMfVqumRTFNTqhs53ZTYr+Op0D1KR4xLkU0FyqUuYJx/8kNenkRSn5vlhUdM+Glsjo1SNfDe+qH0HRJHUz0zKHTC1fxK+7b5J8SIwmXeJOl59ITNIao1SdIUWzXS/PZIne5Awn0ZBlV4veFZdFPE7zEkPRD79CMqYnR6wi/zsfn9m1vgvkVnYPb+w+PbC+RvzF5aFv8zOs1eZ29Atlpk7FL8HQusR2w54h9zr5m35z98ewfcyfnTnHoEyeEzMtM+czN/AmWwIBi</latexit>

T = 0.015

240 samples

N = 20, M = 200(α = 10)

L = 10

0.2 0.4 0.6 0.8 1 0×100 1×104 2×104 3×104 4×104 5×104 6×104 7×104 8×104 9×104 1×105 l=1 2 3 4 5 6 7 8 9 10 0.2 0.4 0.6 0.8 1 0×100 1×104 2×104 3×104 4×104 5×104 6×104 7×104 8×104 9×104 1×105 1 2 3 4 5 6 7 8 9

SLIDE 29

teacher-student setting

Greedy Monte Carlo simulation on Binary perceptron

Training

teacher-student overlaps bond spin

Test

spin

SLIDE 30

Solution space of over-parametrized DNN input

• utput

solid liquid solid “encoding” “decoding”

Summary of this work

• 1. Liquid phase helps equilibration
• 2. Crystal phase + symmetry breaking field

enables generalization

• 3. Spacial evolution of the hierarchal free-energy landscape:

DNN naturally has the power of renormalization: classification,

feature detection

symmetry breaking field

ξ ∝ ln(M/N)

<latexit sha1_base64="FNsqtBcwkWx/yebBj4t5E/qt9Z0=">ACfHichVHLShxBFD3TmkTHJE4iguCmcWIwhIx3ZnyNK9GNmwQfGRUcGbrb0hT2dDXdNYM6+AP+gAtXCoKJn+HGH8jCTwguFbIJIbcfJIgkuQVp86959apKt3ZaiJrjNGR+ejx0+6urM9T5897829eLkSqmbgiKqjXBWs2VYoXOmJqpbaFWt+IKyG7YpVe2cuyq+2RBK5X3Ue7YaFjbntySjqWZquf6a7vSrPmB8rUya65njrwf/fCmnstToTJWqVDJTMD4eApKk2axQHkcaCyp2jhk0oOGiAQEPmrELCyGPdRB8JnbQJu5gJGM8wIHyLK2yVWCKyxmd3je5t16ynq8j3qGsXqXKxQrFXc5gIlh+kqf6Zau6IK+0Y+/dmvHXSI3e7zaiVb49d7DgeXv/1U1eNX49Ef1T9caW5iK3Up278dMdA8n0bf2j26Xp5eG26/plG7Y/wld0yXfwGvdOWeLYukY2eQLoij/fvCHYKVUKJYL5cWx/Mxs+hldGMQRvjFJzGDeSygyue2cYIvuMj8NF4Zb413SamRSTV9uBfGxC/Q5KQ</latexit>

N L

O(log(N)/N)

<latexit sha1_base64="oM8Gsp8WsKFYDS584EbxWte52U=">ACcXichVHLSiNBFD1pHXijIm6mcFNMDhEhHjzQjMrmdm48h0VjEh3W8bGTlfT3Qk6wR/wA3ThQkcQFD/DjT/gwk8QlxFmMwtvPxiRQb0FVafOvefWqSrNg3XI7qLKR2dH7q6ez7Gez97ksk+weWXdlwdFHRpSmdVU1hWlYouIZnilWbUeodc0UK9rOTz+/0hSOa0hryduzxXpdrVnGlqGrHlPV2UzVlLXMzOj4zOhGMk3ZcrFcpnwqBKVSBPITqVyWgkgjijmZvEAVm5DQ0UAdAhY8xiZUuDzWkAPBZm4dLeYcRkaQF9hHnLUNrhJcoTK7w3ONd2sRa/He7+kG6l2ukKyU3GUfKYzQLV1Sm27oiu7p76vdWkEX380er1qoFfZG4uDL4p93VXVePWw/q9507WELk4Fbg93bAePfQw/1zV9H7cXvCyOtb3RGD+z/N93RNd/Aaj7q5/Ni4Rjx8Av8KPx78P/Bcj6bK2QL8X01I/oM3owhGFk+MUnMIVpzKHC59o4xAlOY23lq5JShsNSJRZpBvEilLEnjtiOcQ=</latexit>

SLIDE 31

Outlook

student machine input

• utput

Various statistical inference problems Theories in

Like “Landau to Ginzburg-Landau” but more microscopic

Numerical simulations to test theoretical predctions Complex systems with heterogeneity ultra-stable glass, rheology gene regulatory network,… functionality vs robustness in biology allostericity

d = 1 + ∞

Thank you!