Generating solutions of "impossible-to- solve" problems - - PowerPoint PPT Presentation
Generating solutions of "impossible-to- solve" problems and simulating "impossible-to-simulate" models Florent Krzakala Espci - Paristech L. Zdeborov ( Los Alamos, LANL ) http://www.pct.espci.fr/~florent fl @espci.fr
Espci-Paristech
fl@espci.fr http://www.pct.espci.fr/~florent
Espci-Paristech
fl@espci.fr http://www.pct.espci.fr/~florent
A 3-coloring of a random graph with c=4.36
Text
Some optimization problems such as COL and SAT are almost impossible to solve!
ex: Hard Instances of random graph coloring
A 3-coloring of a random graph with c=4.36
Text
The “ ” problem q log q
Some optimization problems such as COL and SAT are almost impossible to solve!
ex: Hard Instances of random graph coloring
A 3-coloring of a random graph with c=4.36
Text
The “ ” problem q log q
Some optimization problems such as COL and SAT are almost impossible to solve!
ex: Hard Instances of random graph coloring
random graph of average degree c
.h.p this graph is colorable if c<2q log q
(polynomial) for c> q log q !
Average
degree c
q log q
2q log q
A 3-coloring of a random graph with c=4.36
Text
The “ ” problem q log q
Some optimization problems such as COL and SAT are almost impossible to solve!
ex: Hard Instances of random graph coloring
random graph of average degree c
.h.p this graph is colorable if c<2q log q
(polynomial) for c> q log q !
Average
degree c
q log q
2q log q
COL UNCOL
A 3-coloring of a random graph with c=4.36
Text
The “ ” problem q log q
Some optimization problems such as COL and SAT are almost impossible to solve!
ex: Hard Instances of random graph coloring
random graph of average degree c
.h.p this graph is colorable if c<2q log q
(polynomial) for c> q log q !
Average
degree c
q log q
2q log q
COL UNCOL Possible Impossible
A 3-coloring of a random graph with c=4.36
Text
The “ ” problem q log q
Some optimization problems such as COL and SAT are almost impossible to solve!
ex: Hard Instances of random graph coloring
random graph of average degree c
.h.p this graph is colorable if c<2q log q
(polynomial) for c> q log q !
No one has ever seen the solution of, say 5-coloring, for large enough c and N=106
Average
degree c
q log q
2q log q
COL UNCOL Possible Impossible
Some optimization problems such as COL and SAT are also hard to sample
Average
degree c
q log q
2q log q
Some optimization problems such as COL and SAT are also hard to sample
Average
degree c
q log q
2q log q
Consider the foowing “coloring”
Hamiltonian H =
δ(si, sj)
si = 1, 2, . . . , q
Some optimization problems such as COL and SAT are also hard to sample
Average
degree c
q log q
2q log q
Consider the foowing “coloring”
Hamiltonian H =
δ(si, sj)
si = 1, 2, . . . , q
Dynamic transition Temperature
Some optimization problems such as COL and SAT are also hard to sample
Average
degree c
q log q
2q log q
Consider the foowing “coloring”
Hamiltonian H =
δ(si, sj)
si = 1, 2, . . . , q
Impossible-to-sample region Possible-to-sample region
Dynamic transition Temperature
W e know that some random problems DO have solutions, but we cannot find them! Sampling and performing Monte-Carlo is even Harder! Many predictions from statistical physics in random problems.... but impossible to test most of them !
Problem Solution
Problem Solution Solution Problem
Problem Solution Solution Problem
Problem Solution Solution Problem
Consider the 3-coloring problem with N nodes and M links. 1) Color randomly the N nodes
Consider the 3-coloring problem with N nodes and M links. 1) Color randomly the N nodes 11) Put the M links randomly such that the planted configuration is a proper coloring
Consider the 3-coloring problem with N nodes and M links. 1) Color randomly the N nodes 11) Put the M links randomly such that the planted configuration is a proper coloring 111) Now, we have created a problem for which we know the solution
Consider the 3-coloring problem with N nodes and M links. 1) Color randomly the N nodes 11) Put the M links randomly such that the planted configuration is a proper coloring 111) Now, we have created a problem for which we know the solution
IV) W e could also have prepared a configuration with a known cost/energy
Random ensemble Planted ensemble
Choose a random graph with N nodes and M links Choose a random coloring of N nodes Choose a random graph such that this is a correct coloring...
Is it really the same to look for a solution a random problem and to look for a random problem that matches a random solution ?
Random ensemble Planted ensemble
Choose a random graph with N nodes and M links Choose a random coloring of N nodes Choose a random graph such that this is a correct coloring...
Is it really the same to look for a solution a random problem and to look for a random problem that matches a random solution ?
The surprising answer is: in some cases YES ! Random ensemble Planted ensemble
Choose a random graph with N nodes and M links Choose a random coloring of N nodes Choose a random graph such that this is a correct coloring...
Is it really the same to look for a solution a random problem and to look for a random problem that matches a random solution ?
The surprising answer is: in some cases YES ! Random ensemble Planted ensemble
Choose a random graph with N nodes and M links Choose a random coloring of N nodes Choose a random graph such that this is a correct coloring...
Montanari and Semerjian, Jstat. ‘06 & Achlioptas and Coja-Oghlan, arXiv:0803.2122:
The two ensembles are asymptotically (N➔∞) equivalent for low enough degree c !
Random ensemble Planted ensemble
Choose a random graph with N nodes and M links Choose a random coloring of N nodes Choose a random graph such that this is a correct coloring...
Definition : Two ensembles of random graphs are asymptotically equivalent if and only if in the thermodynamic limit every property which is almost surely true on a graph from one ensemble is also almost surely true on a graph from the other ensemble.
Until which connectivity/degree c the planted and random ensembles are equivalent ? Is the planted ensemble interesting beyond this connectivity ? Can we generalize this approach to finite energy (coloring with a finite fraction of mistakes ?) How can we use a planted configuration ? What are the models where a “quiet” planting is possible ?
* W e use the formalism described in Zdeborová’s talk
Consider a model where the annealed computation is correct in some region (high temperature or low degree) f = − 1 N β[log Z]dis fannealed = − 1 N β log [Z]dis
Consider a model where the annealed computation is correct in some region (high temperature or low degree) f = − 1 N β[log Z]dis fannealed = − 1 N β log [Z]dis Consider a model where a factorized (i.e. identical for all nodes) Belief Propagation solution is correct in some region (high temperature or low degree)
ertex-Cover (independent set)
This condition is fulfied (at least in some region) for many models:
ertex-Cover (independent set)
This condition is fulfied (at least in some region) for many models: This condition is not fulfied for :
Consider a model where the annealed computation is correct in some region (high temperature or low degree) Consider a model where a factorized (i.e. identical for all nodes) Belief Propagation solution is correct in some region (high temperature or low degree)
Consider a model where the annealed computation is correct in some region (high temperature or low degree)
In the region where the two free energies are equal, the two ensembles are equivalent
Consider a model where a factorized (i.e. identical for all nodes) Belief Propagation solution is correct in some region (high temperature or low degree)
Consider a model where the annealed computation is correct in some region (high temperature or low degree)
In the region where the two free energies are equal, the two ensembles are equivalent In the region where the two free energies are difgerent, the planted configuration induces an additional “Gibbs” state (or BP fixed point)
Consider a model where a factorized (i.e. identical for all nodes) Belief Propagation solution is correct in some region (high temperature or low degree)
ertex-Cover (independent set)
This condition is fulfied (at least in some region) for many models:
ertex-Cover (independent set)
This condition is fulfied (at least in some region) for many models:
For all these models, the cavity method allows to compute the value of the threshold beyond which f ≠ fannealed ➩“Phase transition”
ertex-Cover (independent set)
This condition is fulfied (at least in some region) for many models:
For all these models, the cavity method allows to compute the value of the threshold beyond which f ≠ fannealed ➩“Phase transition”
In some models, the equivalence can be proven
Conjecture 1: the planted model is equivalent to the
solution is correct (for physicists: up to the static spin glass transition...) and the planted configuration is a “typical” one.
ex: 5-coloring of Erdös-Renyi random graphs
Average
degree c
Uncol Glass Transition
c=13.23(1) c=13.669(2)
ex: 5-coloring of Erdös-Renyi random graphs
Average
degree c
Uncol Glass Transition
c=13.23(1) c=13.669(2)
ex: 5-coloring of Erdös-Renyi random graphs
Average
degree c
Uncol Glass Transition
c=13.23(1) c=13.669(2)
16 18 20 22 Average degree Random start cl 10-4 10-2 1 1 100 10000 %Unsatisfied vs #sweeps Walk-COL c=11.5 12.0 12.5 13.0 13.5 Normal Planted
5-coloring using walkcol with N=106
#violated edges
One can create impossible to solve problems of any size where the solution is known
Number of flips /N
ex: q-coloring of Erdös-Renyi random graphs for large q
Average
degree c
q log q
2q log q
COL UNCOL Possible Impossible
ex: q-coloring of Erdös-Renyi random graphs for large q
Average
degree c
q log q
2q log q
COL UNCOL Possible Impossible
Planting !
Conjecture 1: the planted model is equivalent to the
solution is correct (for physicists: up to the spin glass transition...) and the planted configuration is a “typical” one
Conjecture 1: the planted model is equivalent to the
solution is correct (for physicists: up to the spin glass transition...) and the planted configuration is a “typical” one
Proceedings of Random'06, LNCS 4410,
Planted configuration easy to find for large enough c
Conjecture 1: the planted model is equivalent to the
solution is correct (for physicists: up to the spin glass transition...) and the planted configuration is a “typical” one
Conjecture 2: Planted configuration are hard to find until the so-called Kesten-Stigum threshold, (for physicists: this is the local spin glass instability) beyond which they can be solved easily using BP .
Average
degree c
Uncol Glass Transition
c=13.23(1) c=13.669(2)
The planted solution is we hiden until cKS=(q-1)2
Equivalence
q=5
Equivalence
Average
degree c
c=13.23(1) c=13.669(2)
The planted solution is we hiden until cKS=(q-1)2
c=16
Glass Transition Uncol KS
q=5
Equivalence Planted configuration easy to find!
Average
degree c
c=13.23(1) c=13.669(2)
The planted solution is we hiden until cKS=(q-1)2
c=16
Glass Transition Uncol KS
q=5
Equivalence Planted configuration easy to find!
Average
degree c
c=13.23(1) c=13.669(2)
The planted solution is we hiden until cKS=(q-1)2
c=16
Glass Transition Uncol KS
Average
degree c
q log q
2q log q
COL Impossible
(q-1)2
Equivalence Planted configuration easy to find!
q=5 large q
How to perform simulations that are usuay considered to be impossible?
Random optimization problems & mean-field spin glasses
Average
degree c
Temperature Static Spin-Glass transition
Random optimization problems & mean-field spin glasses
Average
degree c
Dynamic transition Temperature Static Spin-Glass transition
Random optimization problems & mean-field spin glasses
Average
degree c
Dynamic transition Temperature Static Spin-Glass transition
Impossible-to-sample region Possible-to-sample region
configuration satisfies all constraints
e now have a random instance and a “typical” equilibrium solution at zero temperature
e use it !
Prediction: beyond the so-called “dynamic” threshold, a non-trivial non-factorized fixed point of BP is obtained if one starts from an equilibrium configuration
M = 1 qN
q
ψc,i
BP − 1 q
1 − 1
q
!
equilibrium
snon!planted!clusters
planted!cluster
s stot
Cs Cd Cc
Average degree
14 13.6 13.4 13.2 13 12.8 12.6 0.2 0.15 0.1 0.05 13.8
cluster FK, Montanari, Semerjian, Ricci-Tersenghi, Zdeborova, PNAS 07 & FK and Zdeborova, PRE 07
ψfactorized = (1 q , 1 q . . . 1 q )
M = 1 qN
q
ψc,i
BP − 1 q
1 − 1
q
Simulation with N=106 Prediction: beyond the so-called “dynamic” threshold, a non-trivial non-factorized fixed point of BP is obtained if one starts from an equilibrium configuration ψfactorized = (1 q , 1 q . . . 1 q ) average degree Magnetization
configuration has exactly the equilibrium energy
e now have a random instance and a “typical” equilibrium solution at temperature T
e use it !
Average
degree c
Dynamic transition Temperature Glass transtion
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, T=0.255,c=3
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255
C(t) = 1 N
N
Si(tinit = 0)Si(t)
Usual Approach:
1) Start with a random initial condition 2) compute the correlation function:
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255
C(t) = 1 N
N
Si(tinit = 0)Si(t)
Usual Approach:
1) Start with a random initial condition 2) compute the correlation function:
time
Correlation
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255 Usual Approach:
1) Start with a random initial condition 2) Try to find an equilibrium configuration 2) compute the correlation function:
C(t) = 1 N
N
Si(tinit = tw)Si(t − tw)
time
Correlation
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255 Usual Approach:
1) Start with a random initial condition 2) Try to find an equilibrium configuration 2) compute the correlation function:
C(t) = 1 N
N
Si(tinit = tw)Si(t − tw)
tw=10 time
Correlation
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255 Usual Approach:
1) Start with a random initial condition 2) Try to find an equilibrium configuration 2) compute the correlation function:
C(t) = 1 N
N
Si(tinit = tw)Si(t − tw)
tw=100 time
Correlation
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255 Usual Approach:
1) Start with a random initial condition 2) Try to find an equilibrium configuration 2) compute the correlation function:
C(t) = 1 N
N
Si(tinit = tw)Si(t − tw)
tw=1000 time
Correlation
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255 Usual Approach:
1) Start with a random initial condition 2) Try to find an equilibrium configuration 2) compute the correlation function:
C(t) = 1 N
N
Si(tinit = tw)Si(t − tw)
tw=10000 time
Correlation
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255
C(t) = 1 N
N
Si(tinit = 0)Si(t)
A better Approach:
1) Start with an equilibrated initial condition 2) compute the correlation function:
time
Correlation
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255
C(t) = 1 N
N
Si(tinit = 0)Si(t)
A better Approach:
1) Start with an equilibrated initial condition 2) compute the correlation function:
time
Correlation
Prediction: beyond the so-called “dynamic” threshold, the Monte-Carlo Dynamic is trapped! Ex: 3-XORSAT, Td=0.255 A better Approach:
Start with an equilibrated initial condition Many temperatures: Divergence of the relaxation time
T=0.3
T=0.28
T=0.27
T=0.255
T=0.265 T=0.29
T=0.26
time Correlation
Prediction: cf. Zdeborová’s talk: Monte Carlo cooling and heating follow a well defined line
0.01 0.02 0.03 0.1 0.2 0.3 0.4 e(T) in XORSAT (c=3,K=3) Td
XOR-SAT problems (Parity-check) H({S}) =
1 + JijkSiSjSk 2
Prediction: cf. Zdeborová’s talk: Monte Carlo cooling and heating follow a well defined line XOR-SAT problems (Parity-check) H({S}) =
1 + JijkSiSjSk 2 N=200 000 spins
Energy
Temperature
Prediction: cf. Zdeborová’s talk: Monte Carlo cooling and heating follow a well defined line XOR-SAT problems (Parity-check) H({S}) =
1 + JijkSiSjSk 2 N=200 000 spins
Energy
Temperature
Prediction: cf. Zdeborová’s talk: Monte Carlo cooling and heating follow a well defined line XOR-SAT problems (Parity-check) H({S}) =
1 + JijkSiSjSk 2 N=200 000 spins
Energy
Temperature
Prediction: cf. Zdeborová’s talk: Monte Carlo cooling and heating follow a well defined line XOR-SAT problems (Parity-check) H({S}) =
1 + JijkSiSjSk 2 N=200 000 spins
Energy
Temperature
H({S}) =
1 + JijkSiSjSk 2
H({S}) =
1 + JijkSiSjSk 2
+ΓHperturb Start with an equilibrated configuration at Γ=0 and increase Γ
H({S}) =
1 + JijkSiSjSk 2
H =
1 + Jijksz
i sz jsz k
2 + Γ
sx
i
Example: include a quantum transverse field!
H =
1 + Jijksz
i sz jsz k
2 + Γ
sx
i
Γ Energy
Starting om equilibrium Starting om random
H =
1 + Jijksz
i sz jsz k
2 + Γ
sx
i
Γ Energy
Starting om equilibrium Starting om random
First order Quantum transition
Imply the failure of Quantum Annealing (or Quantum Adiabatic Algorithm)
A quiet planting is possible in many models.
up to the condensation threshold.
FK and L. Zdeborová:
* Phys. Rev. Lett. 102, 238701 (2009)
* arXiv:0902.4185, submitted in SIAM Journal on Discrete Mathematics
* And more to come...
A quiet planting is possible in many models.
up to the condensation threshold.
There is a free lunch: instantaneous simulations.
simulated efficiently using planting at zero or finite temperature.
FK and L. Zdeborová:
* Phys. Rev. Lett. 102, 238701 (2009)
* arXiv:0902.4185, submitted in SIAM Journal on Discrete Mathematics
* And more to come...