Phase Transitions in Semidefinite Relaxations Andrea Montanari - PowerPoint PPT Presentation

Spectral relaxation bad in the sparse regime! d ❂ 1 ✿ 5 d ❂ 15 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0.0 0.0 3 2 1 0 1 2 3 3 2 1 0 1 2 3 p γ p γ eigenvalues/ eigenvalues/ Theorem ( Krivelevich, Sudakov 2003 + Vu 2005 ) With high probability, ✭ ♣ if d ✢ ✭ log n ✮ 4 ❀ 2 ✭ 1 ✰ o ✭ 1 ✮✮ ✕ max ✭ A cen ❂ d ✮ ❂ ♣ C log n ❂ ✭ log log n ✮✭ 1 ✰ o ✭ 1 ✮✮ if d ❂ O ✭ 1 ✮ ✿ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 28 / 75

Example: d ❂ 20, ✕ ❂ 1 ✿ 2, n ❂ 10 4 0.04 0.03 0.02 0.01 0.00 0.01 0.02 0.03 0.04 0 2000 4000 6000 8000 10000 v 1 ✭ A cen ✮ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 29 / 75

Example: d ❂ 3, ✕ ❂ 1 ✿ 2, n ❂ 10 4 0.10 0.05 0.00 0.05 0.10 0 2000 4000 6000 8000 10000 v 1 ✭ A cen ✮ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 30 / 75

Why should SDP work better? ❤ A cen ❀ X ✐ ❀ maximize X ✷ R n ✂ n ❀ X ✗ 0 ❀ subject to X ii ❂ 1 ✿ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 31 / 75

Recall the ultimate limit G ✭ n ❀ d ❀ ✕ ✮ graph distribution with parameters d ❂ a ✰ b a � b ❃ 1 ❀ ✕ ❂ ♣ 2 2 ✭ a ✰ b ✮ Theorem ( Mossel, Neeman, Sly, 2012) If ✕ ❁ 1 , then ✌ ✌ ✌ G ✭ n ❀ d ❀ 0 ✮ � G ✭ n ❀ d ❀ ✕ ✮ ✌ lim sup TV ❁ 1 ✿ n ✦✶ If ✕ ❃ 1 , then ✌ ✌ ✌ G ✭ n ❀ d ❀ 0 ✮ � G ✭ n ❀ d ❀ ✕ ✮ ✌ lim TV ❂ 1 ✿ n ✦✶ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 32 / 75

SDP has nearly optimal threshold Theorem ( Montanari, Sen 2015) Assume G ✘ G ✭ n ❀ d ❀ ✕ ✮ . If ✕ ✔ 1 , then, with high probability, 1 ♣ SDP ✭ A cen G ✮ ❂ 2 ✰ o d ✭ 1 ✮ ✿ n d If ✕ ❃ 1 , then there exists ✁✭ ✕ ✮ ❃ 0 such that, with high probability, 1 ♣ SDP ✭ A cen G ✮ ❂ 2 ✰ ✁✭ ✕ ✮ ✰ o d ✭ 1 ✮ ✿ n d Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 33 / 75

Consequence ♣ ✭ if SDP ✭ A cen 1 G ✮ ✕ ✭ 2 ✰ ✍ ✮ n d , T SDP ✭ G ✮ ❂ 0 otherwise. Corollary ( Montanari, Sen 2015) Assume ✕ ✕ 1 ✰ ✧ . Then there exists d 0 ✭ ✧ ✮ and ✍ ✭ ✧ ✮ such that the SDP-based test succeeds with high probability, provided d ✕ d 0 ✭ ✧ ✮ . Namely ✄ ❂ 0 ✿ ✂ P 0 ✭ T SDP ✭ G ✮ ❂ 1 ✮ ✰ P 1 ✭ T SDP ✭ G ✮ ❂ 0 ✮ lim n ✦✶ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 34 / 75

Earlier/related work Optimal spectral tests ◮ Massoulie 2013 ◮ Mossel, Neeman, Sly, 2013 ◮ Bordenave, Lelarge, Massoulie, 2015 SDP, d ❂ ✂✭ log n ✮ ◮ Abbe, Bandeira, Hall 2014 ◮ Hajek, Wu, Xu 2015 SDP, detection ◮ Guédon, Vershynin, 2015 (requires ✕ ✕ 10 4 , very different proof) Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 35 / 75

How does SDP work ‘in practice’? Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 36 / 75

Thresholds ◮ ✕ opt c ✭ d ✮ ✑ Threshold for optimal test ◮ ✕ SDP ✭ d ✮ ✑ Threshold for SDP-based test c Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 37 / 75

What we know ◮ ✕ opt c ✭ d ✮ ❂ 1 [Mossel, Neeman, Sly, 2013] ◮ ✕ SDP ✭ d ✮ ❂ 1 ✰ o d ✭ 1 ✮ [Montanari, Sen, 2015] c How big is the o d ✭ 1 ✮ gap? Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 38 / 75

Simulations: d ❂ 5, N sample ❂ 500 (with Javanmard and Ricci) 1.0 GOE theory n =2 , 000 n =4 , 000 0.8 n =8 , 000 n =16 , 000 0.6 Overlap 0.4 0.2 0.0 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 q ( a − b ) / 2( a + b ) x SDP ✷ ❢ ✰ 1 ❀ � 1 ❣ n SDP estimator ˆ ✟☞ ☞✠ ✿ ☞ x ✮ ❂ 1 ☞ ❤ ˆ x SDP ✭ G ✮ ❀ x 0 ✐ Overlap n ✭ ❜ n E Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 39 / 75

Simulations: d ❂ 5, N sample ❂ 500 1.0 GOE theory n =2 , 000 n =4 , 000 0.8 n =8 , 000 n =16 , 000 0.6 Overlap 0.4 0.2 0.0 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 q ( a − b ) / 2( a + b ) ✕ SDP ✭ d ❂ 5 ✮ ✙ 1 ✿ c Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 40 / 75

Simulations: d ❂ 10, N sample ❂ 500 1.0 GOE theory n =2 , 000 n =4 , 000 0.8 n =8 , 000 n =16 , 000 0.6 Overlap 0.4 0.2 0.0 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 q a − b/ 2( a + b ) ✕ SDP ✭ d ❂ 10 ✮ ✙ 1 ✿ c Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 41 / 75

Simulations: d ❂ 10, N sample ❂ 500 1.0 GOE theory n =2 , 000 n =4 , 000 0.8 n =8 , 000 n =16 , 000 0.6 Overlap 0.4 0.2 0.0 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 q a − b/ 2( a + b ) How to estimate ✕ SDP ✭ d ✮ from data? c Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 42 / 75

A technique from physics: Binder cumulant Q ✭ G ✮ ✑ 1 x SDP ✭ G ✮ ❀ x 0 ✐ ❀ n ❤ ˆ ✟ Q ✭ G ✮ 4 ✠ Bind ✭ n ❀ ✕❀ d ✮ ✑ E ✟ Q ✭ G ✮ 2 ✠ 2 E CLT heuristics ✭ 3 if ✕ ❁ ✕ SDP ✭ d ✮ , c n ✦✶ Bind ✭ n ❀ ✕❀ d ✮ ❂ lim 1 if ✕ ❃ ✕ SDP ✭ d ✮ . c Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 43 / 75

Simulations: d ❂ 5, N sample ✕ 10 5 ! 3 n=2000 n=4000 n=8000 2.5 Binder 2 1.5 1 0.7 0.8 0.9 1 1.1 1.2 λ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 44 / 75

Zoom ( ✘ 2 years CPU time) 2.1 n=2000 n=4000 2 n=8000 n=16000 1.9 1.8 d = 5 Binder 1.7 1.6 1.5 1.4 1.3 1 1.005 1.01 1.015 1.02 1.025 1.03 λ Estimate ✕ SDP ✭ d ✮ by the crossing point c Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 45 / 75

✕ c ✭ d ✮ SDP 1.02 1.01 ✕ SDP ✭ d ✮ c 1.00 0.99 1 10 d ◮ Dots: Numerical estimates ◮ Line: Non-rigorous analytical approximation (using statistical physics) ◮ At most 2 ✪ sub-optimal! Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 46 / 75

One last question Is this approach robust to model miss-specifications? Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 47 / 75

An experiment ◮ Select S ✒ V uniformly at random. with ❥ S ❥ ❂ n ☛ . ◮ For each i ✷ S , connect all of its neighbors. Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 48 / 75

An experiment 1 SDP, α = 0 0.9 BH, α = 0 SDP, α = 0.025 0.8 BH, α = 0.025 SDP, α = 0.05 0.7 BH, α = 0.05 0.6 0.5 0.4 0.3 0.2 0.1 0 0.8 1 1.2 1.4 1.6 1.8 2 ◮ Solid line:SDP ◮ Dashed line: Spectral (Non-backtracking walk [Krzakala, Moore, Mossel, Neeman, Sly, Zdeborova, Zhang, 2013]) Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 49 / 75

A simple robustness result Lemma ( Montanari, Sen, 2015) If ❢ G is obtained from the hidden partition model by flipping at most n ✧ edges, then ☞ ☞ ☞ ⑦ ☞ ✔ ✍ ✭ ✧ ✮ ✿ ✕ SDP ✭ d ✮ � ✕ SDP ✭ d ✮ c c Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 50 / 75

Proof ideas Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 51 / 75

What we want to prove Theorem ( Montanari, Sen 2015) Assume G ✘ G ✭ n ❀ d ❀ ✕ ✮ . If ✕ ✔ 1 , then, with high probability, 1 ♣ SDP ✭ A cen G ✮ ❂ 2 ✰ o d ✭ 1 ✮ ✿ n d If ✕ ❃ 1 , then there exists ✁✭ ✕ ✮ ❃ 0 such that, with high probability, 1 ♣ SDP ✭ A cen G ✮ ❂ 2 ✰ ✁✭ ✕ ✮ ✰ o d ✭ 1 ✮ ✿ n d Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 52 / 75

Strategy I. Prove equivalence to Gaussian model II. Analyze Gaussian model Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 53 / 75

Gaussian model: x 0 ✷ ❢ ✰ 1 ❀ � 1 ❣ n B ✭ ✕ ✮ ✑ ✕ n x 0 x 0 T ✰ W ✿ W ✘ GOE ✭ n ✮ : ◮ ✭ W ij ✮ i ❁ j ✘ iid N ✭ 0 ❀ 1 ❂ n ✮ ◮ W ❂ W T ◮ A lot is known about spectral properties of B Need to characterize the SDP value with Gaussian data Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 55 / 75

Notation 1 s ✭ ✕ ✮ ✑ lim sup n SDP ✭ B ✭ ✕ ✮✮ ❀ n ✦✶ 1 s ✭ ✕ ✮ ✑ lim inf n SDP ✭ B ✭ ✕ ✮✮ ✿ n ✦✶ ✟ ❤ B ❀ X ✐ ✿ X ✗ 0 ❀ X ii ❂ 1 ✽ i ✠ SDP ✭ B ✮ ✑ max Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 56 / 75

Phase transition at ✕ ❂ 1! Theorem ( Montanari, Sen, 2015) The following holds almost surely ✕ ✷ ❬ 0 ❀ 1 ❪ ✮ s ✭ ✕ ✮ ❂ s ✭ ✕ ✮ ❂ 2 ❀ ✕ ✷ ✭ 1 ❀ ✶ ✮ ✮ s ✭ ✕ ✮ ❃ 2 (strictly) ✿ For explicit probability bounds, see the paper Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 57 / 75

Proof of Gaussian phase transition Simple facts: ◮ ✭ 2 if ✕ ✷ ❬ 0 ❀ 1 ❪ , s ✭ ✕ ✮ ✔ lim n ✦✶ ✛ max ✭ B ✭ ✕ ✮✮ ❂ ✕ ✰ ✕ � 1 if ✕ ✷ ✭ 1 ❀ ✶ ✮ . [Baik, Ben Arous, Peche, 2005] ◮ 1 s ✭ ✕ ✮ ✔ lim n ❤ 1 ❀ B ✭ ✕ ✮ 1 ✐ ❂ ✕ n ✦✶ ◮ s ✭ ✕ ✮ , s ✭ ✕ ✮ are non-random, non-decreasing Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 58 / 75

Summarizing 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 λ ◮ Red: Upper bound ◮ Blue: Lower bound Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 59 / 75

Proof of Gaussian phase transition Part 1: Prove that s ✭ ✕ ❂ 0 ✮ ✕ 2 Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 60 / 75

Hence 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 λ ◮ Red: Upper bound ◮ Blue: Lower bound ◮ Purple: Non-trivial lower bound Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 61 / 75

Proof of Gaussian phase transition Part 1: Prove that s ✭ ✕ ❂ 0 ✮ ✕ 2 Part 2: Prove that s ✭ ✕ ❂ 1 ✰ ✧ ✮ ❃ 2 Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 62 / 75

Hence 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 λ ◮ Red: Upper bound ◮ Blue: Lower bound ◮ Purple: Non-trivial lower bound Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 63 / 75

Proof of Gaussian phase transition Part 1: Prove that s ✭ ✕ ❂ 0 ✮ ✕ 2 Part 2: Prove that s ✭ ✕ ❂ 1 ✰ ✧ ✮ ❃ 2 Technique: Construct feasible X , such that ❤ A ❀ X ✐ ✕ ✿ ✿ ✿ . Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 64 / 75

Part 1: ✕ ❂ 0 Limiting Spectral Density − 2 2 First idea: ◮ v 1 ❂ v 1 ✭ B ✮ ✑ principal eginvector of B ◮ Take X ❂ n v 1 v T 1 ◮ Wrong: X ii ✙ N ✭ 0 ❀ 1 ✮ 2 ✻ ❂ 1 Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 65 / 75

Part 1: ✕ ❂ 0 Limiting Spectral Density − 2 2 Good idea: ◮ Let U ❂ ❬ v 1 ❥ v 2 ❥ ✁ ✁ ✁ ❥ v n ✍ ❪ ✷ R n ✂ n ✍ , ✍ small. ◮ D ✑ Diag ✭ U U T ✮ ✷ R n ✂ n . Claim D ✙ ✍ I ( D ii ✘ n � 1 ✤ n ✍ ) ◮ Set X ❂ D � 1 ❂ 2 ✭ U U T ✮ D � 1 ❂ 2 Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 66 / 75

Part 2: ✕ ❂ 1 ✰ ✧ Limiting Spectral Density − 2 2 Construction: ◮ T ✭ x ✮ ✑ max ✭ min ✭ x ❀ ✰ 1 ✮ ❀ � 1 ✮ , ϕ ✷ R n ✬ i ✑ T ✭ ✧ ♣ n v 1 ❀ i ✮ ✿ ◮ U ❂ ❬ v 2 ❥ v 3 ❥ ✁ ✁ ✁ ❥ v n ✍ ✰ 1 ❪ ✷ R n ✂ n ✍ q ◮ D ✷ R n ✂ n diagonal D ii ✑ 1 � ✬ 2 i ❂ ❦ U e i ❦ 2 . ◮ X ✑ ϕϕ T ✰ DU U T D Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 67 / 75

A parenthesis Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 68 / 75

The Gaussian model is very interesting 1.0 Max Likelihood Bayes 0.8 SDP PCA SDP, n =200 0.6 SDP, n =400 MSE SDP, n =800 SDP, n =1600 0.4 0.2 0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 λ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 69 / 75

We want to prove 1 ♣ SDP ✭ A cen ✮ ✙ SDP ✭ B ✭ ✕ ✮✮ d Lindeberg method: Replace the entries one-by-one Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 71 / 75

A simple Lindeberg lemma ◮ X 1 ❀ X 2 ❀ ✿ ✿ ✿ X M iid ✽ ✏ ✑ 1 � d with probability d ❁ 1 n ❀ ♣ n d ✏ ✑ X i ❂ ♣ ✿ d 1 � d � with probability ✿ n n E ❢ X i ❣ ❂ 0, E ❢ X 2 i ❣ ❂ ✭ 1 ❂ n ✮ � ✭ d ❂ n 2 ✮ ◮ Z 1 ❀ Z 2 ❀ ✿ ✿ ✿ Z M ✘ i ✿ i ✿ d ✿ N ✭ 0 ❀ 1 ❂ n ✮ Lemma Assume M ❂ n ✭ n � 1 ✮ ❂ 2 , F ✷ C 3 ✭ R M ✮ , d ✔ n 2 ❂ 3 ❂ 10 . Then ☞ ☞ ✏✌ ✑ n ✌ ✌ ✌ ☞ ☞ ✌ ❅ 2 ✌ ✌ ❅ 3 ✌ ☞ E F ✭ X ✮ � E F ✭ Z ✮ ♣ max i F i F ☞ ✔ ✶ ❴ ✿ ✶ 3 d i ✷ ❬ M ❪ i F ✭ x ✮ ✑ ❅ ❵ F where ❅ ❵ i , and ❦ ❅ ❵ i F ❦ ✶ ✑ sup x ✷ R M ❥ ❅ ❵ i F ✭ x ✮ ❥ . ❅ x ❵ Problem: F ✭ ✁ ✮ ❂ SDP ✭ ✁ ✮ ✻✷ C 3 ✭ R M ✮ Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 72 / 75

Phase Transitions in Semidefinite Relaxations Andrea Montanari - PowerPoint PPT Presentation

Phase Transitions in Semidefinite Relaxations Andrea Montanari [with Adel Javanmard, Federico Ricci-Tersenghi, Subhabrata Sen] Stanford University December 7, 2015 Andrea Montanari (Stanford) SDP Phase Transitions December 7, 2015 1 / 75

On the local stability of semidefinite relaxations Diego Cifuentes Department of Mathematics

Approximating Pareto Curves using Semidefinite Relaxations Victor MAGRON Postdoc LAAS-CNRS

Topological defects and cosmological phase transitions Mark Hindmarsh 1 , 2 1 Department of Physics

Random Graphs (2 nd part) Omid Etesami Phase transitions for CNF-SAT Phase transitions for other

Phase transitions A. O. Lopes Inst. Mat. - UFRGS 25 de fevereiro de 2015 A. O. Lopes (Inst.

Detecting Phase Transitions with Artificial Neural Networks Sebastian J. Wetzel Institute for

NON-EQUILIBRIUM PHASE TRANSITIONS Michael J. Kastoryano Coogee 2015 Tuesday, February 10, 15

Phase Transitions, Gravitational Waves, and Composite Dark Matter Pedro Schwaller (DESY)

Gravitational waves from first-order phase transitions: some developments in ultra-supercooled

Gravitational waves from first-order phase transitions: Towards understanding ultra-supercooled

Stability, phase transitions, phase diagrams following Callen 1985 book 8Stability of

Early-Warning Signals and Phase Transitions in Psychotherapy Early-warning signals for phase

Lecture 2. Random Matrix Theory and Phase Transitions of PCA Yuan Yao Hong Kong University of

Thermo modynami mics and ph phase More on the theory of tricritical transitions (see

Thermodynamic limit and phase transitions in non-cooperative games: some mean-field examples

Phase Transitions in Classical Planning: Formalization An Experimental Study Experiments

On sub-determinants and the diameter of polyhedra Martin Niemeier, EPF Lausanne Joint work with:

Graphs and limits Mathias Schacht Institut f ur Informatik Humboldt-Universit at zu Berlin

Decomposing Cubic Graphs into Connected Subgraphs of Size Three Laurent Bulteau Guillaume Fertin

Volume of alcoved polyhedra and Mahler conjecture M.J. de la Puente F. Matem aticas, U.

Improved Clustering Algorithms for the Random Cluster Graph Model Ron Shamir Dekel Tsur Tel

Counting Problems over Incomplete Databases Mikal Monet Formal Methods team seminar at LaBRI

Kauffman bracket polynomials of Conway-Coxeter Friezes (joint work with Michihisa Wakui) Takeyoshi

Probabilistic Analysis of Christofides Algorithm Markus Bl aser Konstantinos Panagiotou B.