Spatial coupling: Algorithm and Proof Technique Workshop on Local Algorithms - WOLA 2018 Boston, June 15th, 2018 1
Physics inspiration: nucleation, crystallization, meta-stability 2
Supercooled water
Heat packs Sodium acetate , C 2 H 3 NaO 2 ,
Nucleation
Spatial-Coupling as an Algorithm 6
Introduction - Graphical Codes Low-density Parity-Check (LDPC) Codes d l (= 3) d r (= 6) x 1 c 1 x 6 + x 7 + x 10 + x 20 = 0 x 2 x 3 c 2 x 4 x 5 c 3 rate ≥ r design = #variables − #checks = 20 − 10 = 1 x 6 #variables 20 2 x 7 c 4 x 8 x 9 c 5 rate ∼ r design x 10 x 11 c 6 x 12 x 13 c 7 variable check x 14 x 15 nodes nodes c 8 x 16 x 17 c 9 x 18 x 19 c 10 x 20 x 4 + x 9 + x 13 + x 14 + x 20 = 0
Ensemble of Codes - Configuration Construction (3, 6) ensemble 1 2 1 3 2 3 4 4 5 5 6 6 7 8 7 9 8 10 9 11 12 10 11 12 each configuration code is sampled u.a.r. has uniform from the ensemble probability and used for transmission
BP Decoder - BEC 0 0 0 ? 0=0 0= 0 ? 0 ? decoded ? ? 0+?= 0+?=? ? ? ? ? ?
How does BP perform on the BEC? x 0.4 0.5 0.6 0.7 0.8 0.9 1.0 P error 1.0 0.8 0.6 0.4 0.3 0.2 0.2 0.1 0.0 eps 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 (3, 6) ensemble
Asymptotic Analysis - Density Evolution (DE) y d l − 1 1 − (1 − x ) d r − 1 � ? ? � channel erasure fraction ? ? ? ? ? ? y y y x x x one iteration one iteration of BP at variable of BP at check node node
Asymptotic Analysis - Density Evolution (DE) erasure fraction at the root after iterations � x ( ` ) = ✏ ( y ( ` ) ) d l − 1 y ( ` ) = 1 − (1 − x ( ` − 1) ) d r − 1 x ( ` =2) = ✏ ( y ( ` =2) ) d l − 1 y ( ` =2) = 1 − (1 − x ( ` =1) ) d r − 1 x ( ` =1) = ✏ ( y ( ` =1) ) d l − 1 y ( ` =1) = 1 − (1 − x ( ` =0) ) d r − 1 x ( ` =0) = ✏
Asymptotic Analysis - Density Evolution (DE) f ( ✏ , x ) = ✏ (1 − (1 − x ) d r − 1 ) d l − 1 f ( ✏, x ) is increasing in both its arguments x ( ` ) ≤ x ( ` − 1) x ( ` +1) = f ( ✏, x ( ` ) ) f ( ✏, x ( ` − 1) ) = x ( ` ) ≤ x (1) = f ( ✏, x (0) = 1) = ✏ ≤ x (0) = 1 Note: DE sequence is decreasing and bounded from below ⇒ converges
EXIT Curve for (3, 6) Ensemble x x x x x x x x x 1.0 EXIT 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 EXIT value as a 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 function of increasing iterations for a given channel value 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 eps eps eps eps eps eps eps eps eps 0.0 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
A look back ... x 0.4 0.5 0.6 0.7 0.8 0.9 1.0 P error 1.0 EXIT 0.8 0.6 0.4 0.3 0.2 0.2 0.1 0.0 eps 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 (3, 6) ensemble
BP decoder ends up in meta-stable state. Optimal (MAP) decoder would reach stable state. Can we use nucleation? 16
The Spatially Coupled Ensemble
The Spatially Coupled Ensemble
(d l , d r , w, L) (d l , d r , w, L) (d l , d r , w, L) M w L
DE for Coupled Ensemble ✏ MAP ✏ BP �
DE for Coupled Ensemble ✏ MAP ✏ BP �
DE for Coupled Ensemble ✏ MAP ✏ BP �
Thresholds capacity 1/2 BEC BAWGNC BSC (3, 6) 0.488 0.48 0.468 (4, 8) 0.498 0.496 0.491 (5, 10) 0.499 0.499 0.497 (6, 12) 0.4999 0.4996 0.499
Back to the Physics Interpretation Krzakala, Mezard, Sausset, Sun, and Zdeborova metastability and nucleation
Spatially Coupled Ensembles — Summary • achieve capacity for any BMS channel • block length: O (1 / δ 3 ) • encoding complexity per bit: O (log(1 / δ )) • number of iterations: O (1 / δ ) (educated guess :-)) • number of bits required for processing of messages: O (log(1 / δ )) • decoding complexity per bit: O (1 / δ log 2 (1 / δ )) bit operations
Main Message Coupled ensembles under BP decoding behave like uncoupled ensembles under MAP decoding. Since coupled ensemble achieve the highest threshold they can achieve (namely the MAP threshold) under BP we speak of the threshold saturation phenomenon. Via spatial coupling we can construct codes which are capacity-achieving universally across the whole set of BMS channels. On the downside, due to the termination which is required, we loose in rate. We hence have to take the chain length large enough in order to amortize this rate loss. Therefore, the blocklength has to be reasonably large.
Spatial Coupling as a Proof Technique (coding) shows that MAP threshold is given by Maxwell conjecture
Spatial Coupling as a Proof Technique 30
Paradigmatic CSP: random K -SAT I Random graph with n variable nodes and m clauses. I Each variable node is connected to K clauses u.a.r by an edge. I Edge is dashed or full with probability 1 / 2. Degree of variable nodes is Poisson ( ↵ K ) . I Boolean variables: x i ∈ { T , F } or ∈ { 0 , 1 } , i = 1 , · · · , n i = 1 x n ( a i ) ∨ K I Clauses: � � , a i a = 1 , · · · , m i = 1 x s ( a i ) I F n , α , K = ∧ M ∨ K � � a i a = 1 #( clauses ) #( variables ) = m Control parameter ↵ = n : Phase Transitions. Based on joint work with D. Achlioptas (UCSD), H. Hassani (UPenn), and Nicolas Macris (EPFL) 31
I Friedgut 1999: ∃ ↵ s ( n , K ) s.t ∀ ✏ > 0 ⇢ 1 if ↵ < ( 1 − ✏ ) ↵ s ( n , K ) , � lim F n , α , K is SAT = n →∞ Pr if ↵ > ( 1 − ✏ ) ↵ s ( n , K ) . 0 Existence of lim n → + ∞ ↵ s ( n , K ) is still an open problem. I This talk: MAX-SAT or Hamiltonian version of the problem: m X � � �� i = 1 x s ( a i ) ∨ K H F ( x ) = 1 − 1 , a i a = 1 the MAX-SAT/UNSAT threshold is defined as: 1 � ↵ s ( K ) ≡ inf ↵ | lim n E [ min H F ( x )] > 0 n → + ∞ x | {z } exists and continuous function of α In particular ↵ s exists. [Interpolation methods: Franz-Leone, Panchenko, Gamarnik-Bayati-Tetali] . 32
The Physics Picture Parisi-Mezard-Zechina 2001 Semerjian-RicciTersenghi-Montanari, Krazkala-Zdeborova 2008
Known Lower bounds on the SAT-UNSAT threshold I Algorithmic lower bounds: find analyzable algorithm and find solutions for ↵ alg ( K ) < ↵ s ( K ) . [long history ...] I Second Moment lower bounds, weighted s.m with cavity inspired weights [long history, ... Achlioptas - Coja Oghlan] . K 3 4 large K · · · 2 K ln 2 − 3 3 . 52 alg 7 . 91 s . m 2 ln 2 + o ( 1 ) s . m best lower bound · · · 2 K ln K 3 . 52 5 . 54 ( 1 + o ( 1 )) best algor bound · · · K 2 K ln K 3 . 86 9 . 38 ( 1 + o ( 1 )) α dyn · · · K 2 K ln 2 − 3 2 ln 2 + o ( 1 ) 3 . 86 9 . 55 α cond · · · 2 K ln 2 − 1 4 . 26 9 . 93 2 ( 1 + ln 2 ) + o ( 1 ) α s · · · 34
New Lower bounds by the Spatial Coupling Method Recall: H F ( x ) = number of UNSAT clauses of F for x ∈ { 0 , 1 } n � ↵ | lim n → + ∞ 1 and ↵ s = inf n E [ min x H F ( x )] > 0 K 3 4 large K · · · 2 K × 1 3 . 67 7 . 81 α new · · · 2 2 K ln K 3 . 52 5 . 54 ( 1 + o ( 1 )) best algor bound · · · K 2 K ln 2 − 3 3 . 52 alg 7 . 91 s . m 2 ln 2 + o ( 1 ) s . m best lower bound · · · 2 K ln K 3 . 86 9 . 38 ( 1 + o ( 1 ) α dyn · · · K 2 K ln 2 − 3 3 . 86 9 . 55 2 ln 2 + o ( 1 ) α cond · · · 2 K ln 2 − 1 4 . 26 9 . 93 2 ( 1 + ln 2 ) + o ( 1 ) α s · · · 35
Strategy construct spatially coupled model α coupled = α uncoupled SAT SAT ≤ α (un)coupled α uncoupled ≤ α coupled SAT alg alg
Unit Clause Propagation algorithm 1. Repeat until all variables are set: 2. Forced Step: If F contains unit clauses choose one at random and satisfy it by setting unique variable. Remove or shorten unit clause other clauses that contain this variable. 3. Free Step: If there are no unit clauses choose a variable at random and set it at random. Remove or shorten clauses that contain this variable. 37
Analysis by differential equations [Chao-Franco 1986] A "Round" = "free step immediately followed by forced steps and ends when all forced steps have ended". (Rescaled) time t is number of rounds. For K = 3: d ` ( t ) 8 = − 2 � ( t ) , � ( t ) = #( variables set in a round ) dt > > ✓ ◆ > dc 3 ( t ) 3 c 3 ( t ) > = − � ( t ) < dt ` ( t ) / 2 ✓ ◆ ✓ ◆ > > dc 2 ( t ) 3 c 3 ( t ) 2 c 2 ( t ) 1 = + � ( t ) 2 − � ( t ) > > dt ` ( t ) / 2 ` ( t ) / 2 : d ` ( t ) 2 1 = − = − → 4 ( 1 − ` ( t ) dt 1 − r 1 ( t ) ` ( t )( 1 − 3 ↵ 2 ) 3 ≈ 2 . 66, d ` ( t ) For ↵ → 8 → + ∞ and rate r 1 ( t ) of unit clauses dt ⇒ ↵ UC ( 3 ) = 8 production → 1; = 3 ≈ 2 . 66. 38
Recommend
More recommend
