Local-Optimality Guarantees for Optimal Decoding Based on Paths - - PowerPoint PPT Presentation

Local-Optimality Guarantees for Optimal Decoding Based on Paths

Nissim Halabi Guy Even

School of Electrical Engineering, Tel-Aviv University

August 29, 2012

1/23

Communication Over a Noisy Channel

Channel Encoder Channel Decoder codeword Noisy Channel noisy codeword λ(y) ∈ RN ˆ u ∈ {0, 1}k ˆ c ∈ {0, 1}N u ∈ {0, 1}k c ∈ C ⊂ {0, 1}N

MBIOS channel: memoryless, binary-input, output-symmetric Log-Likelihood-Ratio (LLR): λi(yi) ln Pr(yi | ci = 0) Pr(yi | ci = 1)

• Linear Code: C ⊆ {0, 1}N is subspace of FN

2 of dimension k.

Optimal decoding: Maximum Likelihood decoding. Input: y. Output: ml(y). ml(y) arg max

x∈C

Pr{y | c = x} = arg min

x∈C

λ(y), x

2/23

Tanner Codes Defined by Tanner Graphs

v1 x1 v2 x2 v4 x4 v3 x3 v5 x5 v6 x6 v7 x7 v8 x8 v10 x10 x9 v9

G = (V ∪ J , E)

Variable Nodes Local-Code Nodes

V J

C4 C4 C1 C1 C2 C2 C3 C3 C5 C5

Tanner code C(G, CJ ) represented by bipartite graph x ∈ C(G, CJ ) iff x ∈ Cj for every j ∈ {1, . . . , J} In general:

degrees: can be regular, irregular, bounded, or arbitrary can allow arbitrary linear local codes

Examples: LDPC codes [Gallager’63], Expander codes [Sipser-Spielman’96]

3/23

Linear Programming (LP) Decoding

conv(X) ⊆ RN - the convex hull a set of points X ⊆ RN. ML-decoding can be rephrased: ml(y) arg min

x∈conv(C)

λ(y), x Generalized fundamental polytope of a Tanner code C(G, CJ )

• relaxation of conv(C) [following

Feldman-Wainwright-Karger’05] P(G, CJ )

• Cj∈CJ

conv(Cj) LP-decoding: lp(y) arg min

x∈P(G,CJ )

λ(y), x

4/23

LP Decoding with ML Certificate

LP-decode(λ) solve LP: ˆ xlp ← arg minx∈P(G,CJ )λ, x. if ˆ xlp ∈ {0, 1}N then return ˆ xlp is an ML codeword else return fail end if Polynomial time algorithm Applies to any MBIOS channel! Integral solution ⇒ ML-certificate

5/23

Goal: Analysis of Finite Length Codes

Problem (Finite Length Analysis) Design: Constant rate code C(G, CJ ) and an efficient decoding algorithm dec. Analyze: If SNR > t, then Pr(dec(λ) = x|c = x) exp(−N α) for some 0 < α. Goal: Minimize t (lower bound on SNR). Remarks: Not an asymptotic problem Code is not chosen randomly from an ensemble Successful decoding = ML decoding

6/23

Unified Analysis Framework via Local-Optimality

Advances in analysis of finite-length codes via local-optimality: [Koetter-Vontobel’06], [Arora-Daskalakis-Steurer’09], [H-Even’10], [Vontobel’10], [H-Even’11]

Based on complicated combinatorial structures embedded in the Tanner graph of the codes and non-trivial analyses of random processes

Today Demonstrate the proof technique - use simple characterization

• f local-optimality based on paths

Simpler proofs obtained via local-optimality based on paths for the case of repeat-accumulate codes [Feldman-Karger’02], [Goldenberg-Burshtein’11]

7/23

Certificate for ML-Optimality / LP-Optimality

Problem (Optimality Certificate) Input: Channel observation λ and a codeword x ∈ C Question 1: Is x ML-optimal with respect to λ? is it unique? (NP-Hard) Question 2: Is x LP-optimal with respect to λ? is it unique? Relax: one-sided error test A positive answer = certificate for the optimality of x w.r.t. λ A negative answer = don’t know if optimal or not (allow one sided error) “Local-Optimality” criterion: efficient test via local computations

8/23

Definition of Local-Optimality

[Feldman’03] For x ∈ {0, 1}N and f ∈ [0, 1]N ⊆ RN, define the relative point x ⊕ f by (x ⊕ f)i |xi − fi| Consider a finite set of “deviations” B ⊂ [0, 1]N Definition (following [Arora-Daskalakis-Steurer’09]) A codeword x ∈ C is locally-optimal w.r.t. λ ∈ RN if for all vectors β ∈ B, λ, x ⊕ β > λ, x Goal Find a set B of locally-structured devia- tions such that: 1 x ∈ lo(λ) ⇒ x = ml(λ) & unique 2 x ∈ lo(λ) ⇒ x = lp(λ) & unique 3 Prλ{x ∈ lo(λ) | c = x} = 1 − o(1)

λ ML(λ) LP(λ) LO(λ)

9/23

Even Tanner Codes

Definition (Even Tanner codes) Variables nodes have even degree All local codewords have even weight Example LDPC codes with even left degrees Irregular repeat accumulate codes where the repetition factors are even Expander codes with even variable node degrees and even weighted local codes

10/23

Deviations Based on Paths

p is a path of length h: h can be greater than girth, p may be not simple Each path p defines a “characteristic” vector χG(p) ∈ RN [χG(p)]v 1 degG(v) ·

• {v | v ∈ p}
• .

B(h) ⊂ [0, 1]N is the set of deviations B(h) χG(p) h + 1

• p is a backtrackless path of length h
• Example

Y Z b a c d X

p = (a, X, b, Z, a, Y, c, Z, b) h = 8 χG(p) = {2

3, 2 3, 1 3, 0} χG(p) h+1 = { 2 27, 2 27, 1 27, 0}

11/23

Local-Optimality based on h-Paths

Set of deviations B(h) = normalized characteristic vectors of h-paths. B(h) χG(p) h + 1

• p is a backtrackless path of length h
• Definition

A codeword x ∈ C is h-locally optimal w.r.t. λ ∈ RN if for all vectors β ∈ B(h), λ, x ⊕ β > λ, x

12/23

Local Optimality ⇒ Unique ML-codeword

Theorem If x is h-locally optimal w.r.t. λ, then x is the unique ML-codeword w.r.t. λ. Proof method: Lemma (Decomposition Lemma) Every codeword is a conical combination of h-paths in G x = α · Eβ∈ρB(h)[β] Proof of decomposition lemma

1

Every codeword is a conical combination of simple cycles in G

2

Every cycle is a conical combination h-paths in G Following [ADS’09]: decomposition lemma ⇒ unique ML

13/23

Verifying Local Optimality

Hard: Is x the unique ML-codeword? Easy: Is x is locally optimal?

Codeword can be efficiently verified to be locally-optimal w.r.t. λ (dynamic programming / ∼Floyd’s algorithm)

14/23

Local Optimality ⇒ Unique LP optimality

Theorem If x is a h-locally optimal codeword w.r.t. λ, then x is also the unique optimal LP solution given λ. Proof method: reduction to “ML” using graph covers.

Base Graph M-Covering Graph [Vontobel-Koetter’05] ˜ z∗ = ML

• λ↑M

z∗ = LP Opt. In graph covers, realization of LP-Opt and ML codeword are the same

15/23

Local Optimality ⇒ Unique LP optimality

Theorem If x is a h-locally optimal codeword w.r.t. λ, then x is also the unique optimal LP solution given λ. Proof method: reduction to “ML” using graph covers.

Base Graph M-Covering Graph Lemma: Local-optimality is invariant x is locally-optimal w.r.t. λ ˜ x x↑M is locally-optimal w.r.t. λ↑M w.r.t. lifting to covering graphs

16/23

Local Optimality ⇒ Unique LP optimality

Theorem If x is a h-locally optimal codeword w.r.t. λ, then x is also the unique optimal LP solution given λ. Proof method: reduction to “ML” using graph covers.

˜ z∗ = ML

• λ↑M

˜ x x↑M is locally-optimal w.r.t. λ↑M Base Graph M-Covering Graph Thm: Local-Opt

=

⇒ ML Opt.

17/23

Local Optimality ⇒ Unique LP optimality

Theorem If x is a h-locally optimal codeword w.r.t. λ, then x is also the unique optimal LP solution given λ. Proof method: reduction to “ML” using graph covers.

Base Graph M-Covering Graph [Vontobel-Koetter’05] ˜ z∗ = ML

• λ↑M

z∗ = LP Opt. Lemma: Local-optimality is invariant x is locally-optimal w.r.t. λ ˜ x x↑M is locally-optimal w.r.t. λ↑M w.r.t. lifting to covering graphs

=

Thm: Local-Opt ⇒ ML Opt.

=

18/23

Probabilistic Analysis

Symmetry of local-optimality implies: Pr{LP decoding fails} Pr

• ∃β ∈ B(h) s.t. λ, β 0
• c = 0N

. Let D dmax

L

· dmax

R

Bounds rely on: girth(G) > logD(N). [Gallager’63] gives an explicit construction of such graphs. Theorem Consider BSC with crossover probability p. For every ǫ > 0, if p < D

−2·(1+

dmin L dmax L

)·(ǫ+ 3

2+ 1 2 logD(2)), then

Pr{lp(λ) = x | c = x} N −ǫ Analogous theorem derived for the BI-AWGN channel Obtain same results as in [Feldman-Karger’02], [Goldenberg-Burshtein’11] for RA(2) and RA(2q).

19/23

Local-Optimality - Proof Technique

The proof technique in [KV’06], [ADS’09], [HE’10], [HE’11] is based on the following steps:

1

Define a set of deviations. A deviation is induced by combinatorial structures in the Tanner graph or the computation tree.

2

Define local-optimality. Loosely speaking, a codeword x is locally-optimal if its cost is smaller than the cost of every relative point.

3

Local-optimality ⇒ Unique ML-codeword.

Decomposition Lemma: Every codeword is a conical sum of deviations.

4

Local-optimality ⇒ Unique LP-codeword.

Lifting Lemma: Local-optimality is invariant under liftings of codewords to covering graphs.

5

Analyze the probability that there does not exist a locally-optimal codeword.

20/23

Summary

Conclusions Simple application of the proof technique of ”local-optimality” for bounds on the word error probability with LP-decoding

Even Tanner codes (both regular and irregular) Local-optimality: deviations induced by paths in the Tanner graph Inverse polynomial error bounds for the BSC and AWGNC

Unified analysis framework that captures recent advances by [KV’06] [ADS’09] [HE’10] [HE’11] which present inverse exponential bounds on the decoding error probability for regular LDPC codes and Tanner codes. Open questions Extend analysis of inverse exponential error bounds also to irregular Tanner codes Probabilistic analysis beyond the girth

21/23

Advances: Analysis of Finite-Length Codes - LDPC Codes

Form of finite length bounds: ∃c > 1.∃t.∀noise < t. Pr{LP decoder fails} exp(−cgirth) If girth = θ(log N), then Pr{LP decoder fails} exp(−N α), for 0 < α < 1 N → ∞ : t is a lower bound on the threshold of LP-decoding with LO-certificate

[Koetter-Vontobel’06] [Arora-Daskalakis- Steurer’09] [H-Even’10][HE’11] Decoder LP LP LP, Message-Passing Technique Dual LP witness, union bound Primal LP, local opti- mality, rand. min-sum process [ADS’09] + graph cov- ers [VK’05] channels MBIOS BSC MBIOS Example: (3, 6)-reg

BSC(p) threshold: plp > 0.01 BSC(p) threshold: plp > 0.05 AWGN threshold:

Eb N0 lp < 2.67dB

LDPC code

AWGN threshold:

Eb N0 lp < 5.07dB

pBP = 0.084

Eb N0 max−prod ≈ 1.7dB 22/23

Advances: Analysis of Finite-Length Codes - Tanner Codes

Form of finite length bounds: ∃c > 1.∃t.∀noise < t. Pr{LP decoder fails} exp(−cgirth) If girth = θ(log N), then Pr{LP decoder fails} exp(−N α), for 0 < α < 1 N → ∞ : t is a lower bound on the threshold of LP-decoding with LO-certificate

[Skachek-Roth’03] [Feldman-Stein’05] [H-Even’11] Decoder Iterative LP LP Channels Bit-flipping (worst- case) Bit-flipping (worst- case) MBIOS (average- case) Technique Expansion Expansion local-optimality + [Von’10] + sum-min- sum rand. process Example: BSC(p) threshold dR >> 2 dR >> 2 dR = 16 (2, dR)-reg Tan- ner code, d∗ >> 2 d∗ >> 2 d∗ = 4 Rate=0.375

• piterat. > 0.0016

plp > 0.0008 plp > 0.044

23/23