LP Decoding of Regular LDPC Codes in Memoryless Channels Nissim - - PowerPoint PPT Presentation

SLIDE 1

1

LP Decoding of Regular LDPC Codes in Memoryless Channels

ISIT 2010

Nissim Halabi Guy Even

SLIDE 2

2

Low-Density Parity-Check Codes

Factor graph representation of LDPC codes:

Code C(G) and codewords x : Local-codes Cj = Cj (G) : (dL,dR)-regular LDPC code :

Variable nodes Check nodes

1

c

2

c

3

c

5

c

4

c

10

x

1

x

3

x

2

x

4

x

5

x

6

x

7

x

8

x

9

x

( ) ( )

( )

. 0 mod 2

i j

j i x N c

x G c x

∈

∈ ⇔ ∀ =

∑

C

( )

( )

0 mod 2

i j

j i x N c

x x

∈

∈ ⇔ =

∑

C

( )

• Variables. degG

L

v v d ∀ ∈ =

( )

• Checks. degG

R

c c d ∀ ∈ =

(2,4)-regular LDPC code

SLIDE 3

3

{ }

0,1

k

u ∈

Maximum-Likelihood (ML) Decoding

Log-likelihood ratio (LLR) λi for a received observation yi:

Any memoryless binary-input output-symmetric (MBIOS) channel can be described by an LLR function.

Maximum-likelihood (ML) decoding for any binary-input memory-less channel:

( ) ( ) ( )

/ /

/ ln / 1

i i i i

Y X i i i i Y X i i

y x y y x λ   = =     =    

( ) ( )

ˆ arg min ,

ML

x

x y y x λ

∈

=

C

MBIOS Channel Channel Decoding Channel Encoding

{ }

ˆ 0,1

n

x∈

{ }

0,1

n

x∈

n

y ∈

SLIDE 4

4

Linear Programming (LP) Decoding

Maximum-likelihood (ML) decoding formulated as a linear program:

( ) ( )

( )

( )

ˆ arg min , arg min ,

ML

x x

x y y x y x λ λ

∈ ∈

= =

conv C C

C

conv(C)

SLIDE 5

5

Linear Programming (LP) Decoding

Maximum-likelihood (ML) decoding formulated as a linear program: Linear Programming (LP) decoding [Fel03, FWK05] – relaxation of the polytope conv(C)

( ) ( )

check nodes

( )

ˆ arg min ,

LP

j

x j

x y y x λ

∈

=

conv  C

C

conv(C) conv(Cj)

( ) ( )

( )

( )

ˆ arg min , arg min ,

ML

x x

x y y x y x λ λ

∈ ∈

= =

conv C C

SLIDE 6

6

Linear Programming (LP) Decoding

Maximum-likelihood (ML) decoding formulated as a linear program: Linear Programming (LP) decoding [Fel03, FWK05] – relaxation of the polytope conv(C)

ˆLP x integral  success! We also know ˆLP x = ˆML x  C (“ML certificate”)

ˆLP x fractional  fail

Solve LP

( ) ( )

( )

( )

ˆ arg min , arg min ,

ML

x x

x y y x y x λ λ

∈ ∈

= =

conv C C

( ) ( )

check nodes

( )

ˆ arg min ,

LP

j

x j

x y y x λ

∈

=

conv  C

SLIDE 7

7

Previous Bounds for LP Decoding (1)

No tree assumption! ⇒ Bounds relevant for finite lengths Bounds for specific families of codes:

Cycle codes / RA(2) codes over memoryless channels [FK02,HE03]. Expander LDPC codes over bit flipping channels (e.g., BSC, adversarial) [FMSSW04, DDKW07]. Capacity achieving binary expander codes over memoryless channels [FS05]. Non-binary expander codes [Ska09].

SLIDE 8

8

Previous Bounds for LP Decoding (2)

(dL,dR)-regular LDPC codes [KV06, ADS09]

• Form of finite length bounds: ∃ c > 1. ∃ t. ∀ noise < t.

Pr(LP decoder success)  1 - exp(-cgirth)

• If girth = Θ(logn), then

Pr(LP decoder success)  1 - exp(-nγ), for 0 <γ < 1 n → ∞ : t is a lower bound on the threshold of LP decoding

SLIDE 9

9

Previous Bounds for LP Decoding (2)

(dL,dR)-regular LDPC codes [KV06, ADS09]

• Form of finite length bounds: ∃ c > 1. ∃ t. ∀ noise < t.

Pr(LP decoder success)  1 - exp(-cgirth)

• If girth = Θ(logn), then

Pr(LP decoder success)  1 - exp(-nγ), for 0 <γ < 1 n → ∞ : t is a lower bound on the threshold of LP decoding

Koetter and Vontobel ’06

Dual witness technique Memoryless channels BSC(p) threshold: pLP > 0.01 BI-AWGNC(σ) threshold: σLP > 0.5574 Eb/N0

LP < 5.07dB

Channels Technique Example for (3,6)-regular LDPC code

σMax-Product = 0.8223 Eb/N0

Max-Product ~1.7dB

SLIDE 10

10

Previous Bounds for LP Decoding (2)

(dL,dR)-regular LDPC codes [KV06, ADS09]

• Form of finite length bounds: ∃ c > 1. ∃ t. ∀ noise < t.

Pr(LP decoder success)  1 - exp(-cgirth)

• If girth = Θ(logn), then

Pr(LP decoder success)  1 - exp(-nγ), for 0 <γ < 1 n → ∞ : t is a lower bound on the threshold of LP decoding

Koetter and Vontobel ’06

Dual witness technique Memoryless channels BSC(p) threshold: pLP > 0.01 BI-AWGNC(σ) threshold: σLP > 0.5574 Eb/N0

LP < 5.07dB

Arora, Daskalakis and Steurer ’09

Primal LP analysis BSC BSC(p) threshold: pLP > 0.05

Channels Technique Example for (3,6)-regular LDPC code

σMax-Product = 0.8223 Eb/N0

Max-Product ~1.7dB

pBP= 0.084

SLIDE 11

11

Our Results

Extension of ADS’09 from BSC to MBIOS channels: Combinatorial characterization: Local Opt. ⇒ LP Opt. Alternative proofs using graph covers [VK05] Finite length bound: decoding errors decrease doubly exponential in the girth of the factor graph

Example: for (3,6)-regular LDPC code,  σ  0.605 for some constant c < 1.

Lower bound on thresholds of LP decoding for regular LDPC codes

Analytic bounds for MBIOS “Density evolution” bounds on thresholds for BI-AWGNC

Example: for (3,6)-regular LDPC code BI-AWGNC(σ) threshold: σLP > 0.735

(σ Max-Product = 0.8223) Eb/N0

LP < 2.67dB (Eb/N0 Max-Product ~1.7dB)

2

3 2

1 4 2

1 125

err

g

P e n c

σ

     

< ⋅

SLIDE 12

12

Skinny Trees Embedded in Factor Graphs

Consider a subgraph τ of G:

root = v0 ∈ VL τ ⊆ Ball(v0 , 2h) ∀ v ∈ τ∩VL : deg τ (v) = deg G (v). ∀ c ∈ τ∩VR : deg τ (c) = 2.

girth(G) > 4h ⇒ τ is a tree – Skinny Tree Moreover, in a dL left regular graph all skinny trees are isomorphic to:

v0

τ, h=1

dL dL-1

dL-1

vars. checks

SLIDE 13

13

Cost of a Weighted Skinny Tree [ADS09]

Given layer weights  :  → , define -weighted skinny tree τ of height 2h Given assignment of LLR values λ to variable nodes, define the cost of an -weighted skinny tree τ

( )

2

1

,

l

h l v l v V

valω

τ

τ λ ω λ

− = ∈ ∩

⋅

∑ ∑



ω

dL dL-1

1

ω

weights

2

ω

2h

SLIDE 14

14

Local optimality – sufficient condition for the (global)

• ptimality of a decoded codeword based on skinny

trees Task: bound the probability that there exists a weighted skinny tree with non-positive cost.

Proving Error Bounds using Local Optimality

[following ADS09]

Theorem: Fix

( )

1 4

h girth G < and

h

ω

+

∈ . Then

{ } ( )

{ }

LP decoding fails skinny tree . , 0 | 0n val x

ω

τ τ λ ≤ ∃ ≤ =   .

All-Zeros Assumption

SLIDE 15

15

Computing [min val(τ;λ)  0]

 – induced graph of factor graph G on Ball(v0 , 2h) {γ} – values associated with variable nodes. Yl – variable nodes of  at height 2l. Xl – check nodes of  at height 2l+1.

• Dyn. Prog. recurrence for computing min cost skinny tree in  :

 for (3,6)-regular graph, h=2 v0

Basis: leaves: Y ω γ = Step: checks:

( ) ( )

{ }

1 1

min ,...,

R

d l l l

X Y Y

−

= vars:

( ) ( )

1 1 1 1

...

L

d l l l l

Y X X ω γ

− − −

= + + +

X1 τ

SLIDE 16

16

Computing [min val(τ;λ)  0]

 – induced graph of factor graph G on Ball(v0 , 2h) {γ} – values associated with variable nodes. Yl – variable nodes of  at height 2l. Xl – check nodes of  at height 2l+1.

• Dyn. Prog. recurrence for computing min cost skinny tree in  :

Process: let {γ} = components of LLR random vector λ.

 for (3,6)-regular graph, h=2 v0

BI-AWGN(σ ) + all zeros assumption: 1

i i

λ φ = + where

( )

2

0,

i

φ σ   . Basis: leaves: Y ω γ = Step: checks:

( ) ( )

{ }

1 1

min ,...,

R

d l l l

X Y Y

−

= vars:

( ) ( )

1 1 1 1

...

L

d l l l l

Y X X ω γ

− − −

= + + +

τ

SLIDE 17

17

Density Evolution Based Bound for BI-AWGNC(σ)

Theorem: Let G denote a (dL,dR)-regular bipartite graph with girth Ω(log n), and let C(G) denote the LDPC code defined by

• G. Consider the BI-AWGNC(σ). Then, LP decoding

succeeds with probability at least 1 – exp(–nγ) for some constant 0 < γ < 1, provided that: (1) s < ¼ girth(G) , and (2) Condition (2) holds for σ < σ0, where

σ0 = Lower bound on the

threshold of LP decoding

R.V. – min cost of a skinny tree with height s

SLIDE 18

18

Probability density functions of Xl for l = 0,…,4 (dL,dR) = (3,6), and σ = 0.7.

Numeric computation based on quantization following methods used in implementations of density evolution

Gaussian PDFs’ Evolution

Y ω γ =

( ) ( )

{ }

1 1

min ,...,

R

d l l l

X Y Y

−

=

( ) ( )

1 1 1 1

...

L

d l l l l

Y X X ω γ

− − −

= + + +

SLIDE 19

19

Threshold bound values for finite s, (dL,dR)=(3,6)

Eb/N0 [dB] σ0 s 4.36 0.605 3.94 0.635 1 3.61 0.66 2 3.41 0.675 3 3.29 0.685 4 3.1 0.7 6 2.91 0.715 10 2.67 0.735 22

s = 4 Max-Product threshold: σ = 0.82, Eb/N0 ~ 1.7dB

SLIDE 20

20

Summary

Extended analysis of ADS’09 to MBIOS channels:

We saw a sketch of one of the main results:

Bound on the threshold of LP decoding for regular LDPC codes with log girth over BI-AWGNC.

“Density evolution” bounds: a step towards closing the gap to BP-based threshold

More in the paper:

Reformulations of some results of ADS’09 in terms of graph covers [VK’05] Combinatorial characterization: Local Opt. ⇒ LP Opt. Derivation of finite length bound

“LP Decoding of Regular LDPC Codes in Memoryless Channels” @ arXiv

SLIDE 21

21

Future Directions

Further understanding the gap to BP-based algorithms thresholds For BI-AWGNC, applying Guassian approximation techniques to “density evolution” bound ⇒ Better thresholds (?) Vontobel [Von10] generalized the geometrical aspects of ADS’09 via normal graphs. Can a modified “DE style” analysis improve performance guarantees?

SLIDE 22

22

Thank You!