Error Exponents, Weight and Magnetization Enumerators in LDPC Error - - PowerPoint PPT Presentation

▶
error exponents weight and magnetization enumerators in
SMART_READER_LITE
LIVE PREVIEW

Error Exponents, Weight and Magnetization Enumerators in LDPC Error - - PowerPoint PPT Presentation

May 23, 2023 100 likes •306 views

Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes http://www.ncrg.aston.ac.uk/ D. Saad J. van Mourik, N. Skantzos, Y. Kabashima Aston University & TITECH 11 July 2003 Hayashibara Forum 11 July 2003

slide-1
SLIDE 1

Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes

http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos, Y. Kabashima

Aston University & TITECH 11 July 2003 Hayashibara Forum

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 1) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-2
SLIDE 2

Outline ✦ We briefly introduce LDPC error correcting codes and their relation to statistical physics ✦ Studying the entropy of solutions we provide an intuitive description relating MAP, MPM and typical set decoding ✦ We compare the critical noise level results obtained with those derived in the IT literature ✦ We will discuss the role of average error and reliability exponents and derive results for specific cases ✦ Finally, we will present some open questions

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 2) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-3
SLIDE 3

The General Communication Framework

✦ The message is an N dimensional binary vector s. ✦ Encoded to an M dimensional Boolean (binary) vector t and transmitted through a noisy channel (BSC). The code rate is R = N/M. ✦ The received message r = t + n (mod 2) is decoded to retrieve s. ✦ Error-free communication is theoretically possible below Shannon’s bound Rc = 1 + p log2 p + (1 − p) log2(1 − p).

Encoding Decoding Transmission Noise

t r s s

Message

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 3) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-4
SLIDE 4

Regular LDPC/MN Codes

✦ Gallager’s code (1962) – Construct two sparse matrices A and B of dimensionality (M − N)×N and (M − N)×(M − N) respectively. ✦ MN code (1996) – Construct two sparse matrices A and B of dimensionality M ×N and M ×M respectively. ✦ The matrix A has K non-zero (unit) elements per row and C per column, B has L per row/column.

A =

                   1 . . . 1 . . . . . . . . . . . . ...                 

  • −

− − − − − − − − − − − − − − → B =

                   1 1 . . . 1 . . . 1 . . . 1 . . . . . . . . . ...                 

  • −

− − − − − − − − − − − − − − − − − − − − →

˙ M M N M C K L L In Gallager’s code we may refer to A = [A | B] and K + L only.

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 4) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-5
SLIDE 5

Gallager’s Code

Encoding t

  • ∈{0,1}M

= GT s

  • ∈{0,1}N

(mod 2). The (dense) generator matrix – G = [I | B−1A] (mod 2). The received vector – r = GT s + n (mod 2). Decoding z = Ar = AGT s

=0

+An (mod 2) , A = [A | B] sparse M ×M binary matrix. The problem: z = Aτ (mod 2) . ✦ MPM – nj = argmax P(τj | z) minimises pb = 1 −

1 M

M

j=1 δ[

nj; nj]. ✦ MAP – n = argmax P(τ | z) minimises pB.

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 5) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-6
SLIDE 6

MN Code

Encoding t

  • ∈{0,1}M

= GT s

  • ∈{0,1}N

(mod 2). The (dense) generator matrix – G = [B−1A] (mod 2). The received vector – r = GT s + n (mod 2). Decoding z = Br = As + Bn (mod 2) . The problem: z = Aσ + Bτ (mod 2) . ✦ MPM – sj = argmax P(σj | z) , nj = argmax P(τj | z) . ✦ MAP – minimises block error pB.

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 6) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-7
SLIDE 7

Statistical Physics - Gallager

✦ Mapping from Boolean to binary variables {0, 1, +} → {1, −1, ×} . ✦ We look for the conditional probability P(τ | z) = 1 Z δ (z; Aτ) P(τ)

prior

= 1 Z exp [−βH(τ; z)] ✦ The ground state of the Hamiltonian H =

  • µ

χ

  • zµ = [Aτ]µ
  • − F

M

  • j=1

τj, corresponds to the Bayes-optimal estimate of the noise n. ✦ The parity checks χ

  • zµ = [Aτ]µ
  • = 0 if the parity check is obeyed

by the vector τ and χ (·) = ∞ otherwise. ✦ F = (1/2) ln[(1 − p)/p] is the prior knowledge about the noise n.

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 7) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-8
SLIDE 8

The Free Energy

✦ One can then calculate the free energy 1 M ln Z{A,n} ; Z (A, n) = Tr{τ }exp [−βH] using the replica methoda or the Bethe approximationb. ✦ Free energy -related to probability, self-averaging, extensive, typical, quenched vs. annealed averages. ✦ The choice of 1/T = β = 1 is finite temperature decoding (Nishimori’s condition) - MPM. ✦ Typical Solutions: – Ferro solution (m = 1): Perfect decoding – Para solution (m = 0): Catastrophic failure – Sub-optimal ferro solution (m < 1): failure where m= 1

M

M

i=1 niˆ

ni

aKabashima, Murayama, Saad PRL 84 1355 (2000) bVicente, Saad, Kabashima EPL 51 698 (2000)

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 8) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-9
SLIDE 9

Weight & Magnetisation Enumerators

✦ To examine different decoding methods within this framework we calculate the entropy. ✦ The magnetisation enumerator M(m) can be calculated from the Hamiltonian H =

  • µ

χ

  • zµ = [Aτ]µ
  • − δ

  1 M

M

  • j=1

τj − m   , by forcing the magnetisation constraint. ✦ To calculate the weight enumerator W(ω) we use a different constraint δ

  • 1

M

M

j=1 njτj − ω

  • .

✦ The entropies can be calculated using the replica method -

1 M ln Z{A,n}

  • r

annealed approximation -

1 M ln Z{A,n} .

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 9) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-10
SLIDE 10

Decoding

✦ Weight enumerator - independent of n after gauging, annealed approximation = replica result W(m) = Ma(m)|p=0 = Mq(m)|p=0 ✦ Magnetisation enumerator M(m) - different results for annealed approximation/replica method. Decoding ✦ Maximum likelihood (MAP) decoding - selects solution with highest magnetisation. ✦ Typical pairs decoding - selecting a (single) vector from typical set. ✦ Finite temperature (MPM) decoding - energy term −Fm added, solution chosen by minimising the free energy.

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 10) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-11
SLIDE 11

Entropy and Decoding

  • ✁
✂ ✄ ✂✆☎ ✝ ✞✠✟ ✁ ✟ ✟ ✡ ✞ ✂ ✁ ☛ ☞ ☞ ✌ ✁ ✂ ✍ ✂✆☎ ✝ ✞ ✟ ✁ ✟ ✟ ✡ ✞ ✂ ✁ ☛ ☞ ☞ ✎ ✁ ✂ ✏ ✂ ☎ ✝ ✞ ✟ ✁ ✟ ✟ ✡ ✞ ✂ ✁ ☛ ☞ ☞ ✟ ✑ ✞ ✂ ✁ ✟ ✑ ✞ ✂ ✁ ✟ ✑ ✞ ✂ ✁

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 11) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-12
SLIDE 12

Critical Noise Level - Gallager

(K, C) (6, 3) (5, 3) (6, 4) (4, 3) Code rate 1/2 2/5 1/3 1/4 IT (Wa) 0.0915 0.129 0.170 0.205 SP 0.0990 0.136 0.173 0.209 pc,a (Ma) 0.031 0.066 0.162 0.195 pc,q (Mq) 0.0998 0.1365 0.1725 0.2095 Shannon 0.109 0.145 0.174 0.214

1 m p pc,a pc,q m0(p) m+,a(p) m+,q(p) 0.5

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 12) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-13
SLIDE 13

Critical Noise Level - MN

(K, C, L) (1, 3, 2) (2, 6, 2) (2, 3, 2) (3, 9, 3) Code rate 1/3 1/3 2/3 1/3 pc,q (Mq) 0.15 ≈ 0.174 0.06 0.174 Shannon pt 0.174 0.174 0.0615 0.174

1 m p pc,q = pt m0(p) m+,q(p) = m0(pt) 0.5

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 13) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-14
SLIDE 14

Error Exponents

✦ Block error probability below the critical noise level (MAP) - PB(p) ∝ e−MQ(p) ✦ Average exponenta - Averaged over the code ensemble ✦ Reliability exponent- Averaged over best codes in the ensemble ✦ Indicator function (0/1)- ∆(n, A) = lim

β→∞

  • τ ∈Ipc(n,A)\n e−βH(τ )
  • τ ∈Ipc(n,A) e−βH(τ )

H(n) = −FMm(n) and Ipc(n, A) the set of vectors τ that obey all parity checks ✦ AE/RE- Qr = lim

M→∞

1 Mr log[∆(n, A)n]rA r = 1 corresponds to AE and r → −∞ to RE

aSkantzos, van Mourik, Saad, Kabashima JPA in press (2003)

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 14) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-15
SLIDE 15

Average Error Exponent - Gallager

0.16 0.18 0.2 0.22 0.002 0.004 0.006 0.008 0.06 0.08 0.1 0.12 0.005 0.01 0.015

Q Q p p R = 1/4 R = 1/2

(4, 3) (6, 3) (K, C) → ∞ (K, C) → ∞

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 15) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-16
SLIDE 16

Average Error Exponent - Gallager vs. MN

0.1 0.15 0.2 0.02 0.04 0.06 0.06 0.08 0.1 0.12 0.005 0.01 0.015 0.02

Q Q p p R = 1/4 R = 1/2

(1, 4, 2) (6, 3)

(2, 8, 3), (K, C, L) → ∞ (3, 6, 3), (K, C, L) → ∞

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 16) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-17
SLIDE 17

Reliability Exponent

✦ Reliability expa: Qr = lim(−)r,M→∞

1 Mr log[∆(n, A)n]rA

✦ For K, C, L → ∞ RE of MN and Gallager codes become identical

0.2 0.4 0.6 0.2 0.4 0.6 0.8 1

Q/Qr R | |

  • ×

×

aSkantzos, van Mourik, Saad, Kabashima JPA in press (2003)

Kabashima, Sazuka, Nakamura, Saad PRE 64 046113 (2001)

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 17) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-18
SLIDE 18

Conclusions ✦ We studied properties of LDPC codes using magnetisation and weight enumerator techniques for MN/Gallager and finite connectivity codes ✦ Results in agreement with those obtained before using SP, more optimistic than those obtained in the IT literature ✦ Analysis of average error exponents allows for the derivation of typical finite connectivity results ✦ Results for infinite connectivity agree with IT bounds and previous SP ’typical bounds’

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 18) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima
slide-19
SLIDE 19

Open Questions ✦ MN vs. Gallager - SP vs. IT results ✦ Reliability exponents for finite connectivity codes ✦ RSB in AE/RE ✦ New decoding techniques

11 July 2003 Hayashibara Forum Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes (page 19) http://www.ncrg.aston.ac.uk/

  • D. Saad
  • J. van Mourik, N. Skantzos,
  • Y. Kabashima