Error Exponents, Weight and Magnetization Enumerators in LDPC Error Correcting Codes http://www.ncrg.aston.ac.uk/ D. Saad J. van Mourik, N. Skantzos, Y. Kabashima Aston University & TITECH 11 July 2003 Hayashibara Forum 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 1) Y. Kabashima http://www.ncrg.aston.ac.uk/
Outline ✦ We briefly introduce LDPC error correcting codes and their relation to statistical physics ✦ Studying the entropy of solutions we provide an intuitive description relating MAP, MPM and typical set decoding ✦ We compare the critical noise level results obtained with those derived in the IT literature ✦ We will discuss the role of average error and reliability exponents and derive results for specific cases ✦ Finally, we will present some open questions 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 2) Y. Kabashima http://www.ncrg.aston.ac.uk/
The General Communication Framework ✦ The message is an N dimensional binary vector s . ✦ Encoded to an M dimensional Boolean (binary) vector t and transmitted through a noisy channel (BSC). The code rate is R = N/M . ✦ The received message r = t + n (mod 2) is decoded to retrieve s . ✦ Error-free communication is theoretically possible below Shannon’s bound R c = 1 + p log 2 p + (1 − p ) log 2 (1 − p ) . Message Noise s t r s Encoding Transmission Decoding 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 3) Y. Kabashima http://www.ncrg.aston.ac.uk/
Regular LDPC/MN Codes ✦ Gallager’s code (1962) – Construct two sparse matrices A and B of dimensionality ( M − N ) × N and ( M − N ) × ( M − N ) respectively. ✦ MN code (1996) – Construct two sparse matrices A and B of dimensionality M × N and M × M respectively. ✦ The matrix A has K non-zero (unit) elements per row and C per column, B has L per row/column. N M � �� � � �� � 1 0 . . . 1 0 0 1 . . . 0 1 . . . 0 0 1 0 . . . M M 0 0 . . . 0 0 0 1 . . . C L A = B = 0 0 . . . 0 1 0 0 . . . . . . ... ... . . . . � . . � − − − − − − − − − − − − − − − → − − − − − − − − − − − − − − − − − − − − − → K L ˙ In Gallager’s code we may refer to A = [ A | B ] and K + L only. 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 4) Y. Kabashima http://www.ncrg.aston.ac.uk/
Gallager’s Code = G T Encoding (mod 2) . t s ���� ���� ∈{ 0 , 1 } M ∈{ 0 , 1 } N G = [ I | B − 1 A ] (mod 2) . The (dense) generator matrix – r = G T s + n (mod 2) . The received vector – z = A r = A G T s Decoding + A n (mod 2) , � �� � =0 A = [ A | B ] sparse M × M binary matrix. The problem: z = A τ (mod 2) . ✦ MPM – � n j = argmax P ( τ j | z ) minimises � M 1 p b = 1 − j =1 δ [ � n j ; n j ] . M ✦ MAP – � n = argmax P ( τ | z ) minimises p B . 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 5) Y. Kabashima http://www.ncrg.aston.ac.uk/
MN Code = G T Encoding t s (mod 2) . ���� ���� ∈{ 0 , 1 } M ∈{ 0 , 1 } N G = [ B − 1 A ] (mod 2) . The (dense) generator matrix – r = G T s + n (mod 2) . The received vector – Decoding z = Br = As + Bn (mod 2) . The problem: z = Aσ + Bτ (mod 2) . ✦ MPM – � s j = argmax P ( σ j | z ) , � n j = argmax P ( τ j | z ) . ✦ MAP – minimises block error p B . 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 6) Y. Kabashima http://www.ncrg.aston.ac.uk/
Statistical Physics - Gallager ✦ Mapping from Boolean to binary variables { 0 , 1 , + } → { 1 , − 1 , ×} . ✦ We look for the conditional probability P ( τ | z ) = 1 = 1 Z δ ( z ; A τ ) P ( τ ) Z exp [ − β H ( τ ; z )] � �� � prior ✦ The ground state of the Hamiltonian � � M � � H = χ z µ = [ A τ ] µ − F τ j , µ j =1 corresponds to the Bayes-optimal estimate of the noise n . � � ✦ The parity checks χ z µ = [ A τ ] µ = 0 if the parity check is obeyed by the vector τ and χ ( · ) = ∞ otherwise. ✦ F = (1 / 2) ln[(1 − p ) /p ] is the prior knowledge about the noise n . 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 7) Y. Kabashima http://www.ncrg.aston.ac.uk/
The Free Energy ✦ One can then calculate the free energy 1 M � ln Z� {A , n } ; Z ( A , n ) = Tr { τ } exp [ − β H ] using the replica method a or the Bethe approximation b . ✦ Free energy -related to probability, self-averaging, extensive, typical, quenched vs. annealed averages. ✦ The choice of 1 /T = β = 1 is finite temperature decoding (Nishimori’s condition) - MPM. ✦ Typical Solutions: – Ferro solution ( m = 1 ): Perfect decoding – Para solution ( m = 0 ): Catastrophic failure – Sub-optimal ferro solution ( m < 1 ): failure � M where m = 1 i =1 n i ˆ n i M a Kabashima, Murayama, Saad PRL 84 1355 (2000) b Vicente, Saad, Kabashima EPL 51 698 (2000) 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 8) Y. Kabashima http://www.ncrg.aston.ac.uk/
Weight & Magnetisation Enumerators ✦ To examine different decoding methods within this framework we calculate the entropy. ✦ The magnetisation enumerator M ( m ) can be calculated from the Hamiltonian � � M � � 1 , H = χ z µ = [ A τ ] µ − δ τ j − m M µ j =1 by forcing the magnetisation constraint. ✦ To calculate the weight enumerator W ( ω ) we use a different � � � M 1 constraint δ j =1 n j τ j − ω . M ✦ The entropies can be calculated using the 1 replica method - M � ln Z� {A , n } or 1 annealed approximation - M ln �Z� {A , n } . 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 9) Y. Kabashima http://www.ncrg.aston.ac.uk/
Decoding ✦ Weight enumerator - independent of n after gauging, annealed approximation = replica result W ( m ) = M a ( m ) | p =0 = M q ( m ) | p =0 ✦ Magnetisation enumerator M ( m ) - different results for annealed approximation/replica method. Decoding ✦ Maximum likelihood (MAP) decoding - selects solution with highest magnetisation. ✦ Typical pairs decoding - selecting a (single) vector from typical set. ✦ Finite temperature (MPM) decoding - energy term − Fm added, solution chosen by minimising the free energy. 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 10) Y. Kabashima http://www.ncrg.aston.ac.uk/
✏ ✞ ✎ ✁ ✂ � ✂ ☎ ✝ ✞ ✟ ✁ ✟ ✟ ✡ ✂ ☞ ✁ ☛ ☞ ☞ ✟ ✑ ✞ ✂ ✁ ✟ ✑ ✞ ✂ ✁ ☞ ☛ ✑ ☞ ✁ ✂ ✄ ✁ ✝ ✂ ✁ ✟ ✟ ✡ ✞ ✂ ✁ ☛ ☞ ✁ ✌ ✁ ✂ ✍ ✞ ✝ ✞ ✟ ✁ ✟ ✟ ✡ ✞ ✂ ✟ Entropy and Decoding ✞✠✟ ✂✆☎ ✂✆☎ 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 11) Y. Kabashima http://www.ncrg.aston.ac.uk/
Critical Noise Level - Gallager ( K, C ) (6 , 3) (5 , 3) (6 , 4) (4 , 3) Code rate 1 / 2 2 / 5 1 / 3 1 / 4 IT ( W a ) 0.0915 0.129 0.170 0.205 SP 0.0990 0.136 0.173 0.209 p c,a ( M a ) 0.031 0.066 0.162 0.195 m p c,q ( M q ) 0.0998 0.1365 0.1725 0.2095 Shannon 0.109 0.145 0.174 0.214 1 m + ,a ( p ) m + ,q ( p ) p c,a p c,q p 0 0 . 5 m 0 ( p ) 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 12) Y. Kabashima http://www.ncrg.aston.ac.uk/
Critical Noise Level - MN ( K, C, L ) (1 , 3 , 2) (2 , 6 , 2) (2 , 3 , 2) (3 , 9 , 3) Code rate 1 / 3 1 / 3 2 / 3 1 / 3 p c,q ( M q ) 0 . 15 ≈ 0 . 174 0 . 06 0 . 174 m Shannon p t 0 . 174 0 . 174 0 . 0615 0 . 174 1 m + ,q ( p ) = m 0 ( p t ) p c,q = p t p 0 0 . 5 m 0 ( p ) 11 July 2003 Error Exponents, Weight and Magnetization Enumerators in LDPC Error D. Saad J. van Mourik, N. Skantzos, Hayashibara Forum Correcting Codes (page 13) Y. Kabashima http://www.ncrg.aston.ac.uk/
Recommend
More recommend
