Finite-Blocklength and Error-Exponent Analyses for LDPC Codes in - PowerPoint PPT Presentation

Finite-Blocklength and Error-Exponent Analyses for LDPC Codes in Point-to-Point and Multiple Access Communication* Yuxin Liu and Michelle Effros Department of Electrical Engineering, California Institute of Technology June 21-26, 2020 IEEE International Symposium on Information Theory * Extended version available on arXiv <2005.06428>. This material is based upon work supported in part by the National Science Foundation under Grant No. 1817241. The work of Y. Liu is supported in part by the Oringer Fellowship Fund in Information Science and Technology. 1

Background • Introduced by [Gallager’62] • Low density enables low-complexity decoding • Commercial Standards: Ethernet (IEEE 803.3an), WiFi (IEEE 802.11n), WiMAX (IEEE 802.16e), 5G Code 𝑌 𝑍 Channel • [Davey and MacKay ’98] binary → 𝐻𝐺 𝑟 • [Richardson and Urbanke ’01] d ensity evolution • [Bennatan and Burshtein ’04] arbitrary DMC • [Erez and Miller ’05] modulo-additive channel • [Di et al.’02 ] finite-blocklength in BEC • Many more. 2

Summary of LDPC Results • These analyses capture the tradeoff between rate 𝑆 , error probability 𝜗 , and blocklength 𝑜 : – Error-Exponent: most accurate for 𝜗 small – Dispersion: most accurate for 𝑜 small – Scaling Law: useful under iterative decoding 3

Outline • Code Construction • Error-Exponent Analysis • Finite-Blocklength (Dispersion-style) Analysis • Main Takeaways 4

Code Construction • Follow the construction from [Bennatan and Burshtein’04 ] 5

Code Construction • Different from the design of (𝜇, 𝜍, 𝑜, 𝑟) → 𝑆 ≜ 1 − 𝜇 • 𝜍 Gallager ensemble • Originated from [Luby et al.’01] 1 • Socket values uniformly from 2 1 … 2 𝐻𝐺 𝑟 \{0} 𝜇 … 𝜍 • Expurgation: remove codes with small minimum distance 𝑜 𝑠 = 𝑜(1 − 𝑆) … … • 𝑠 − 1 𝜍 + 1 Uniform Codeword Selection: 𝑠 − 1 𝜍 + 2 … Restrict the operational rate to 𝑜 − 1 𝜇 + 1 𝑠𝜍 𝑜 − 1 𝜇 + 2 … equal to 𝑆 𝑜𝜇 6

Code Construction • Coset Vector: – Uniformly and Independently over 𝐻𝐺(𝑟) – Fixed across codewords • Quantizer: – Mapping 𝐻𝐺 𝑟 → Channel Input alphabet component-wise 7

Decoding Strategy • Maximum likelihood (ML) decoding • Why not iterative: – Distinguish decoding performance penalty from encoding performance penalty – Iterative decoding next step 8

Summary of Our Results for LDPC Codes Error-Exponent Dispersion-style Point-to-Point Channel [Bennatan and Burshtein ’04] (PPC) Symmetrical MAC General MAC 9

MAC LDPC Code • For each transmitter: Different Symmetrical Same LDPC Same Coset MAC Encoder Quantizer Vectors Different Different Different Asymmetrical LDPC Coset Quantizers MAC Encoders Vectors • Symmetrical: 𝒴 1 = 𝒴 2 = ⋯ = 𝒴 𝐿 , 𝑄 𝑍 𝑌 1 , 𝑌 2 , … 𝑌 𝐿 = 𝑄 𝑍 𝜌 𝑌 1 , 𝑌 2 , … 𝑌 𝐿 10

Error-Exponent Analysis Tx 1: 𝜇 1 , 𝜍 1 , 𝑟, 𝑜 → 𝑆 1 = 1 − 𝜇 1 • 𝜍 1 , 𝜀 1 ⋅ → 𝑄 𝑌 1 Tx 2: 𝜇 2 , 𝜍 2 , 𝑟, 𝑜 → 𝑆 2 = 1 − 𝜇 2 • 𝜍 2 , 𝜀 2 ⋅ → 𝑄 𝑌 2 11

Error-Exponent Analysis • Expected error probability in a general discrete memoryless 2-MAC – Follow the approach [Bennatan and Burshtein’04 ] for PPC • 𝐹 𝑞 12 ⋅ : Gallager’s error • 𝐹 𝑞 1 ⋅ , 𝐹 𝑞 2 (⋅) : Gallager’s error exponent exponents • 𝛽 12 = 𝛽 1 𝛽 2 : encoder penalty • 𝑟, 𝑜, 𝑆 1 , 𝑆 2 are design for decoding both messages parameters • incorrectly 𝛽 1 , 𝛽 2 : LDPC encoder penalties for transmitters 1 and 2 Ensemble-average error probability 12

𝛽 1 , 𝛽 2 , 𝛽 12 • Penalties for LDPC encoder, 𝛽 12 = 𝛽 1 𝛽 2 , 𝑗 ∈ {1,2} 𝑈𝑠𝑏𝑜𝑡𝑛𝑗𝑢𝑢𝑓𝑠−𝑗 # 𝑀𝐸𝑄𝐷 𝑓𝑜𝑡𝑓𝑛𝑐𝑚𝑓 𝑢𝑧𝑞𝑓−𝑢 𝑤𝑓𝑑𝑢𝑝𝑠𝑡 𝛽 𝑗 ∝ max 𝑜 # 𝑢𝑧𝑞𝑓−𝑢 𝑤𝑓𝑑𝑢𝑝𝑠𝑡 𝑢 ∈ 𝒰 𝑟 – LDPC Ensemble for transmitter 𝑗 refers to the ensemble from “LPDC Encoder” with design parameters (𝜇 𝑗 , 𝜍 𝑗 , 𝑜, 𝑟) – Type is the relative frequency of each symbol in 𝐻𝐺(𝑟) 𝑜 is the set of possible types with 𝑜 𝐻𝐺(𝑟) symbols – 𝒰 𝑟 log 𝛽 𝑗 • is the rate offset in Gallager’s error exponent 𝑜 𝜍 – If connectivity parameter 𝜆 = 𝑜 grows at least as quickly as log 𝑜 log 𝛽 𝑗 log 𝑜 Θ = 𝑃 , then 𝑜 𝑜 𝑜 13

Comparison with I.I.D. Codes • MAC LDPC Code: • MAC I.I.D Code: • Gallager’s error exponents 𝐹 𝑞 1 𝑆 1 , 𝐹 𝑞 2 𝑆 2 , 𝐹 𝑄 12 𝑆 1 + 𝑆 2 > 0 for any (𝑆 1 , 𝑆 2 ) ∈ ℛ • Capacity-achieving with sufficient connectivity! 14

Finite-Blocklength Analysis • Information Densities: 𝑗 𝑌 1 ; 𝑍 𝑌 2 = log 𝑄(𝑍|𝑌 1 , 𝑌 2 ) 𝑄(𝑍|𝑌 2 ) 𝑗 𝑌 2 ; 𝑍 𝑌 1 = log 𝑄(𝑍|𝑌 1 , 𝑌 2 ) 𝑄(𝑍|𝑌 1 ) 𝑗 𝑌 1 , 𝑌 2 ; 𝑍 = log 𝑄(𝑍|𝑌 1 , 𝑌 2 ) 𝑄(𝑍) 15

Finite-Blocklength Analysis • Generalized random coding union (RCU) bound [ Liu and Effros’20 , Theorem 13], Berry-Esseen Theorem (thanks to coset vector) and [Polyanskiy et al.’10 , Lemma 47] • DM-2-MAC under moment assumptions [Liu and Effros’20 , Theorem 16]: 2 nd order 1 st order • 3 rd order LDPC Code Penalty, with sufficient connectivity: 𝑃 log 𝑜 • V : Covariance Matrix of information densities 𝟐 𝑜 • 𝜗 : Target error probability 𝑅 inv ⋅,⋅ : Extension of Gaussian complementary CDF 𝑅 −1 ⋅ • 16

𝑅 𝑗𝑜𝑤 V, 𝜗 Let 𝑎 ∈ ℝ 𝑒 ~ Gaussian with mean zero and covariance matrix V • 𝑅 𝑗𝑜𝑤 V, 𝜗 = 𝑨 ∈ ℝ 𝑒 : Pr Z ≤ 𝑨 ≥ 1 − 𝜗 • 𝑒 = 2: 𝑅 𝑗𝑜𝑤 V, 𝜗 Probability ≥ 1 − 𝜗 17

Finite-Blocklength Analysis • Compare with MAC i.i.d. Code [ Liu and Effros’20 , Theorem 14] • MAC LDPC Code [ Liu and Effros’20 , Theorem 16] • MAC I.I.D Codes improves third order term in [ Tan and Kosut’14 , 𝜉 log 𝑜 log 𝑜 1 Theorem 4] from − 𝟐 with 𝜉 ≥ 2|𝒴 1 ||𝒴 2 ||𝒵| to 2𝑜 𝟐 − 𝑃 𝑜 𝟐 𝑜 • MAC LDPC Code matches MAC I.I.D Code up to the second-order (dispersion) term 18

Generalized RCU Bound • RCU bound in [Polyanskiy et al.’10 , Theorem 16] designed for i.i.d. random codes. • Many practical codes have dependent codewords • Generalize to codes with symmetric code design – Dependence: 𝑄 𝑌 1 1 𝑌 1 2 ≠ 𝑄 𝑌 1 1 𝑄 𝑌 1 2 – Symmetry: 𝑄 𝑌 1 1 𝑌 1 2 = 𝑄 𝑌 1 1 𝑌 1 3 • Potentially useful for other practical codes 19

Key Step 𝑜 𝑌 1 Tx 1 𝑍 𝑜 𝑄(𝑍 𝑜 |𝑌 1 𝑜 , 𝑌 2 𝑜 ) Rx 𝑜 𝑌 2 Tx 2 MAC LDPC Codes: bound on codeword dependence 20

Main Takeaways • Bound on the codeword dependence may be applicable to other (e.g., linear) codes log 𝛽 log 𝑜 • LDPC encoder penalty: is 𝑃 with sufficient 𝑜 𝑜 density log 𝛽 • Our results on give an upper bound on the LDPC 𝑜 rate penalty. We are working to understand whether/when that bound is tight 21

Thank you! • References: – [1] A. Bennatan and D. Burshtein , “On the application of LDPC codes to arbitrary discrete -memoryless channels,” IEEE Trans. Inf. Theory,vol. 50, no. 3, pp. 417 – 438, March 2004 – [2] R. Gallager , “Low -density parity- check codes,” IEEE Trans. Inf. Theory , vol. 8, no. 1, pp. 21 – 28, 1962. – [3] Y. Polyanskiy, H. V. Poor, and S. Verdu , “Channel coding rate in the finite blocklength regime,”IEEE Trans. Inf. Theory, vol. 56, no. 5, pp.2307 – 2359, May 2010. – [4] Y. Liu and M. Effros. Finite-blocklength and error-exponent analysisfor LDPC codes in point-to- point and multiple access communication.[Online]. Available: https://arxiv.org/abs/2005.06428 – [5] V. Y. F. Tan and O. Kosut , “On the dispersions of three network information theory problems,” IEEE Trans. Inf. Theory, vol. 60, no. 2, pp. 881 – 903, Feb 2014. – [6] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi , and D. A. Spielman, “Improved low -density parity-check codes using irregular graphs,” IEEE Trans. Inform. Theory, vol. 47, pp. 585– 598, Feb. 2001. – [7] T. Richardson and R. Urbanke , “The capacity of low -density parity-check codes under message-passing decoding,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 599– 618, Feb. 2001. – [8] Z. Mei, K. Cai, and G. Song, “Performance analysis of finite -length LDPC codes over asymmetric memoryless channels,” IEEE Trans. Veh. Technol., vol. 68, no. 11, pp. 11 338 – 11 342, Nov 2019. – [9] A. Amraoui, R. Urbanke, and A. Montanari , “Finite - length scaling of irregular LDPC code ensembles,” in Proc. IEEE Inf. Theory Workshop, Aug 2005, pp. 5 – 10. – [10 ] A. Amraoui, A. Montanari, T. Richardson, and R. Urbanke , “ Finitelength scaling for iteratively decoded LDPC ensembles,” IEEE Trans. Inf. Theory, vol. 55, no. 2, pp. 473– 498, Feb 2009. – [11] R. Yazdani and M. Ardakani , “Waterfall performance analysis of finitelength LDPC codes on symmetric channels,” IEEE Trans. Comm., vol. 57, no. 11, pp. 3183– 3187, Nov 2009. 22