Finite-Blocklength and Error-Exponent Analyses for LDPC Codes in - - PowerPoint PPT Presentation

Finite-Blocklength and Error-Exponent Analyses for LDPC Codes in Point-to-Point and Multiple Access Communication* June 21-26, 2020 IEEE International Symposium on Information Theory Yuxin Liu and Michelle Effros

Department of Electrical Engineering, California Institute of Technology

1

* Extended version available on arXiv <2005.06428>.

This material is based upon work supported in part by the National Science Foundation under Grant No. 1817241. The work of Y. Liu is supported in part by the Oringer Fellowship Fund in Information Science and Technology.

Background

• Introduced by [Gallager’62]
• Low density enables low-complexity decoding
• Commercial Standards: Ethernet (IEEE 803.3an), WiFi (IEEE

802.11n), WiMAX (IEEE 802.16e), 5G Code

• [Davey and MacKay’98] binary → 𝐻𝐺 𝑟
• [Richardson and Urbanke’01] density evolution
• [Bennatan and Burshtein’04] arbitrary DMC
• [Erez and Miller’05] modulo-additive channel
• [Di et al.’02] finite-blocklength in BEC
• Many more.

2 Channel 𝑌 𝑍

Summary of LDPC Results

3

• These analyses capture the tradeoff between rate 𝑆, error probability 𝜗,

and blocklength 𝑜:

– Error-Exponent: most accurate for 𝜗 small – Dispersion: most accurate for 𝑜 small – Scaling Law: useful under iterative decoding

Outline

• Code Construction
• Error-Exponent Analysis
• Finite-Blocklength (Dispersion-style) Analysis
• Main Takeaways

4

Code Construction

5

• Follow the construction from [Bennatan and Burshtein’04]

Code Construction

6

• Different from the design of

Gallager ensemble

• Originated from [Luby et al.’01]
• Socket values uniformly from

𝐻𝐺 𝑟 \{0}

• Expurgation: remove codes with

small minimum distance

• Uniform Codeword Selection:

Restrict the operational rate to equal to 𝑆 …

𝑜 …

1 2 𝜇

…

𝑜𝜇

𝑜 − 1 𝜇 + 1 𝑜 − 1 𝜇 + 2

2 𝑠 = 𝑜(1 − 𝑆) 1

…

𝜍

𝑠 − 1 𝜍 + 2 𝑠 − 1 𝜍 + 1

…

𝑠𝜍

…

• (𝜇, 𝜍, 𝑜, 𝑟) → 𝑆 ≜ 1 − 𝜇

𝜍

Code Construction

• Coset Vector:

– Uniformly and Independently over 𝐻𝐺(𝑟) – Fixed across codewords

• Quantizer:

– Mapping 𝐻𝐺 𝑟 → Channel Input alphabet component-wise 7

Decoding Strategy

• Maximum likelihood (ML) decoding
• Why not iterative:

– Distinguish decoding performance penalty from encoding performance penalty – Iterative decoding next step

8

Summary of Our Results for LDPC Codes

Error-Exponent Dispersion-style Point-to-Point Channel (PPC) [Bennatan and Burshtein’04] Symmetrical MAC General MAC 9

MAC LDPC Code

10

• Symmetrical: 𝒴1 = 𝒴2 = ⋯ = 𝒴𝐿, 𝑄 𝑍 𝑌1, 𝑌2, … 𝑌𝐿 = 𝑄 𝑍 𝜌 𝑌1, 𝑌2, … 𝑌𝐿

Same LDPC Encoder Different Coset Vectors Same Quantizer

Different LDPC Encoders Different Coset Vectors Different Quantizers Symmetrical MAC Asymmetrical MAC

• For each transmitter:

Error-Exponent Analysis

11

• Tx 1: 𝜇1, 𝜍1, 𝑟, 𝑜 → 𝑆1 = 1 − 𝜇1

𝜍1 , 𝜀1 ⋅ → 𝑄 𝑌1

• Tx 2: 𝜇2, 𝜍2, 𝑟, 𝑜 → 𝑆2 = 1 − 𝜇2

𝜍2 , 𝜀2 ⋅ → 𝑄 𝑌2

Error-Exponent Analysis

• Expected error probability in a general discrete

memoryless 2-MAC

– Follow the approach [Bennatan and Burshtein’04] for PPC

• 𝐹𝑞1 ⋅ , 𝐹𝑞2(⋅): Gallager’s error

exponents

• 𝑟, 𝑜, 𝑆1, 𝑆2 are design

parameters

• 𝛽1, 𝛽2: LDPC encoder penalties

for transmitters 1 and 2

Ensemble-average error probability

12

• 𝐹𝑞12 ⋅ : Gallager’s error

exponent

• 𝛽12 = 𝛽1𝛽2: encoder penalty

for decoding both messages incorrectly

𝛽1, 𝛽2, 𝛽12

• Penalties for LDPC encoder, 𝛽12 = 𝛽1𝛽2, 𝑗 ∈ {1,2}

𝛽𝑗 ∝ max

𝑢 ∈ 𝒰

𝑟 𝑜

𝑈𝑠𝑏𝑜𝑡𝑛𝑗𝑢𝑢𝑓𝑠−𝑗 # 𝑀𝐸𝑄𝐷 𝑓𝑜𝑡𝑓𝑛𝑐𝑚𝑓 𝑢𝑧𝑞𝑓−𝑢 𝑤𝑓𝑑𝑢𝑝𝑠𝑡 # 𝑢𝑧𝑞𝑓−𝑢 𝑤𝑓𝑑𝑢𝑝𝑠𝑡

– LDPC Ensemble for transmitter 𝑗 refers to the ensemble from “LPDC Encoder” with design parameters (𝜇𝑗, 𝜍𝑗, 𝑜, 𝑟) – Type is the relative frequency of each symbol in 𝐻𝐺(𝑟) – 𝒰

𝑟 𝑜 is the set of possible types with 𝑜 𝐻𝐺(𝑟) symbols

• log 𝛽𝑗

𝑜

is the rate offset in Gallager’s error exponent

– If connectivity parameter 𝜆 =

𝜍 𝑜 grows at least as quickly as

Θ

log 𝑜 𝑜

, then

log 𝛽𝑗 𝑜

= 𝑃

log 𝑜 𝑜

13

Comparison with I.I.D. Codes

• MAC LDPC Code:
• MAC I.I.D Code:
• Gallager’s error exponents

𝐹𝑞1 𝑆1 , 𝐹𝑞2 𝑆2 , 𝐹𝑄12 𝑆1 + 𝑆2 > 0 for any (𝑆1, 𝑆2) ∈ ℛ

• Capacity-achieving with sufficient connectivity!

14

Finite-Blocklength Analysis

15

• Information Densities:

𝑗 𝑌1; 𝑍 𝑌2 = log 𝑄(𝑍|𝑌1, 𝑌2) 𝑄(𝑍|𝑌2) 𝑗 𝑌2; 𝑍 𝑌1 = log 𝑄(𝑍|𝑌1, 𝑌2) 𝑄(𝑍|𝑌1) 𝑗 𝑌1, 𝑌2; 𝑍 = log 𝑄(𝑍|𝑌1, 𝑌2) 𝑄(𝑍)

Finite-Blocklength Analysis

• Generalized random coding union (RCU) bound [Liu and Effros’20,

Theorem 13], Berry-Esseen Theorem (thanks to coset vector) and [Polyanskiy et al.’10, Lemma 47]

• DM-2-MAC under moment assumptions [Liu and Effros’20, Theorem

16]:

• V: Covariance Matrix of information densities
• 𝜗: Target error probability
• 𝑅inv ⋅,⋅ : Extension of Gaussian complementary CDF 𝑅−1 ⋅

16

1st order 3rd order LDPC Code Penalty, with

sufficient connectivity: 𝑃 log 𝑜 𝑜 𝟐

• 2nd order

𝑅𝑗𝑜𝑤 V, 𝜗

• Let 𝑎 ∈ ℝ𝑒~ Gaussian with mean zero and covariance matrix V

𝑅𝑗𝑜𝑤 V, 𝜗 = 𝑨 ∈ ℝ𝑒: Pr Z ≤ 𝑨 ≥ 1 − 𝜗

• 𝑒 = 2:

17

𝑅𝑗𝑜𝑤 V, 𝜗

Probability ≥ 1 − 𝜗

Finite-Blocklength Analysis

• Compare with MAC i.i.d. Code [Liu and Effros’20, Theorem 14]
• MAC LDPC Code [Liu and Effros’20, Theorem 16]
• MAC I.I.D Codes improves third order term in [Tan and Kosut’14,

Theorem 4] from −

𝜉 log 𝑜 𝑜

𝟐 with 𝜉 ≥ 2|𝒴1||𝒴2||𝒵| to

log 𝑜 2𝑜 𝟐 − 𝑃 1 𝑜 𝟐

• MAC LDPC Code matches MAC I.I.D Code up to the second-order

(dispersion) term

18

Generalized RCU Bound

• RCU bound in [Polyanskiy et al.’10, Theorem 16]

designed for i.i.d. random codes.

• Many practical codes have dependent codewords
• Generalize to codes with symmetric code design

– Dependence: 𝑄𝑌1 1 𝑌1 2 ≠ 𝑄𝑌1 1 𝑄𝑌1 2 – Symmetry: 𝑄𝑌1 1 𝑌1 2 = 𝑄𝑌1 1 𝑌1 3

• Potentially useful for other practical codes

19

Key Step

MAC LDPC Codes: bound on codeword dependence

20 Tx 1 Tx 2 𝑌1

𝑜

𝑌2

𝑜

𝑄(𝑍𝑜|𝑌1

𝑜, 𝑌2 𝑜)

𝑍𝑜 Rx

Main Takeaways

• Bound on the codeword dependence may be applicable

to other (e.g., linear) codes

• LDPC encoder penalty:

log 𝛽 𝑜

is 𝑃

log 𝑜 𝑜

with sufficient density

• Our results on

log 𝛽 𝑜

give an upper bound on the LDPC rate penalty. We are working to understand whether/when that bound is tight

21

Thank you!

• References:

– [1] A. Bennatan and D. Burshtein, “On the application of LDPC codes to arbitrary discrete-memoryless channels,” IEEE Trans. Inf. Theory,vol. 50, no. 3, pp. 417–438, March 2004 – [2] R. Gallager, “Low-density parity-check codes,” IEEE Trans. Inf. Theory, vol. 8, no. 1, pp. 21–28, 1962. – [3] Y. Polyanskiy, H. V. Poor, and S. Verdu, “Channel coding rate in the finite blocklength regime,”IEEE Trans. Inf. Theory, vol. 56, no. 5, pp.2307–2359, May 2010. – [4] Y. Liu and M. Effros. Finite-blocklength and error-exponent analysisfor LDPC codes in point-to- point and multiple access communication.[Online]. Available: https://arxiv.org/abs/2005.06428 – [5] V. Y. F. Tan and O. Kosut, “On the dispersions of three network information theory problems,” IEEE Trans.

• Inf. Theory, vol. 60, no. 2, pp. 881–903, Feb 2014.

– [6] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman, “Improved low-density parity-check codes using irregular graphs,” IEEE Trans. Inform. Theory, vol. 47, pp. 585–598, Feb. 2001. – [7] T. Richardson and R. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 599–618, Feb. 2001. – [8] Z. Mei, K. Cai, and G. Song, “Performance analysis of finite-length LDPC codes over asymmetric memoryless channels,” IEEE Trans. Veh. Technol., vol. 68, no. 11, pp. 11 338–11 342, Nov 2019. – [9] A. Amraoui, R. Urbanke, and A. Montanari, “Finite-length scaling of irregular LDPC code ensembles,” in

• Proc. IEEE Inf. Theory Workshop, Aug 2005, pp. 5–10.

– [10 ] A. Amraoui, A. Montanari, T. Richardson, and R. Urbanke, “Finitelength scaling for iteratively decoded LDPC ensembles,” IEEE Trans. Inf. Theory, vol. 55, no. 2, pp. 473–498, Feb 2009. – [11] R. Yazdani and M. Ardakani, “Waterfall performance analysis of finitelength LDPC codes on symmetric channels,” IEEE Trans. Comm., vol. 57, no. 11, pp. 3183–3187, Nov 2009.

22

Thank you!

• References (continued):

– [12] M. C. Davey and D. J. C. MacKay, “Low density parity check codes over GF(q),” in Proc. 1998 IEEE Information Theory Workshop, June 1998. – [13] U. Erez and G. Miller, “The ml decoding performance of LDPC ensembles over z/sub q/,” IEEE Trans.

• Inf. Theory, vol. 51, no. 5, pp. 1871–1879, May 2005.

– [14] C. Di, D. Proietti, I. E. Telatar, T. Richardson, and R. Urbanke, “Finitelength analysis of low-density parity-check codes on the binary erasure channel,” IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1570–1579, June 2002. – [15] H. Yagi and H. V. Poor, “Coset codes for compound multiple access channels with common information,” IEEE Trans. Inf. Theory, vol. 57, no. 6, pp. 3429–3448, 2011. – [16] A. Roumy and D. Declercq, “Characterization and optimization of LDPC codes for the 2-user Gaussian multiple access channel,” EURASIP J. Wirel. Comm. Netw., vol. 2007, no. 1, p. 074890, Jun 2007. [Online]. Available: https://doi.org/10.1155/2007/74890 – [17] E. Yang and J. Meng, “New nonasymptotic channel coding theorems for structured codes,” IEEE Trans.

• Inf. Theory, vol. 61, no. 9, pp. 4534–4553, Sep. 2015.

– [18] S. Sharifi, A. K. Tanc, and T. M. Duman, “LDPC code design for the two-user Gaussian multiple access channel,” IEEE Trans. on Wirel. Comm., vol. 15, no. 4, pp. 2833–2844, 2015. – [19] M. Ebrahimi, F. Lahouti, and V. Kostina. Two-layer coded channel access with collision resolution: Design and analysis. [Online]. Available: https://arxiv.org/abs/1909.00065

23