Decoding ReedMuller Codes Using Redundant Code Constraints Mengke - - PowerPoint PPT Presentation

decoding reed muller codes using redundant code
SMART_READER_LITE
LIVE PREVIEW

Decoding ReedMuller Codes Using Redundant Code Constraints Mengke - - PowerPoint PPT Presentation

Decoding ReedMuller Codes Using Redundant Code Constraints Mengke Lian , Christian H ager , Henry D. Pfister Department of Electrical and Computer Engineering, Duke University Department of Electrical Engineering,


slide-1
SLIDE 1

Decoding Reed–Muller Codes Using Redundant Code Constraints

Mengke Lian∗, Christian H¨ ager†, Henry D. Pfister∗‡

∗Department of Electrical and Computer Engineering, Duke University †Department of Electrical Engineering, Chalmers University of Technology ‡Department of Mathematics, Duke University

IEEE International Symposium on Information Theory June 21-26, 2020

slide-2
SLIDE 2

Outline

1

Introduction

2

Factor Graphs for Reed–Muller Codes

3

Recursive Decoding Algorithms

4

Collapsed Decoding Algorithms

5

Simulation Results

6

Conclusions

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 1 / 26

slide-3
SLIDE 3

Outline

1

Introduction

2

Factor Graphs for Reed–Muller Codes

3

Recursive Decoding Algorithms

4

Collapsed Decoding Algorithms

5

Simulation Results

6

Conclusions

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 2 / 26

slide-4
SLIDE 4

Introduction

Since the discovery of polar codes, interest in Reed–Muller (RM) codes has surged because of their excellent ML performance for short block lengths Many existing decoding methods exploit the symmetry of RM codes to achieve improved performance in practice [BH86, HC06, SHP18] The recursive projection–aggregation (RPA) decoding algorithm [YA20] was recently shown to perform well on a variety of low-order RM codes

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 3 / 26

slide-5
SLIDE 5

Introduction

Since the discovery of polar codes, interest in Reed–Muller (RM) codes has surged because of their excellent ML performance for short block lengths Many existing decoding methods exploit the symmetry of RM codes to achieve improved performance in practice [BH86, HC06, SHP18] The recursive projection–aggregation (RPA) decoding algorithm [YA20] was recently shown to perform well on a variety of low-order RM codes

Main contributions of this work

1

We connect RPA to (weighted) belief-propagation (BP) decoding by interpreting it as message-passing on a redundant factor graph

2

We propose a novel decoder, called recursive puncturing-aggregation (RXA), that uses puncturing to extend its idea to high-rate RM codes

3

We investigate non-recursive (i.e., collapsed) versions of RPA/RXA and compare with BP decoding using all minimum-weight parity checks

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 3 / 26

slide-6
SLIDE 6

Outline

1

Introduction

2

Factor Graphs for Reed–Muller Codes

3

Recursive Decoding Algorithms

4

Collapsed Decoding Algorithms

5

Simulation Results

6

Conclusions

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 4 / 26

slide-7
SLIDE 7

Factor Graphs and LDPC Decoding

v1 v2 v3 v4 v5 v6 v7 ℓ7 c1 c2 c3 ˆ λ3→5 λ7→3

C [M] V [N]

∂c1 ∂v3 Belief-propagation (BP) decoding in the LLR domain: λ(t)

v→c = ℓv +

  • c′∈∂v\c

ˆ λ(t−1)

c′→v

ˆ λ(t)

c→v = 2 tanh−1

  • v′∈∂c\v

tanh λ(t)

v′→c

2

  • ,

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 5 / 26

slide-8
SLIDE 8

Recursive Construction of Reed–Muller Codes

Reed–Muller code RM(r, m) has length N = 2m

Each message is given by an m-variate binary polynomial of degree at most r The codeword formed by evaluating its message at all (x1, . . . , xm) ∈ {0, 1}m

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 6 / 26

slide-9
SLIDE 9

Recursive Construction of Reed–Muller Codes

Reed–Muller code RM(r, m) has length N = 2m

Each message is given by an m-variate binary polynomial of degree at most r The codeword formed by evaluating its message at all (x1, . . . , xm) ∈ {0, 1}m

Definition: RM(−1, 0) ∅, RM(0, 0) {0, 1}, and RM(r, m)

  • (u, u + v)
  • u∈RM(r, m − 1), v∈RM(r − 1, m − 1)
  • Puncturing second half leaves shorter RM(r, m − 1) of same order

Adding both halves projects onto shorter RM(r − 1, m − 1) of lower order

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 6 / 26

slide-10
SLIDE 10

Factor Graphs for Reed–Muller Codes (1)

b b b b b b

RM(r, m−1) RM(r, m−1)

b b b b b b

RM(r−1, m−1)

Generalized check node represents code constraints of labeled RM code RM(r, m) {(u, u + v) | u∈RM(r, m − 1), v∈RM(r − 1, m − 1)} Puncturing second half leaves shorter RM(r, m − 1) of same order Adding both halves projects onto shorter RM(r − 1, m − 1) of lower order

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 7 / 26

slide-11
SLIDE 11

Factor Graphs for Reed–Muller Codes (2)

b b b b b b b b b

degree 2m−1

b b b b b b b b b

2m−1 subcodes

b b b b b b b b b b b b

RM(r−1, m−1)

b b b b b b

RM(r−1, m−1) π

Automorphism group of length-2m RM code

Affine general linear group over F2: AGL(m, 2) For invertible A ∈ {0, 1}m×m and any b ∈ {0, 1}m, contains permutations σ : {0, 1}m → {0, 1}m x → Ax + b AGL(m, 2) generates 2m−1 distinct RM(r−1, m−1) constraints

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 8 / 26

slide-12
SLIDE 12

Outline

1

Introduction

2

Factor Graphs for Reed–Muller Codes

3

Recursive Decoding Algorithms

4

Collapsed Decoding Algorithms

5

Simulation Results

6

Conclusions

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 9 / 26

slide-13
SLIDE 13

Recursive Projection–Aggregation (RPA) [YA20]

b b b b b b b b b

degree 2m−1

b b b b b b b b b

2m−1 subcodes

b b b b b b

1 2 3 4

b b b b b b

RM(r−1, m−1)

b b b b b b

RM(r−1, m−1) π Yellow VN messages initialized to channel LLRs. Then iterate Tmax times: 1 Projection: Standard message-passing rule from code bits to subcode bits 2 Subcode decoding: Recurse or soft-in/hard-out RM(1, m−1) decoding 3 Backward update: Hard output from subcode allows soft code-bit combining 4 VN update: Average of all input LLRs update due to redundant constraints

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 10 / 26

slide-14
SLIDE 14

Decoding Path in the Reed–Muller Tableau

RM(1,2) RM(0,2) RM(2,3) RM(1,3) RM(0,3) RM(3,4) RM(2,4) RM(1,4) RM(0,4) RM(4,5) RM(3,5) RM(2,5) RM(1,5) RM(0,5) RM(4,6) RM(3,6) RM(2,6) RM(1,6) RM(3,7)

(u, u+v) construction

SPC codes extended Hamming codes augmented Hadamard codes repeat codes

a a d d a recursive puncturing–aggregation (RXA) d recursive projection–aggregation (RPA)

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 11 / 26

slide-15
SLIDE 15

Recursive Puncturing–Aggregation (RXA)

All outgoing VN messages are initialized to the channel LLRs. Then, iterate Tmax times: 1 Subcode decoding: if r<m − 3, recurse. Else, extrinsic soft-in/soft-out decoding of RM(m−3, m−1) extended Hamming code 2 VN update: Sum channel LLR and extrinsic weighted sum1 of input LLRs

b b b b b b b b b

degree 2m−1

b b b b b b b b b

2(2m−1) total subcodes

b b b b b b

1 2

b b b

RM(r, m−1)

b b b

RM(r, m−1)

b b b b b b

RM(r, m−1)

b b b

RM(r, m−1) π

1Many redundant constraints means averaging (e.g., w=α/(deg−

1) for α∈(0, 1)) works well

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 12 / 26

slide-16
SLIDE 16

Hard vs. Soft Decoding

Factor graph description naturally leads to a variety of decoders

sum product: defines extrinsic soft-in/soft-out decoding max product / min sum – defines another extrinsic soft-in/soft-out decoding weights and corrections: improved performance vs. complexity intrinsic hard decision – RPA uses soft-in/hard-out ML decoder of RM code possible extensions: guided decimation and neural BP

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 13 / 26

slide-17
SLIDE 17

Hard vs. Soft Decoding

Factor graph description naturally leads to a variety of decoders

sum product: defines extrinsic soft-in/soft-out decoding max product / min sum – defines another extrinsic soft-in/soft-out decoding weights and corrections: improved performance vs. complexity intrinsic hard decision – RPA uses soft-in/hard-out ML decoder of RM code possible extensions: guided decimation and neural BP

RPA uses soft-projection and intrinsic hard decision decoding

Works quite well with low-complexity and no parameters to tune!

RXA uses extrinsic soft-in/soft-out RM decoding (no projection required)

Increases subcode decoding complexity and requires some α tuning

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 13 / 26

slide-18
SLIDE 18

Outline

1

Introduction

2

Factor Graphs for Reed–Muller Codes

3

Recursive Decoding Algorithms

4

Collapsed Decoding Algorithms

5

Simulation Results

6

Conclusions

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 14 / 26

slide-19
SLIDE 19

Collapsing the Recursion1

Recursive complexity

RPA for RM(3, m) uses Tmax(2m−1) calls to RPA for RM(2, m−1) RPA for RM(2, m−1) uses Tmax(2m−1−1) calls to RPA for RM(1, m−2) Thus, the operation complexity for RM(1, m−2) decoding is roughly T 2

max(2m − 1)(2m−1 − 1) (2m−2)(m − 2)

  • FWHT
  • Tmax=4,m=7

= 20, 482, 560 While the recursive approach is elegant, the complexity is quite large!

1[YA20] journal paper posted after ISIT submission considers a simplified collapsed operation

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 15 / 26

slide-20
SLIDE 20

Collapsing the Recursion1

Recursive complexity

RPA for RM(3, m) uses Tmax(2m−1) calls to RPA for RM(2, m−1) RPA for RM(2, m−1) uses Tmax(2m−1−1) calls to RPA for RM(1, m−2) Thus, the operation complexity for RM(1, m−2) decoding is roughly T 2

max(2m − 1)(2m−1 − 1) (2m−2)(m − 2)

  • FWHT
  • Tmax=4,m=7

= 20, 482, 560 While the recursive approach is elegant, the complexity is quite large!

Collapsed complexity

To skip the recursion, one can directly enumerate all RM(1, m−r+1) subcodes Subcodes given by m

r−1

  • 2 distinct (r−1)-dimensional subspaces in Fm

2

For m = 3, number of subcodes is reduced by 3

1[YA20] journal paper posted after ISIT submission considers a simplified collapsed operation

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 15 / 26

slide-21
SLIDE 21

Decoding Path in the Reed–Muller Tableau

RM(1,2) RM(0,2) RM(2,3) RM(1,3) RM(0,3) RM(3,4) RM(2,4) RM(1,4) RM(0,4) RM(4,5) RM(3,5) RM(2,5) RM(1,5) RM(0,5) RM(4,6) RM(3,6) RM(2,6) RM(1,6) RM(3,7)

(u, u+v) construction

SPC codes extended Hamming codes augmented Hadamard codes repeat codes

a a b c d d e a recursive puncturing–aggregation (RXA) b collapsed puncturing–aggregation (CXA) c minimum-weight parity-checks (MWPCs) d recursive projection–aggregation (RPA) e collapsed projection–aggregation (CPA)

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 16 / 26

slide-22
SLIDE 22

Collapsed Projection–Aggregation (CPA)

All outgoing VN messages are initialized to the channel LLRs. Then, iterate Tmax times: 1 Projection: Standard BP rule from code bits to subcode bits 2 Subcode decoding: Int/Ext decoding of soft-in/hard-out RM(1, m−r+1) 3 Backward update: Hard/soft output from subcode for soft code-bit combining 4 VN update: Average of all input LLRs update due to redundant constraints

b b b b b b b b b

degree m

r−1

  • 2
b b b b b b b b b

m

r−1

  • 2 total

subcodes

b b b b b b

1 2 3 4

b b b

CN degree

2r−1 + 1

b b b b b b b b b b b b

RM(1, m−r+1)

b b b b b b b b b b b b b b b

RM(1, m−r+1) π

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 17 / 26

slide-23
SLIDE 23

Collapsed Projection–Aggregation (CPA)

All outgoing VN messages are initialized to the channel LLRs. Then, iterate Tmax times: 1 Projection: Standard BP rule from code bits to subcode bits 2 Subcode decoding: Int/Ext decoding of soft-in/hard-out RM(1, m−r+1) 3 Backward update: Hard/soft output from subcode for soft code-bit combining 4 VN update: Average of all input LLRs update due to redundant constraints

b b b b b b b b b

degree m

r−1

  • 2
b b b b b b b b b

m

r−1

  • 2 total

subcodes

b b b b b b

1 2 3 4

b b b

CN degree

2r−1 + 1

b b b b b b b b b b b b

RM(1, m−r+1)

b b b b b b b b b b b b b b b

RM(1, m−r+1) π Works well with intrinsic hard decoding Extrinsic soft-in/soft-out decoding also works well with tuned α

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 17 / 26

slide-24
SLIDE 24

Collapsed Puncturing–Aggregation (CXA)

All outgoing VN messages are initialized to the channel LLRs. Then, iterate Tmax times: 1 Subcode decoding: Extrinsic soft-in soft-out decoding of RM(r, r+2) extended Hamming code 2 VN update: Sum channel LLR and extrinsic weighted sum2of input LLRs

b b b b b b b b b

degree m

r−1

  • 2
b b b b b b b b b

2 m

r−1

  • 2 total

subcodes

b b b b b b

1 2

b b b

RM(r, r+2)

b b b

RM(r, r+2)

b b b b b b

RM(r, r+2)

b b b

RM(r, r+2) π

1Many redundant constraints means averaging (e.g., w=α/(deg−

1) for α∈(0, 1)) works well

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 18 / 26

slide-25
SLIDE 25

Collapsed Puncturing–Aggregation (CXA)

All outgoing VN messages are initialized to the channel LLRs. Then, iterate Tmax times: 1 Subcode decoding: Extrinsic soft-in soft-out decoding of RM(r, r+2) extended Hamming code 2 VN update: Sum channel LLR and extrinsic weighted sum2of input LLRs

b b b b b b b b b

degree m

r−1

  • 2
b b b b b b b b b

2 m

r−1

  • 2 total

subcodes

b b b b b b

1 2

b b b

RM(r, r+2)

b b b

RM(r, r+2)

b b b b b b

RM(r, r+2)

b b b

RM(r, r+2) π CXA plus one stage of puncturing gives RM(r, r + 1) parity-check subcode and equals WBP decoding with all minimum-weight parity checks [SHP18]

1Many redundant constraints means averaging (e.g., w=α/(deg−

1) for α∈(0, 1)) works well

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 18 / 26

slide-26
SLIDE 26

Outline

1

Introduction

2

Factor Graphs for Reed–Muller Codes

3

Recursive Decoding Algorithms

4

Collapsed Decoding Algorithms

5

Simulation Results

6

Conclusions

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 19 / 26

slide-27
SLIDE 27

Simulation Details

Standard additive white Gaussian noise (AWGN) channel Iteration number Tmax = 15 in all cases Convergence-based stopping criterion used by [YA20] List decoder tries all 16 values of 4 least reliable bits Ordered statistics decoding (OSD) ≈ MAP used as a benchmark

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 20 / 26

slide-28
SLIDE 28

RM(3, 7), N = 128, K = 64, R = 0.5

2 2.2 2.4 2.6 2.8 3 3.2 3.4 10−4 10−3 10−2

Eb/N0 (dB) WER

OSD BP Hoc RPA RXA CPA CPA list CPAh CPAh list CXA CXA list

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 21 / 26

slide-29
SLIDE 29

RM(5, 8), N = 256, K = 219, R = 0.856

3.6 3.8 4 4.2 4.4 4.6 4.8 5 10−4 10−3 10−2 10−1

Eb/N0 (dB) WER

CXA CXA list OSD

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 22 / 26

slide-30
SLIDE 30

RM(2, 8), N = 256, K = 37, R = 0.145

1 1.2 1.4 1.6 1.8 2 2.2 2.4 10−3 10−2

Eb/N0 (dB) WER

RPA CPA RPA list CPA list OSD

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 23 / 26

slide-31
SLIDE 31

Comparison with List-Based Methods

List-Based Decoding Methods for RM Codes

Dumer and Shabunov [DS02] defined recursive decoders for RM subcodes Same principles as the SC and SCL decoders (w/permutations) for polar codes Even before PhD thesis [Sto02] that defined polar codes without the proof! DS recently1 posted their code – https://github.com/kshabunov/ecclab For moderate list sizes, it runs very quickly and has very good performance!

1Fair timing comparisons not reported because it’s hard and code posted after ISIT deadline

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 24 / 26

slide-32
SLIDE 32

Comparison with List-Based Methods

List-Based Decoding Methods for RM Codes

Dumer and Shabunov [DS02] defined recursive decoders for RM subcodes Same principles as the SC and SCL decoders (w/permutations) for polar codes Even before PhD thesis [Sto02] that defined polar codes without the proof! DS recently1 posted their code – https://github.com/kshabunov/ecclab For moderate list sizes, it runs very quickly and has very good performance!

RPA and all its factor graph variations

Very interesting new development for decoding RM codes! But, decoding all subcodes repeatedly is very costly Searching a list inside one subcode may be more efficient Likely, the best performance vs. complexity may require a hybrid method

1Fair timing comparisons not reported because it’s hard and code posted after ISIT deadline

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 24 / 26

slide-33
SLIDE 33

Outline

1

Introduction

2

Factor Graphs for Reed–Muller Codes

3

Recursive Decoding Algorithms

4

Collapsed Decoding Algorithms

5

Simulation Results

6

Conclusions

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 25 / 26

slide-34
SLIDE 34

Conclusions

RPA can be seen as weighted BP-like decoding on a redundant factor graph RXA is similar in spirit as RPA, but tailored to high-rate RM codes Collapsed recursions can reduce complexity and increase parallelism Complexity is quite high and more comparison with list decoders needed Future directions include neural BP and/or hybrid BP/list approaches

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 26 / 26

slide-35
SLIDE 35

Conclusions

RPA can be seen as weighted BP-like decoding on a redundant factor graph RXA is similar in spirit as RPA, but tailored to high-rate RM codes Collapsed recursions can reduce complexity and increase parallelism Complexity is quite high and more comparison with list decoders needed Future directions include neural BP and/or hybrid BP/list approaches

Thank you!

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 26 / 26

slide-36
SLIDE 36

References I

[BH86]

  • M. Bossert and F. Hergert.

Hard- and soft-decision decoding beyond the half minimum distance—An algorithm for linear codes. IEEE Trans. Inform. Theory, 32(5):709–714, Sept. 1986. [DS02] Ilya Dumer and Kirill Shabunov. Recursive list decoding for Reed-Muller codes and their subcodes. In Information, Coding and Mathematics, pages 279–298. Springer, 2002. [HC06] Thomas R Halford and Keith M Chugg. Random redundant soft-in soft-out decoding of linear block codes. In Proc. IEEE Int. Symp. Inform. Theory, pages 2230–2234. IEEE, 2006. [KKM+17] Shrinivas Kudekar, Santhosh Kumar, Marco Mondelli, Henry D. Pfister, Eren S ¸a¸ so˘ glu, and R¨ udiger Urbanke. Reed-Muller codes achieve capacity on erasure channels. IEEE Trans. Inform. Theory, 63(7):4298–4316, 2017. [SHP18] Elia Santi, Christian H¨ ager, and Henry D Pfister. Decoding Reed-Muller codes using minimum-weight parity checks. In Proc. IEEE Int. Symp. Inform. Theory, 2018. [Online]. Available: http://arxiv.org/abs/1804.10319.

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 27 / 26

slide-37
SLIDE 37

References II

[Sto02] Norbert Stolte. Recursive codes with the Plotkin-Construction and their Decoding. PhD thesis, University of Technology Darmstadt, Germany, 2002. [YA20] Min Ye and Emmanuel Abbe. Recursive projection-aggregation decoding of reed-muller codes. IEEE Trans. Inform. Theory, 2020.

Decoding Reed–Muller Codes Using Redundant Code Constraints |

  • M. Lian, C. H¨

ager, H. D. Pfister 28 / 26