Data-Driven Ensembles for Deep and Hard-Decision Hybrid Decoding - - PowerPoint PPT Presentation
Introduction Introduction Introduction Data-Driven Ensembles for Deep and Hard-Decision Hybrid Decoding International Symposium on Information Theory Tomer Raviv Nir Raviv Yair Beery School of
Introduction Introduction Introduction
International Symposium on Information Theory Tomer Raviv Nir Raviv Yair Be’ery School of Electrical Engineering Tel Aviv University, Israel
June 2020
I. Background II. On the Importance of Data
I. Background II. On the Importance of Data
Error Correction Codes increases reliability of transmission by redundancy Assumptions Linear Codes AWGN channel Galois Field - GF(2) Encoding
Property of linear codes Known as parity check matrix Size Each row corresponds to a check constraint Each column to a variable
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
[2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988
Hard decision rule:
Cycles degrade performance of BP… How to compensate for cycles?
Cycles degrade performance of BP… How to compensate for cycles? Deep learning!
Deep learning
Intuition Adjusting the weights compensates for small cycles
[3] Nachmani et al. , “Learning to Decode Linear Codes Using Deep Learning”, Sep 16
[3] Nachmani et al. , “Learning to Decode Linear Codes Using Deep Learning”, Sep 16
[3] Nachmani et al. , “Learning to Decode Linear Codes Using Deep Learning”, Sep 16
Binary cross entropy multiloss: The BP and channel exhibit symmetry Training is done with zero codeword
I. Background II. On the Importance of Data
Choice of hypothesis class Learnable parameters assignment
Choice of any state-of-the-art architecture Less limitations on solution space
[4] Tobias Gruber et al. “On deep learning-based channel decoding”. In: 51st Annual Conference on Information Sciences and Systems (CISS). 2017.
Currently, the model based approach dominates Curse of dimensionality Yet, the model isn’t all…
Training data is core in DL Not fully understood
Classical ML
Decoding ML
I. Background II. On the Importance of Data
A single decoder is limited.. Combination of decoders is superior Divide and conquer principle
[5] Lior Rokach, “Pattern Classification using Ensemble Methods”, 2010
List-decoding is suboptimal ”there exists no clear evidence on which graph permutation performs best for a given input”
[6] Elkelesh et al., “Belief Propagation List Decoding of Polar Codes”, 2018
Key points decoding performance complexity
[6] Elkelesh et al., “Belief Propagation List Decoding of Polar Codes”, 2018
Based on Hamming distance Error pattern computation Region assignment Match to dataset Train on Based on syndrome (won’t have time to cover)…
Classical hard decision decoder (HDD) Outputs , calculate Gating types single-choice gating - for realizing all decoders gating - random-choice gating - randomly
Likelihood-maximizing function [7]:
[7] Ralf Koetter et al., “Characterizations of pseudo-codewords of (low-density) parity check codes”, 2007
Results for BCH(63,45):
CR-BCH(63,36) Code Waterfall - 0.3dB FER gain Error-floor – 1.25dB FER gain Compared to best previous results [8] 3-decoders
[8] Ishay Be’ery et al., "Active Deep Decoding of Linear Codes," in IEEE Transactions on Communications, Feb. 2020.
Results for BCH(63,45):
CR-BCH(63,45) Code Waterfall - 0.3dB FER gain Error-floor – 1dB FER gain Compared to best previous results [8] 3-decoders
[8] Ishay Be’ery et al., "Active Deep Decoding of Linear Codes," in IEEE Transactions on Communications, Feb. 2020.
I. Background II. On the Importance of Data
Domain knowledge is vital for ML in communication Data is important as the algorithm (!) How to effectively and efficiently utilize data is still insufficiently researched
[1] Urbanke and Richardson ,“Modern Coding Theory” , 2008 [2] Judea Pearl, “Probabilistic reasoning in intelligent systems: networks of plausible inference” , 1988 [3] Nachmani et al. , “Learning to Decode Linear Codes Using Deep Learning”, Sep 16 [4] Tobias Gruber et al. “On deep learning-based channel decoding”. In: 51st Annual Conference on Information Sciences and Systems (CISS). 2017. [5] Lior Rokach, “Pattern Classification using Ensemble Methods”, 2010 [6] Elkelesh et al., “Belief Propagation List Decoding of Polar Codes”, 2018 [7] Ralf Koetter et al., “Characterizations of pseudo-codewords of (low-density) parity check codes”, 2007 [8] Ishay Be’ery et al., "Active Deep Decoding of Linear Codes," in IEEE Transactions on Communications, Feb. 2020.