LDPC Codes over the q -ary Multi-Bit Channel Rami Cohen Andrew and - - PowerPoint PPT Presentation

LDPC Codes over the q-ary Multi-Bit Channel

Rami Cohen Andrew and Erna Viterbi Faculty of Electrical Engineering Technion - Israel Institute of Technology April 2017 Coding Theory Seminar Computer Science Dept., Technion Based on a PhD research advised by Prof. Yuval Cassuto

Rami Cohen The q-ary Multi-Bit Channel April 2017 1 / 39

Table of Contents

1

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder Decoding Threshold Region Finite-Length Iterative-Decoding Analysis Finite-Length Maximum-Likelihood Decoding Analysis

2

Suggestions for Future Research

Rami Cohen The q-ary Multi-Bit Channel April 2017 2 / 39

The q-ary Multi-Bit Channel (QMBC)

Table of Contents

1

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder Decoding Threshold Region Finite-Length Iterative-Decoding Analysis Finite-Length Maximum-Likelihood Decoding Analysis

2

Suggestions for Future Research

Rami Cohen The q-ary Multi-Bit Channel April 2017 3 / 39

The q-ary Multi-Bit Channel (QMBC)

Partial Erasure Channels

Measurement channel Levels of electric charge represent data symbols New flash memories: up to 32 levels (5 bits) per memory cell The read operation is performed by measuring current/voltage levels Partial read Measurement terminates prematurely (e.g., clock limitation) Imperfect current/voltage sensing (e.g., leakage) Inconclusive measurement result: Set of possible levels

Rami Cohen The q-ary Multi-Bit Channel April 2017 4 / 39

The q-ary Multi-Bit Channel (QMBC)

Partial Erasure Channels

xi are taken from X = {0, 1, ..., q − 1} yi are subsets of X, such that xi ∈ yi Definition: i has a partial erasure when |yi| > 1

Rami Cohen The q-ary Multi-Bit Channel April 2017 5 / 39

The q-ary Multi-Bit Channel (QMBC)

Our First Model: The q-ary Partial-Erasure Channel (QPEC)

QPEC partial erasure: The output is a random set of cardinality M (1 ≤ M ≤ q) that contains the input symbol Contributions:

Set iterative decoder Asymptotic iterative-decoding performance analysis Bounds and approximation models Code design using linear programming

[RC, YC ISIT ’14], [RC, YC Iterative Decoding of LDPC Codes over the q-ary Partial Erasure Channel, IT Trans. ’16] Rami Cohen The q-ary Multi-Bit Channel April 2017 6 / 39

The q-ary Multi-Bit Channel (QMBC)

Our Second Model: The q-ary Multi-Bit Channel (QMBC)

q voltage/current levels represent q-ary (q = 2s) symbols Symbols are read in a threshold-measurement process One symbol bit is provided in each step, starting from the MSB

Rami Cohen The q-ary Multi-Bit Channel April 2017 7 / 39

The q-ary Multi-Bit Channel (QMBC)

The q-ary Multi-Bit Channel

q voltage/current levels represent q-ary (q = 2s) symbols Symbols are read in a threshold-measurement process One symbol bit is provided in each step, starting from the MSB

Rami Cohen The q-ary Multi-Bit Channel April 2017 7 / 39

2 or 3 Binary: 1?

The q-ary Multi-Bit Channel (QMBC)

The q-ary Multi-Bit Channel

q voltage/current levels represent q-ary (q = 2s) symbols Symbols are read in a threshold-measurement process One symbol bit is provided in each step, starting from the MSB

Rami Cohen The q-ary Multi-Bit Channel April 2017 7 / 39

2 Binary: 10 Successful read requires s = log2q measurement steps

The q-ary Multi-Bit Channel (QMBC)

The q-ary Multi-Bit Channel

QMBC partial erasure: Measurement terminates after s − j steps (j = 0, 1, ..., s) Stored symbol belongs to a set of 2j consecutive symbols

Rami Cohen The q-ary Multi-Bit Channel April 2017 8 / 39

2 or 3? Partial erasure of size 2: only one measurement instead of two

The q-ary Multi-Bit Channel (QMBC)

The q-ary Multi-Bit Channel

Model Input alphabet: X = {0, 1, ..., q − 1}, q = 2s Mj

x denotes the partial-erasure set (of 2j symbols) due to s − j

measurements, given input symbol x Transition probabilities: Pr

• Y = Mj

x

• X = x
• = εj

QMBC capacity 1 −

s

• j=1

jεj s [q − ary symbols per channel use]

Rami Cohen The q-ary Multi-Bit Channel April 2017 9 / 39

The q-ary Multi-Bit Channel (QMBC)

The q-ary Multi-Bit Channel

Examples (q = 4) Pr (Y = y| X = 0) =      ε0

∆

= 1 − ε1 − ε2, y = M0

0 = {0}

ε1, y = M1

0 = {0, 1}

ε2, y = M2

0 = {0, 1, 2, 3}

Pr (Y = y| X = 3) =      ε0

∆

= 1 − ε1 − ε2, y = M0

3 = {3}

ε1, y = M1

3 = {2, 3}

ε2, y = M2

3 = {0, 1, 2, 3}

Rami Cohen The q-ary Multi-Bit Channel April 2017 10 / 39

The q-ary Multi-Bit Channel (QMBC)

Mapping to GF(q) elements

Consider a basis {ω1, ω2, ..., ωs} to GF(q) over GF(2), i.e., GF(q) = ω1, ω2, ..., ωs ∆ = s

• i=1

ai · ωi : ai ∈ {0, 1}

• .

Mapping

Mj

0 are mapped to ω1, ω2, ..., ωj, which are subgroups of GF+(q)

Mj

x are mapped to the cosets of the subgroups above, with coset

representatives taken from ωj+1, ωj+2, ..., ωs

Rami Cohen The q-ary Multi-Bit Channel April 2017 11 / 39

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder

LDPC Codes over GF(q)

Rami Cohen The q-ary Multi-Bit Channel April 2017 12 / 39

Example (k = 3, n = 7, GF(4)): H =     2 3 1 1 1 1 2 2 1 2 1 3 1 2     Variable nodes Edge labels Check nodes

Recall: Symbols are mapped to GF(q) elements

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder

LDPC Codes over GF(q)

Rami Cohen The q-ary Multi-Bit Channel April 2017 12 / 39

Example (k = 3, n = 7, GF(4)): H =     2 3 1 1 1 1 2 2 1 2 1 3 1 2    

2 · v1 + 3 · v3 + 1 · v5 + 1 · v7 = 0

Variable nodes Edge labels Check nodes

Recall: Symbols are mapped to GF(q) elements

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder

Set Iterative Decoder

Message passing ”Beliefs” (probabilities) about variable node values are exchanged iteratively over the graph edges in form of messages We use a non-standard message-passing algorithm, in which sets of symbols are exchanged The possible set values are based on local parity-check equations

Rami Cohen The q-ary Multi-Bit Channel April 2017 13 / 39

v1 → c2: I think I am {0, ω1} c2 → v1: I think you are {0, ω2}

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder

Set Iterative Decoder

Check-to-variable (CTV) messages Sumset of edge-label weighted variable-to-check (VTC) message sets Sumset: c4 → v6 = ˜ V1 + ˜ V4

∆

=

• ˜

v1 + ˜ v4 : ˜ v1 ∈ ˜ V1, ˜ v4 ∈ ˜ V4

• Rami Cohen

The q-ary Multi-Bit Channel April 2017 14 / 39

3 · v1 + v4 + 2 · v6 = 0

• V1

∆

= 3·v1

2

: v1 ∈ V1

• V4

∆

= v4

2 : v4 ∈ V4

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder

Set Iterative Decoder

Variable-to-check (VTC) messages Intersection of channel information and CTV message sets v6 → c3 = CI(v6) {c4 → v6}

Rami Cohen The q-ary Multi-Bit Channel April 2017 15 / 39

CI(v6)

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder

Subgroup Property

Theorem Iterative-decoding analysis can be restricted to CTV/VTC messages that are subgroups of GF+(q). Proof outline Sumset/intersection of cosets = coset

⇒ The CTV/VTC messages are cosets

Decoding failure: VTC message-set cardinality larger than 1 |Coset| = |Subgroup| Performance depends on the subgroups underlying the Mj

x cosets

All-zero codeword assumption holds

Rami Cohen The q-ary Multi-Bit Channel April 2017 16 / 39

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder

Complexity

Decoding analysis feasibility The number of subgroups (closed form) grows slowly with q Example (q = 16): 67 subgroups, compared to 65535 subsets (e.g., QPEC)

Rami Cohen The q-ary Multi-Bit Channel April 2017 17 / 39

(q = 2s is the field size)

The q-ary Multi-Bit Channel (QMBC) Decoding Threshold Region

Density-Evolution Analysis

Asymptotic analysis of iterative-decoding performance Tracking the CTV/VTC message probabilities as a function of the iteration number For the QMBC, only subgroups of GF+(q) need to be tracked

Much smaller compared to the 2q − 1 messages for general partial-erasure channels

Pr(Decoding success) = Pr(VTC messages are all {0})

Rami Cohen The q-ary Multi-Bit Channel April 2017 18 / 39

The q-ary Multi-Bit Channel (QMBC) Decoding Threshold Region

Density-Evolution Equations

Let Ht (i = 1, 2, .., T) denote the subgroups of GF+(q) w(l)

t

(z(l)

t ) is the probability of CTV (VTC) subgroup Ht at iteration l

CTV : w(l)

t

=

• SVTC

 

• m∈SVTC

z(l−1)

m

  · Pt (Hm∈SVTC, L) VTC : z(l)

t

=

s

• j=0

εj

• SCTV

 

• m∈SCTV

w(l)

m

  · Ij

t (Hm∈SCTV)

Performance depends on the edge-label distribution L

Rami Cohen The q-ary Multi-Bit Channel April 2017 19 / 39

The q-ary Multi-Bit Channel (QMBC) Decoding Threshold Region

Decoding Threshold Region

Decoding threshold L-region Given an LDPC graph, define for an edge-label distribution L: ΩL =

• ε1, ε2, ..., εs ∈ [0, 1]s : lim

l→∞ P (l) error(L) = 0

• .

Theorem If all the labels have the same value, QMBC is degraded to QEC (partial erasure is asymptotically equivalent to full erasure). How to choose L?

Rami Cohen The q-ary Multi-Bit Channel April 2017 20 / 39

The q-ary Multi-Bit Channel (QMBC) Decoding Threshold Region

Decoding Threshold Region

Theorem Suppose

• αis−1

i=0 is a basis to GF(q) and that the partial erasures are of

type j′, j′|s. Then the uniform distribution on

• αt·j′s/j′−1

t=0

is optimal. Proof outline CTV message is erasure iff the label on the CTV message edge is the same as the label on at least one incoming VTC message edge Optimality: QMBC capacity is achieved with capacity-achieving BEC ensembles + edge labels distributed as above

Rami Cohen The q-ary Multi-Bit Channel April 2017 21 / 39

Pr(VTC is j′-erased in iteration l) = yl Pr(CTV is j′-erased) = 1 −

• 1 −

yl s/j′

i−1

The q-ary Multi-Bit Channel (QMBC) Decoding Threshold Region

(3, 6) LDPC Ensemble The uniform distribution on the non-zero field elements is not optimal!

Rami Cohen The q-ary Multi-Bit Channel April 2017 22 / 39

↑

Optimal! BEC→

The q-ary Multi-Bit Channel (QMBC) Finite-Length Iterative-Decoding Analysis

Resolvable Edge Labels

κ-resolvability The edge labels h1, h2, ..., hκ are said to be κ-resolvable if v1, v2, ..., vκ are decoded correctly independently of other variable nodes Example:

• αt·j′s/j′−1

t=0

resolve s/j′ partial erasures of type at most j′

Rami Cohen The q-ary Multi-Bit Channel April 2017 23 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Iterative-Decoding Analysis

Algebraic Optimization Problem

Problem Find edge labels {hi}κ

i=1 such that the only solution to κ

• i=1

hi · vi = 0 is the trivial one, for any assignment of partial erasures ji such that

κ

• i=1

ji ≤ s. Example Consider {1, α} as a basis to GF(4) over GF(2), where α is a primitive

• element. If v1 and v2 are partially-erased to {0, 1} and h1 = 1, h2 = α, the
• nly solution is the trivial one, since 1 + α = 0.

Rami Cohen The q-ary Multi-Bit Channel April 2017 24 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Iterative-Decoding Analysis

Universal Edge Labeling

Theorem: Universal Edge Labeling (RC, YC, Raviv) There exist 2-resolvable edge labels for any j1 + j2 ≤ s, which can be found in O

• qlog2q
• multiplications.

Proof outline Resolvability is guaranteed if there is h such that the only solution to h · Mj1

0 + Mj2 0 = 0

Mj1

0 + h · Mj2 0 = 0

is the trivial solution (for all j1 + j2 ≤ s) Holds iff both {h · Mj1

0 , Mj2 0 } and {Mj1 0 , h · Mj2 0 } are bases

Existence of h is shown via a counting argument

Rami Cohen The q-ary Multi-Bit Channel April 2017 25 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Iterative-Decoding Analysis

Stopping Sets

Stopping set A subset S of variable nodes, such that all neighbours (check nodes) of S are connected to S at least twice. BEC: The iterative decoder gets stuck at erased stopping sets QMBC: Partially-erased stopping sets might be resolved Depends on the edge-label configuration

Rami Cohen The q-ary Multi-Bit Channel April 2017 26 / 39

This stopping set is resolved iff h1 = h2

The q-ary Multi-Bit Channel (QMBC) Finite-Length Iterative-Decoding Analysis

Edge-Labeling Algorithm

Re-labeling the edge labels of a given GF(q) LDPC graph for improved decoding performance We usually start with uniformly distributed edge labels Algorithm

1 Search for stopping sets using the BEC decoder 2 Rank variable nodes by their number of occurrences in the sets 3 Run over the stopping sets by ascending over of cardinality

Start with highest-ranking variable nodes, and modify edge labels to resolvable/universal edge labels

Rami Cohen The q-ary Multi-Bit Channel April 2017 27 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Iterative-Decoding Analysis

q = 4, (3, 27) LDPC Ensemble, Rate 8/9 The edge-labeling algorithm improves performance by orders of magnitude!

Rami Cohen The q-ary Multi-Bit Channel April 2017 28 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Maximum-Likelihood Decoding Analysis

Standard Non-Binary Random Ensemble

Standard non-binary random ensemble (SNBRE) A code in the SNBRE is defined by a parity-check matrix H whose entries are i.i.d. uniform random variables taken from GF(q). Definition: Partial linear independence Denote by HE the submatrix of H formed by columns corresponding to partially-erased variable nodes (the set E). The columns are said to be partially linearly independent if no non-zero vector consistent with the channel information exists in the null space of HE. Example:

Channel information: v1 ∈ {0, 1}, v2 ∈ {0, 1, 2, 3} (1, 0) is consistent, (2, 0) is not

Rami Cohen The q-ary Multi-Bit Channel April 2017 29 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Maximum-Likelihood Decoding Analysis

Standard Non-Binary Random Ensemble

Lemma Consider partial-erasure types that satisfy j|j′ for every j ≤ j′. Let o contain the partial-erasure types in descending order. Then, the probability that the columns of HE are partially linearly independent is: ψ

• {Ej}s

j=1

• =

|E|

• i=1
• 1 −

i−1

• l=1

2ol

• /qn−k
• .

Proof outline For column i, discard linear combinations of the previous columns Partial erasures are subfields ⇒ the linear combinations are distinct In the general case, we have a lower bound on ψ

Rami Cohen The q-ary Multi-Bit Channel April 2017 30 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Maximum-Likelihood Decoding Analysis

Standard Non-Binary Random Ensemble

Theorem: Expected probability of maximum-likelihood decoding failure Let Ej denote the indices of variable nodes partially-erased to Mj

• 0. Then

ESNBRE

• P ML (H)
• =
• |Ej|

n! |E0|! |E1|!... |Es|!

s

• j=0

εj|Ej|×

• 1 − ψ
• {Ej}s

j=1

• .

Proof outline Run over all combinations of partial erasures Multiply by the probability of the combination and the probability that the columns of HE are partially linearly dependent

Rami Cohen The q-ary Multi-Bit Channel April 2017 31 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Maximum-Likelihood Decoding Analysis

Standard Non-Binary Random Ensemble

QMBC (solid) vs QEC (dashed) SNBRE ML performance (rate 8/9)

Rami Cohen The q-ary Multi-Bit Channel April 2017 32 / 39

Shannon limit

↑

q = 4 n = 128, 256, 512

The q-ary Multi-Bit Channel (QMBC) Finite-Length Maximum-Likelihood Decoding Analysis

LDPC Ensembles

Lemma Consider a vector a of length m ≥ 2, whose entries are i.i.d. random variables uniformly distributed on the non-zero GF(q = 2s) elements. The probability that the entries of a sum to 0 is Pr m

• i=1

ai = 0

• = 1 − (1 − q)1−m

q ≤ 1 q − 1.

Rami Cohen The q-ary Multi-Bit Channel April 2017 33 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Maximum-Likelihood Decoding Analysis

LDPC Ensembles

Lemma The number of consistent vectors with w non-zero entries is η (w) =

• u:

s

• j=1

uj=w, uj≤|Ej| s

• j=1

|Ej| uj

• 2j − 1

uj Note that η (w) reduces to |E|

w

• for the BEC: The erasure sets are {0, 1}

so η (w) is the number of vectors of Hamming weight w

Rami Cohen The q-ary Multi-Bit Channel April 2017 34 / 39

The q-ary Multi-Bit Channel (QMBC) Finite-Length Maximum-Likelihood Decoding Analysis

LDPC Ensembles

Theorem: Expected probability of ML decoding failure ELDPC(dv,dc)

• P ML (G)
• ≤
• |Ej|

n! |E0|! |E1|!... |Es|!

s

• j=0

εj|Ej| · min

• 1,

|E|

• w=1

η(w) coef

• (1 + y)dc − 1 − ydc

n dv

dc , ywdv

• ndv

wdv

• 1

q − 1 w dv

dc

. Proof outline

1 Counting consistent vectors with w non-zero elements 2 Restricting to stopping sets 3 Upper bounding the probability that all the check nodes connected to

non-zero variable nodes are satisfied

Rami Cohen The q-ary Multi-Bit Channel April 2017 35 / 39

(1) (2) (3)

The q-ary Multi-Bit Channel (QMBC) Finite-Length Maximum-Likelihood Decoding Analysis

LDPC Ensembles

QMBC (solid) vs QEC (dashed) LDPC ML performance ((3, 27), rate 8/9, GF(4), code length 252)

Rami Cohen The q-ary Multi-Bit Channel April 2017 36 / 39

Suggestions for Future Research

Table of Contents

1

The q-ary Multi-Bit Channel (QMBC) Set Iterative Decoder Decoding Threshold Region Finite-Length Iterative-Decoding Analysis Finite-Length Maximum-Likelihood Decoding Analysis

2

Suggestions for Future Research

Rami Cohen The q-ary Multi-Bit Channel April 2017 37 / 39

Suggestions for Future Research

Suggestions for Future Research

Approximation of partial-erasure channels by binary channels

Shown for the QPEC [Mayer-Kelley ’16]

Local resolvability and globally optimal edge-label distributions Analysis of the set iterative decoder on additional models Optimal edge-label distributions for combinations of partial-erasure types

Rami Cohen The q-ary Multi-Bit Channel April 2017 38 / 39

Suggestions for Future Research Rami Cohen The q-ary Multi-Bit Channel April 2017 39 / 39