ECEN 5682 Theory and Practice of Error Control Codes Block Code - - PowerPoint PPT Presentation

Block Code Performance

ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Peter Mathys

University of Colorado

Spring 2007

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Performance Measures

“Goodness” Criteria for Block Codes. How does one select the “best” code for a particular application? This is not an easy question to answer in general because many factors, some of which are only marginally related to coding theory (e.g. the wordlength used on a VLSI chip), need to be considered. But since error control codes are used to control the effect of transmission or channel errors, the probability of error after encoding and decoding with a code C is certainly one of the major, if not the most important, criterion for “goodness”. Depending on whether a code is used for error detection and/or error correction, we are interested in the probability of an undetected error Pu(E), or in the probability of a decoding error P(E).

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Probability of Undetected Error

Theorem: Let C be any binary code with minimum (Hamming) distance dmin. Then the probability of undetected error, Pu(E), on a memoryless binary symmetric channel (BSC) with transition probability ǫ satisfies Pu(E) ≤

n

• w=dmin

n w

• ǫw (1 − ǫ)n−w .

Proof: Since the code has minimum distance dmin, it can detect all error patterns of (Hamming) weight dmin − 1 or less. QED

Note: The above theorem also holds for memoryless q-ary symmetric channels (QSC) with transition probabilities (x denotes input, y denotes output of channel) P{y=β|x=α} =  1 − ǫ , if α = β, α, β ∈ {0, 1, . . . q − 1} ,

ǫ q−1 ,

if α = β, α, β ∈ {0, 1, . . . q − 1} .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Note: Usually, many more error patterns can be detected than those indicated by the above theorem. A q-ary linear (n, k) code, for example, can detect qn − qk error patterns. Definition: Let Aw be the number of codewords of a linear (n, k) code with (Hamming) weight w. Then {A0, A1, . . . , An} is called the weight distribution of the code. Example: Code #1 (binary (5, 2, 3) code) with generator matrix G = 1 1 1 1 1 1

• ,

has codewords C = {00000, 01011, 10101, 11101} , and thus weight distribution {A0 = 1, A3 = 2, A4 = 1, Aw = 0, w = 0, 3, 4} .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Theorem: Let C be a binary linear (n, k) code with weight distribution {A0, A1, . . . , An} which is used on a binary symmetric channel (BSC) with transition probability ǫ. Then the probability

• f undetected error, Pu(E), is given by

Pu(E) =

n

• w=dmin

Aw ǫw (1 − ǫ)n−w . Note that if C has minimum distance dmin, then A1, A2, . . . , Admin−1 are all zero. Proof: Every error pattern that is equal to a codeword cannot be detected when a linear code is used. Note: For a q-ary symmetric channel (QSC) with transition probabilities as given earlier, this theorem becomes Pu(E) =

n

• w=dmin

Aw

• ǫ

q − 1 w (1 − ǫ)n−w .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Examples: The weight distributions {Aw} of a few small binary codes are:

• (7, 4, 3) Hamming code:

{A0 = 1, A3 = 7, A4 = 7, A7 = 1} .

• (15, 11, 3) Hamming code:

{A0 = 1, A3 = 35, A4 = 105, A5 = 168, A6 = 280, A7 = 435, A8 = 435, A9 = 280, A10 = 168, A11 = 105, A12 = 35, A15 = 1} .

• (31, 26, 3) Hamming code:

{A0 = 1, A3 = 155, A4 = 1085, A5 = 5208, A6 = 22568, A7 = 82615, A8 = 247845, A9 = 628680, A10 = 1383096, A11 = 2648919, A12 = 4414865, A13 = 6440560, A14 = 8280720, A15 = 9398115, A16 = 9398115, A17 = 8280720, A18 = 6440560, A19 = 4414865, A20 = 2648919, A21 = 1383096, A22 = 628680, A23 = 247845, A24 = 82615, A25 = 22568, A26 = 5208, A27 = 1085, A28 = 155, A31 = 1} .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Examples: (contd.)

• (31, 21, 5) BCH code:

{A0 = 1, A5 = 186, A6 = 806, A7 = 2635, A8 = 7905, A9 = 18910, A10 = 41602, A11 = 85560, A12 = 142600, A13 = 195300, A14 = 251100, A15 = 301971, A16 = 301971, A17 = 251100, A18 = 195300, A19 = 142600, A20 = 85560, A21 = 41602, A22 = 18910, A23 = 7905, A24 = 2635, A25 = 806, A26 = 186, A31 = 1} .

• (31, 16, 7) BCH code:

{A0 = 1, A7 = 155, A8 = 465, A11 = 5208, A12 = 8680, A15 = 18259, A16 = 18259, A19 = 8680, A20 = 5208, A23 = 465, A24 = 155, A31 = 1} .

• (31, 11, 11) BCH code:

{A0 = 1, A11 = 186, A12 = 310, A15 = 527, A16 = 527, A19 = 310, A20 = 186, A31 = 1} .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

The (exact) probability of undetected error for these six codes, when used on a memoryless BSC with transition probability ǫ, is shown in the following graph.

−6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 10

−20

10

−18

10

−16

10

−14

10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 log10(epsilon) for BSC log10Pu(E) Probability of Undetected Error (7,4,3) Code (15,11,3) Code (31,26,3) Code (31,21,5) Code (31,16,7) Code (31,11,11) Code

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

The upper three curves show that if dmin is fixed and the blocklength n is increased, then the probability of undetected error Pu(E) increases. The lower three curves show that if n is fixed and dmin is increased, then Pu(E) decreases quite rapidly. But note that the code rate R = k/n, which does not show up explicitly in the above graph, also needs to be taken into account when designing a system that uses error control. A lower code rate means in general that either data can be sent less rapidly over a given channel, or the channel bandwidth needs to be increased. Definition: Let {A0, A1, . . . , An} be the weight distribution of a linear (n, k) code C and let {B0, B1, . . . , Bn} be the weight distribution of its dual code C⊥. Define the weight distribution polynomials of C and C⊥ as A(z) =

n

• w=0

Aw zw and B(z) =

n

• w=0

Bw zw .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Example: Code #1 with weight distribution {A0=1, A3=2, A4=1} has weight distribution polynomial A(z) = 1 + 2 z3 + z4.

Theorem: MacWilliams Identity for Binary Codes. The weight distribution polynomials, A(z) and B(z), of a binary linear (n, k) code C and its dual code C⊥ are related by A(z) = 2−(n−k) (1 + z)n B 1 − z 1 + z

• .

Theorem: MacWilliams Identity for q-ary Codes. The weight distribution polynomials, A(z) and B(z), of a q-ary linear (n, k) code C and its dual code C⊥ are related by A(z) = q−(n−k) (1 + (q − 1)z)n B

• 1 − z

1 + (q − 1)z

• .

Proof: See F.J. MacWilliams, “A Theorem on the Distribution of Weights in a Systematic Code,” Bell Syst. Tech. J., vol. 42, 1963, pp. 79–94.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Example: Consider the binary (5, 3) code with generator matrix G =   1 1 1 1 1 1 1   . Its dual code is code #1, the (5, 2) code with B(z) = 1 + 2 z3 + z4 given in a previous example. Using the MacWilliams identity, one finds that A(z) = 2−2 (1 + z)5

• 1 + 2
• 1−z

1+z

3 +

• 1−z

1+z

4 = 1

4

• (1 + z)5 + 2 (1 − z)3 (1 + z)2 + (1 − z)4 (1 + z)
• = 1 + 2 z2 + 4 z3 + z4 ,

and thus the weight distribution of this (5, 3) code is {A0 = 1, A2 = 2, A3 = 4, A4 = 1, Ai = 0, i = 0, 2, 3, 4} .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Theorem: In terms of the weight distribution polynomials A(z) (for code C) and B(z) (for code C⊥), the probability of undetected error for a linear binary (n, k) code C used on a BSC with transition probability ǫ, can be expressed as Pu(E) = (1 − ǫ)n A

• ǫ

1 − ǫ

• − 1
• = 2−(n−k) B(1 − 2ǫ) − (1 − ǫ)n .

Proof: For the first equality note that

Pu(E)=

n

X

w=dmin

Awǫw(1−ǫ)n−w=(1−ǫ)n

n

X

w=dmin

Aw ` ǫ 1 − ǫ ´w=(1−ǫ)n“ A ` ǫ 1 − ǫ ´ −1 ” .

For the second equality use MacWilliams’ identity A(z) = 2−(n−k)(1 + z)n B 1 − z 1 + z

• ,

for binary (n, k) codes.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Probability of Decoding Error

Theorem: Let C be any binary code with minimum (Hamming) distance dmin and let t = ⌊(dmin − 1)/2⌋. Then the probability of a (block) decoding error, PB(E), on a memoryless BSC with transition probability ǫ is upper bounded by PB(E) ≤

n

• w=t+1

n w

• ǫw (1−ǫ)n−w = 1−

t

• w=0

n w

• ǫw (1 − ǫ)n−w
• = PB(C)

. Proof: Any code with minimum distance dmin can correct all error patterns of t = ⌊(dmin − 1)/2⌋ or less (random) errors. QED

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Example: Consider bounded distance decoding of all error patterns up to t errors, t = ⌊(dmin − 1)/2⌋, using binary codes with the following parameters (7, 4, 3), (15, 11, 3), (31, 26, 3), (31, 21, 5), (31, 16, 7), and (31, 11, 11). Upper bounds on the probability of decoding error PB(E) when using a BSC with transition probability ǫ are given in the graph on the next slide. Note that the first three codes are Hamming codes and since these are perfect codes, the PB(E) curves are exact rather than bounds. As was the case for error detection, the upper three curves show that if dmin is kept fixed and n is increased, then the probability of error is increased. Conversely, if the blocklength n is kept fixed and dmin is increased, then PB(E) decreases quite rapidly as can be seen from the lower three curves.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error −6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 log10(epsilon) for BSC log10PB(E) Probability of Decoding Error for Bounded Distance Decoder (7,4,3) Code (15,11,3) Code (31,26,3) Code (31,21,5) Code (31,16,7) Code (31,11,11) Code Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Note: In most cases a t-error-correcting code can also correct many patterns of t + 1 or even more errors. A q-ary linear (n, k) code, for example, can correct qn−k − 1 error patterns (including those with t or less errors). The bound on the probability of decoding error given above assumes a bounded distance decoder that only corrects up to t

• errors. If a complete decoder is used, one should in most cases

(except for perfect codes) be able to do better. Let X and Y be random vectors of length n. For any given channel model, let X denote the channel input and Y denote the channel output. A particular channel model is specified by giving the conditional probability mass function (pmf) pY|X(y|x). Assume that the code C = {ci}, i = 0, 1, . . . , M − 1, is used and the transmission of a codeword from C over the specified channel results in the received n-tuple v.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

The decoder then needs to look at the a posteriori probabilities pX|Y(ci|v) = pXY(civ) pY(v) = pY|X(v|ci) pX(ci) pY(v) , for i = 0, 1, . . . , M − 1. Definition: For a given code C = {c0, c1, . . . , cM−1}, a priori pmf pX(c), channel model pY|X(v|c) and received n-tuple v, a maximum a posteriori (MAP) decoder outputs estimate ˆ c = ci iff i is the index which maximizes the expression pY|X(v|ci) pX(ci) . If there is more than one index that maximizes this expression, one

• f the maximizing indexes can be chosen at random without loss of
• ptimality.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Definition: For a given code C = {c0, c1, . . . , cM−1}, channel model pY|X(v|c) and received n-tuple v, a maximum likelihood (ML) decoder outputs estimate ˆ c = ci iff i is the index which maximizes the expression pY|X(v|ci) . If there is more than one index that maximizes this expression, one

• f the maximizing indexes can be chosen at random without loss of
• ptimality.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Note: If an additive error model is used, then the conditional probability pY|X(v|c) is equal to the probability pE(v − c) that the error pattern e = v − c occurs. If the channel model is such that lower (Hamming) weight error patterns are more likely to occur than higher weight patterns, then the ML decoder outputs ˆ c = ci iff i is the index that minimizes the (Hamming) distance dH(v, ci). If there are several indexes that minimize this distance, then one of these indexes can be chosen at random without loss of optimality. Note: If pX(ci) = 1/M for all i = 0, 1, . . . , M − 1 (uniform distribution of codewords), then the MAP decoder and the ML decoder are the same.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

For ML decoders an upper bound for the block error probability PB(E) can be derived as follows. Define Pd(E) = P{Decoding error between two codewords distance d apart}. Then, assuming a linear code with M equally likely codewords, one can use a union bound (i.e., the probability of a union of events is upper bounded by the sum of the probabilities of the events) to write PB(E) ≤

M−1

• m=1

Pwm(E) , where wm, m = 1, 2, . . . , M − 1, are the weights of all nonzero codewords of the code.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Using the weight distribution {Aw} of the code one can sum over all weights and multiply by the number of codewords with each weight, rather than sum over all codewords individually, and thus PB(E) ≤

n

• w=dmin

Aw Pw(E) . The probability Pd(E) of a decoding error between two codewords distance d apart is the probability that an error pattern occurs with d/2 or more errors in the d positions in which the two codewords

• differ. To be precise in the case when d is an even integer, only
• ne half of the cases when exactly d/2 errors occur causes an error
• n the average. Thus, one obtains the following upper bound (or

exact expression when d is an odd integer) for Pd(E) Pd(E) ≤

d

• e=⌈d/2⌉

d e

• ǫe (1 − ǫ)d−e .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Substituting this in the union bound above the following theorem is proved. Theorem: Let C be a binary block code with blocklength n, minimum (Hamming) distance dmin, and weight distribution {Aw}. Then, using a ML decoder, the probability of a (block) decoding error PB(E) on a memoryless BSC with transition probability ǫ is upper bounded by PB(E) ≤

n

• w=dmin

Aw

w

• e=⌈w/2⌉

w e

• ǫe (1 − ǫ)w−e .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

A somewhat weaker bound, based on the so called Bhattacharyya distance between two codewords, which is dB = −0.5 log(4ǫ(1 − ǫ)) for the BSC with transition probability ǫ, is given in the following theorem. Theorem: Let C be a binary block code with blocklength n, minimum (Hamming) distance dmin, and weight distribution {Aw}. Then, using a ML decoder, the probability of a (block) decoding error PB(E) on a memoryless BSC with transition probability ǫ is upper bounded by PB(E) ≤ 1 2

n

• w=dmin

Aw

• 4ǫ (1 − ǫ)

w/2 .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Proof: The probability Pd(E) of a decoding error between two codewords (Hamming) distance d apart can be upper bounded as follows

Pd(E) ≤

d

X

e=⌈d/2⌉

d e ! ǫe (1 − ǫ)d−e = (1 − ǫ)d

d

X

e=⌈d/2⌉

d e ! „ ǫ 1 − ǫ «e ≤ (1 − ǫ)d „ ǫ 1 − ǫ «d/2

d

X

e=⌈d/2⌉

d e ! = ǫd/2 (1 − ǫ)d/2

d

X

e=⌈d/2⌉

d e ! | {z } ≤ 2d/2 ≤ 1 2 2d ǫd/2 (1 − ǫ)d/2 = 1 2 ˆ 4 ǫ (1 − ǫ) ˜d/2 .

Substituting this in the union bound for PB(E) completes the proof

• f the theorem.

QED

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Suppose now that it is desired to compare the “goodness” of two codes with different k. The probability of a block error is then not really the best measure. Rather, one should compare the probability of a data symbol error Ps(E) or, in the binary case, the probability of a data bit error Pb(E). Because a block error will affect between 1 and k data symbols, the probability of a symbol error can be bounded as 1 k PB(E) ≤ Ps(E) ≤ PB(E) , but for large k there is quite a gap between the upper and the lower bounds. Another approach is to extend the concept of the weight distribution of a code so that it also contains information about the weight of the data symbols.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Definition: Let A(w, i) be the number of codewords of a linear (n,k) code with total weight w and data weight i (both using Hamming weight). Then the set {A(w, i)}, w = 0, 1, . . . , n, i = 0, 1, . . . , k, is called the extended weight distribution of the code. If {A(w, i)} is known, then it is easy to obtain {Aw} using Aw =

k

• i=0

A(w, i) , w = 0, 1, . . . , n . Substituting this expression in the union bound for the probability

• f a block error yields

PB(E) ≤

n

• w=dmin

k

• i=0

A(w, i) Pw(E) .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

To obtain an expression for the probability of a symbol error, multiply A(w, i) by the number i of data symbol errors in the formula above and then divide by k to obtain Ps(E) ≤ 1 k

n

• w=dmin

k

• i=1

i A(w, i) Pw(E) . Substituting the expression for Pw(E) on a BSC with transition probability ǫ thus proves the following theorem. Theorem: Let C be a binary (n, k, dmin) block code with extended weight distribution {A(w, i)}. Using a ML decoder, the probability

• f a bit error Pb(E) on a memoryless BSC with transition

probability ǫ is then upper bounded by Pb(E) ≤ 1 k

n

• w=dmin

k

• i=1

i A(w, i)

w

• e=⌈w/2⌉

w e

• ǫe (1 − ǫ)w−e .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Example: Bounds on the probability of block error for a binary (7, 4, 3) Hamming code on a memoryless BSC with transition probability ǫ are shown in the graph on the next page. Clearly, the upper bound on Pu(E) based on dmin is quite a bit weaker than the exact value which uses the weight distribution {Aw} of the code. Because Hamming codes are perfect codes, the value of the block error probability PB(E) based on dmin is exact for both bounded distance and ML decoders. Note that the simpler bound for PB(E) of a ML decoder which uses the Bhattacharyya distance is quite loose, mostly because the (7, 4, 3) code has both even and odd weight codewords.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

−6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 log10(epsilon) for BSC log10 Pu(E), log10 PB(E) Upper Bounds on Pu(E) and PB(E) for (7,4,3) Code on BSC Pu(E) from dmin Pu(E) exact with Aw PB(E) from dmin PB(E) Union Bound PB(E) Bhattacharyya

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Bounds on the probability of bit error are shown below.

−6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 log10(epsilon) for BSC log10 Pb(E) Bounds on Probability of Bit Error for (7,4,3) Code on BSC Union Bound Upper Bound PB(E) Lower Bound PB(E)/k

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

The upper and lower bounds, which use the more complicated but also more tight bound on PB(E) for ML decoding, are not too far apart in this case because k is rather small. Example: Bounds for the block and bit error probabilities of a binary (31, 26, 3) Hamming code on a memoryless BSC with transition probability ǫ are shown in the next two figures. Similar comments as for the (7, 4, 3) code apply. Generally, the bounds that were loose already for the (7, 4, 3) code become even more loose as the blocklength n is increased (and dmin is kept fixed). In part this is due to the fact that Hamming codes are perfect codes, and in part it comes from the increase in the data length k.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

−6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 log10(epsilon) for BSC log10 Pu(E), log10 PB(E) Upper Bounds on Pu(E) and PB(E) for (31,26,3) Code on BSC Pu(E) from dmin Pu(E) exact with Aw PB(E) from dmin PB(E) Union Bound PB(E) Bhattacharyya

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

−6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 log10(epsilon) for BSC log10 Pb(E) Bounds on Probability of Bit Error for (31,26,3) Code on BSC Union Bound Upper Bound PB(E) Lower Bound PB(E)/k

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Example: Bounds for the block and bit error probabilities of a binary (31, 11, 11) BCH code on a memoryless BSC with transition probability ǫ are shown in the next two graphs. Because of the larger dmin, the error probabilities at the decoder

• utput now decrease much more rapidly as ǫ decreases. More

interestingly, the bound on PB(E) for a ML decoder is now below the bound for bounded distance decoding in the main region of interest (i.e., PB(E) < 10−4). This is a direct consequence of the fact that the (31, 11, 11) code is not a perfect code. Note that the bound on Pu(E) based on dmin is quite loose and essentially useless. The upper and lower bounds on the probability of bit error, which are based on PB(E) for a ML decoder, are not too far apart because k is relatively small.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

−6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 log10(epsilon) for BSC log10 Pu(E), log10 PB(E) Upper Bounds on Pu(E) and PB(E) for (31,11,11) Code on BSC Pu(E) from dmin Pu(E) exact with Aw PB(E) from dmin PB(E) Union Bound PB(E) Bhattacharyya

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

−6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 log10(epsilon) for BSC log10 Pb(E) Bounds on Probability of Bit Error for (31,11,11) Code on BSC Union Bound Upper Bound PB(E) Lower Bound PB(E)/k

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

The bounds considered up to now were based on the BSC channel model and thus assumed that hard decisions are made by the receiver before decoding. To get an idea of the performance improvements that are possible if the receiver uses soft decisions, consider binary antipodal signaling with message bits m0 → −√Eb and m1 → +√Eb over an additive white Gaussian noise (AWGN) channel with two-sided noise power spectral density (PSD) N0/2. Assume that the receiver uses a matched filter (MF) and that both Eb and N0 are measured at the output of the MF at the optimum sampling time instant at which the SNR Eb/N0 is maximized. Theorem: The probability ǫ of an uncoded bit error for binary antipodal signaling over an AWGN channel with MF receiver and decision rule ˆ m = m0 iff the MF output b < 0 (at the optimum sampling time instant) is given by ǫ = 1 2erfc

• Eb

N0

• ,

where erfc(x) = 2 √π ∞

x

e−µ2 dµ .

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Proof: The output b of the matched filter at the optimum sampling time instant (which maximizes the SNR Eb/N0) is Gaussian with variance σ2

b = N0/2 and mean either −√Eb or

+√Eb, i.e.,

fb(β|m0) = e−(β+√

Eb)2/N0

√π N0 ,

• r

fb(β|m1) = e−(β−√

Eb)2/N0

√π N0 .

Setting the decision threshold at β = 0 yields

P(E|m0)= 1 √πN0 Z ∞ e−(β+√

Eb)2/N0dβ= 1

√π Z ∞ √

Eb/N0

e−µ2dµ=1 2erfc “r Eb N0 ” .

A similar computation yields P(E|m1) = P(E|m0) and thus

ǫ = P(E|m0) P(m0) + P(E|m1) P(m1) = 1 2erfc “r Eb N0 ” . QED

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Now consider the case where message mi is encoded into codeword ci = (ci0, ci1, . . . , ci n−1) and then transmitted using antipodal signaling such that the receiver (after sampling at the output of the MF) sees mi → ci − → (2ci0−1, 2ci1−1, . . . , 2ci n−1−1)

• Eb .

Thus, if the receiver has to distinguish between mi → ci and mj → cj and d(ci, cj) = d, then this is equivalent to d uncoded decoding decisions (assuming a memoryless channel). This proves the following Theorem: The probability Pd(E) of a decoding error between two codewords (Hamming) distance d apart for binary antipodal signaling over an AWGN channel with MF receiver and ML decision rule is given by Pd(E) = 1 2erfc

• d Eb

N0

• ,

where erfc(x) = 2 √π ∞

x

e−µ2 dµ.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Combining the union bound with the result for Pd(E) from the previous page results in the following theorem. Theorem: Probability of Block Error for Soft Decisions. Let C be a linear binary (n, k, dmin) block code with weight distribution {Aw}. Then, using antipodal signaling over an AWGN channel and (soft-decision) ML decoding with a matched filter at the receiver, the probability of a block decoding error PB(E) is upper bounded by PB(E) ≤ 1 2

n

• w=dmin

Aw erfc

• w Eb

N0

• ,

where Eb is the bit energy of the code bits and N0/2 is the noise power (or noise variance), both at the output of the MF at the

• ptimum sampling time instant.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Replacing {Aw} by {A(w, i)} finally yields the following theorem. Theorem: Probability of Bit Error for Soft Decisions. Let C be a linear binary (n, k, dmin) block code with extended weight distribution {A(w, i)}. Then, using antipodal signaling over an AWGN channel and (soft-decision) ML decoding with a matched filter at the receiver, the probability of a bit decoding error Pb(E) is upper bounded by Pb(E) ≤ 1 2k

n

• w=dmin

erfc

• w Eb

N0

• k
• i=1

i A(w, i) , where Eb is the bit energy of the code bits and N0/2 is the noise power (or noise variance), both at the output of the MF at the

• ptimum sampling time instant.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

Example: Bounds on the probabilities of block error for hard and soft decisions of the binary (7,4,3) Hamming code and the binary (23,12,7) Golay code are shown on the following two pages. For the soft decision error probabilities the SNR Eb/N0 was converted to ǫ of a BSC using ǫ = 1 2erfc

• Eb

N0

• .

For comparison purposes, the probability of undetected error was also included in the graphs.

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

−4 −3.5 −3 −2.5 −2 −1.5 −1 10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 Binary (7,4,3) Hamming Code on BSC log10(ε) PB(E), Pu(E) PB(E) dmin PB(E) union PB(E) Bhattcharyya PB(E) AWGN soft Pu(E)

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Block Code Performance Performance Measures Probability of Undetected Error Probability of Decoding Error

−4 −3.5 −3 −2.5 −2 −1.5 −1 10

−25

10

−20

10

−15

10

−10

10

−5

10 Binary (23,12,7) Golay Code on BSC log10(ε) PB(E), Pu(E) PB(E) dmin PB(E) union PB(E) Bhattcharyya PB(E) AWGN soft Pu(E)

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes