some channel models Input X P(y|x) output Y 1-p Error Source 0 - - PDF document

▶

Sep 05, 2023 276 likes •324 views

Example: binary symmetric channel (BSC) some channel models Input X P(y|x) output Y 1-p Error Source 0 0 E transition probabilities p Y X E X = ! + 1 1 memoryless: Output Input 1-p - output at time i depends only on input at

SLIDE 1

some channel models

Input X P(y|x)

utput Y

transition probabilities memoryless:

output at time i depends only on input at time i
input and output alphabet finite

Example: binary symmetric channel (BSC)

Error Source +

E X

Output Input

E X Y ! =

E is the binary error sequence s.t. P(1) = 1-P(0) = p X is the binary information sequence Y is the binary output sequence 1-p p 1 1 1-p

from AWGN to BSC

Homework: calculate the capacity as a function of A and σ2

p

Other models

1 0 (light on) 1 (light off)

p 1-p

X Y P(X=0) = P0 1 E 1

1-e e e 1-e

P(X=0) = P0 Z-channel (optical) Erasure channel (MAC)

Erasure with errors

1 E 1 p p e e 1-p-e 1-p-e

burst error model (Gilbert-Elliot)

Error Source

Random Random error channel; outputs independent P(0) = 1- P(1); Burst Burst error channel; outputs dependent

Error Source

P(0 | state = bad ) = P(1|state = bad ) = 1/2; P(0 | state = good ) = 1 - P(1|state = good ) = 0.999

State info: good or bad

good bad

transition probability

Pgb Pbg Pgg Pbb

SLIDE 2

channel capacity:

I(X;Y) = H(X) - H(X|Y) = H(Y) – H(Y|X) (Shannon 1948)

H(X) H(X|Y)

notes: capacity depends on input probabilities because the transition probabilites are fixed channel X Y

!

= = = ) ( * ) ( ) ( ) ; ( max i I i p p E Entropy H capacity Y X I

Practical communication system design

message estimate channel decoder

n

Code word in receive

There are 2k code words of length n

k is the number of information bits transmitted in n channel uses

2k

Code book Code book

with errors

Channel capacity

Definition: The rate R of a code is the ratio k/n, where

k is the number of information bits transmitted in n channel uses

Shannon showed that: : for R ≤ C encoding methods exist with decoding error probability 0

Encoding and decoding according to Shannon

Code: 2k binary codewords where p(0) = P(1) = ½ Channel errors: P(0 →1) = P(1 → 0) = p i.e. # error sequences ≈ 2nh(p) Decoder: search around received sequence for codeword with ≈ np differences

space of 2n binary sequences

decoding error probability

1. For t errors: |t/n-p|> Є → 0 for n → ∞

(law of large numbers)

2. > 1 code word in region

(codewords random)

! " # < = " = " # $ >

# # # # #

n and ) p ( h 1 n k R for 2 2 2 2 ) 1 2 ( ) 1 ( P

) R BSC C ( n ) R ) p ( h 1 ( n n ) p ( nh k

channel capacity: the BSC

1-p p 1 1 1-p X Y I(X;Y) = H(Y) – H(Y|X) the maximum of H(Y) = 1

since Y is binary

H(Y|X) = h(p)

= P(X=0)h(p) + P(X=1)h(p)

Conclusion: the capacity for the BSC CBSC = 1- h(p)

Homework: draw CBSC , what happens for p > ½

SLIDE 3

channel capacity: the BSC

0.5 1.0 1.0

Bit error p Channel capacity

Explain the behaviour!

channel capacity: the Z-channel

Application in optical communications 1 0 (light on) 1 (light off)

p 1-p

X Y H(Y) = h(P0 +p(1- P0 ) ) H(Y|X) = (1 - P0 ) h(p) For capacity, maximize I(X;Y) over P0 P(X=0) = P0

channel capacity: the erasure channel

Application: cdma detection 1 E 1

1-e e e 1-e

X Y I(X;Y) = H(Y)– H(Y|X) H(Y) = h(P0 ) H(Y|X) = e h(P0) Thus Cerasure = 1 – e

(check!, draw and compare with BSC and Z)

P(X=0) = P0

Erasure with errors: calculate the capacity!

1 E 1 p p e e 1-p-e 1-p-e example



Consider the following example

1/3 1/3 1 2 1 2 

For P(0) = P(2) = p, P(1) = 1-2p H(Y) = h(1/3 – 2p/3) + (2/3 + 2p/3); H(Y|X) = (1-2p)log23 Q: maximize H(Y) – H(Y|X) as a function of p Q: is this the capacity? hint use the following: log2x = lnx / ln 2; d lnx / dx = 1/x

channel models: general diagram

x1 x2 xn y1 y2 ym : : : : : : P1|1 P2|1 P1|2 P2|2 Pm|n Input alphabet X = {x1, x2, …, xn} Output alphabet Y = {y1, y2, …, ym} Pj|i = PY|X(yj|xi) In general: calculating capacity needs more theory

The statistical behavior of the channel is completely defined by the channel transition probabilities Pj|i = PY|X(yj|xi)

SLIDE 4

* clue:

I(X;Y) is convex ∩ in the input probabilities i.e. finding a maximum is simple

Channel capacity: converse

For R > C the decoding error probability > 0 k/n C Pe

Converse: For a discrete memory less channel

1 1 1 1

( ; ) ( ) ( | ) ( ) ( | ) ( ; )

n n n n n n n i i i i i i i i i i i

I X Y H Y H Y X H Y H Y X I X Y nC

= = = =

= ! " ! = "

# # # # Xi Yi m Xn Yn m‘

encoder channel channel

Source generates one

ut of 2k equiprobable

messages

decoder

Let Pe = probability that m‘ ≠ m

source

converse R := k/n

Pe ≥ 1 – C/R - 1/nR Hence: for large n, and R > C, the probability of error Pe > 0 k = H(M) = I(M;Yn)+H(M|Yn) Xn is a function of M Fano ≤ I(Xn;Yn) + 1 + k Pe ≤ nC + 1 + k Pe 1 – C n/k - 1/k ≤ Pe

some channel models

Input X P(y|x)

transition probabilities memoryless:

Example: binary symmetric channel (BSC)

Error Source +

E X

Output Input

E X Y ! =

E is the binary error sequence s.t. P(1) = 1-P(0) = p X is the binary information sequence Y is the binary output sequence 1-p p 1 1 1-p

from AWGN to BSC

Homework: calculate the capacity as a function of A and σ2

p

Other models

1 0 (light on) 1 (light off)

X Y P(X=0) = P0 1 E 1

P(X=0) = P0 Z-channel (optical) Erasure channel (MAC)

Erasure with errors

1 E 1 p p e e 1-p-e 1-p-e

burst error model (Gilbert-Elliot)

Error Source

Random Random error channel; outputs independent P(0) = 1- P(1); Burst Burst error channel; outputs dependent

Error Source

P(0 | state = bad ) = P(1|state = bad ) = 1/2; P(0 | state = good ) = 1 - P(1|state = good ) = 0.999

State info: good or bad

transition probability

channel capacity:

I(X;Y) = H(X) - H(X|Y) = H(Y) – H(Y|X) (Shannon 1948)

H(X) H(X|Y)

notes: capacity depends on input probabilities because the transition probabilites are fixed channel X Y

!

= = = ) ( * ) ( ) ( ) ; ( max i I i p p E Entropy H capacity Y X I

Practical communication system design

n

There are 2k code words of length n

2k

with errors

Channel capacity

Definition: The rate R of a code is the ratio k/n, where

Shannon showed that: : for R ≤ C encoding methods exist with decoding error probability 0

Encoding and decoding according to Shannon

Code: 2k binary codewords where p(0) = P(1) = ½ Channel errors: P(0 →1) = P(1 → 0) = p i.e. # error sequences ≈ 2nh(p) Decoder: search around received sequence for codeword with ≈ np differences

decoding error probability

1. For t errors: |t/n-p|> Є → 0 for n → ∞

(codewords random)

! " # < = " = " # $ >

n and ) p ( h 1 n k R for 2 2 2 2 ) 1 2 ( ) 1 ( P

channel capacity: the BSC

1-p p 1 1 1-p X Y I(X;Y) = H(Y) – H(Y|X) the maximum of H(Y) = 1

H(Y|X) = h(p)

Conclusion: the capacity for the BSC CBSC = 1- h(p)

channel capacity: the BSC

0.5 1.0 1.0

Bit error p Channel capacity

Explain the behaviour!

channel capacity: the Z-channel

Application in optical communications 1 0 (light on) 1 (light off)

X Y H(Y) = h(P0 +p(1- P0 ) ) H(Y|X) = (1 - P0 ) h(p) For capacity, maximize I(X;Y) over P0 P(X=0) = P0

channel capacity: the erasure channel

Application: cdma detection 1 E 1

X Y I(X;Y) = H(Y)– H(Y|X) H(Y) = h(P0 ) H(Y|X) = e h(P0) Thus Cerasure = 1 – e

P(X=0) = P0

Erasure with errors: calculate the capacity!

1 E 1 p p e e 1-p-e 1-p-e example

Consider the following example

For P(0) = P(2) = p, P(1) = 1-2p H(Y) = h(1/3 – 2p/3) + (2/3 + 2p/3); H(Y|X) = (1-2p)log23 Q: maximize H(Y) – H(Y|X) as a function of p Q: is this the capacity? hint use the following: log2x = lnx / ln 2; d lnx / dx = 1/x

channel models: general diagram

x1 x2 xn y1 y2 ym : : : : : : P1|1 P2|1 P1|2 P2|2 Pm|n Input alphabet X = {x1, x2, …, xn} Output alphabet Y = {y1, y2, …, ym} Pj|i = PY|X(yj|xi) In general: calculating capacity needs more theory

The statistical behavior of the channel is completely defined by the channel transition probabilities Pj|i = PY|X(yj|xi)

* clue:

I(X;Y) is convex ∩ in the input probabilities i.e. finding a maximum is simple

Channel capacity: converse

For R > C the decoding error probability > 0 k/n C Pe

Converse: For a discrete memory less channel

# # # # Xi Yi m Xn Yn m‘

Let Pe = probability that m‘ ≠ m

converse R := k/n

Pe ≥ 1 – C/R - 1/nR Hence: for large n, and R > C, the probability of error Pe > 0 k = H(M) = I(M;Yn)+H(M|Yn) Xn is a function of M Fano ≤ I(Xn;Yn) + 1 + k Pe ≤ nC + 1 + k Pe 1 – C n/k - 1/k ≤ Pe

We used the data processing theorem

Cascading of Channels

I(X;Y)