Channel Upgradation for Non-Binary Input Alphabets and MACs Uzi - - PowerPoint PPT Presentation

Channel Upgradation for Non-Binary Input Alphabets and MACs

Uzi Pereg and Ido Tal Technion, EE

() June 25, 2014 1 / 16

Problem

W : X → Y |Y| ≫ S

• upgrade(W , S)

W ′ : X → Y′ |Y′| ≤ S Goal: make I(W ′) − I(W ) as small as possible, where I(·) is symmetric capacity. X need not be binary W need not be symmetric W can be a t-user MAC W : X t → Y

Concurrent work

• A. Ghayoori and T. A. Gulliver, Upgraded Approximation of Non-Binary

Alphabets for Polar Code Construction, on arXiv

() June 25, 2014 2 / 16

Upgradation Vs. Degradation

• riginal

channel W another channel P

• degraded channel Q

upgraded channel Q′ another channel P

• riginal channel W

W (y|u) is the probability of receiving y ∈ Y, given that u ∈ X was sent. The input is assumed to have symmetric distribution over X.

() June 25, 2014 3 / 16

Motivation: polar codes

Assure Eve’s channels are very bad (wiretap setting) Upper bound on rate (error correcting code)

Theorem

Let W : X → Y be a given channel, with |X| = q. Let µ ≥ max{5, q(q − 1)} be a given fidelity parameter. We can construct W ′ : X → Y′ such that W ′ is upgraded with respect to W I(W ′) − I(W ) ≤ q−1

µ (2 + q · ln q)

|Y′| ≤ S(q, µ) (2µ)q + q

• impractical for big q (i.e. large input alphabet.)

() June 25, 2014 4 / 16

Key ideas

Use bins:

“close” output symbols of W go into the same bin “merge” all symbols in bin to one symbol

Use “boost” symbols (sparingly):

New output alphabet will have q = |X| special symbols, {κu}u∈X Symbol κu received on W ′ only if u sent Small probability of receiving κu

() June 25, 2014 5 / 16

W

y1 y2 . . . u=0 W (y1|0) W (y2|0) u=1 W (y1|1) W (y2|1) u=2 W (y1|2) W (y2|2)

Q

y1 y2 . . . κ0 κ1 κ2 u=0 α0(y1) · W (y1|0) α0(y2) · W (y2|0) ǫ0 u=1 α1(y1) · W (y1|1) α1(y2) · W (y2|1) ǫ1 u=2 α2(y1) · W (y1|2) α2(y2) · W (y2|2) ǫ2 ǫu =

• y∈Y

(1 − αu(y))W (y|u) , 0 ≤ αu(y) ≤ 1 , αu(y) ≈ 1

() June 25, 2014 6 / 16

Example

W

Consider a bin {y1, y2, y3, y4}, where y1 y2 y3 y4 u = 0 0.2625 0.0042 0.0500 2.1 · 10−12 u = 1 0.75 0.0118 0.1475 5.8 · 10−12 u = 2 0.2375 0.004 0.0525 2.1 · 10−12

Q

We choose αu(y) such that the columns are multiples of one another: y1 y2 y3 y4 κ0 κ1 κ2 u = 0 0.2459 0.0038 0.0483 1.901 · 10−12 0.0185 u = 1 0.75 0.0118 0.1475 5.8 · 10−12 u = 2 0.2336 0.0036 0.0459 1.806 · 10−12 0.0107 Q(y1|u) = 63.55· Q(y2|u) = 5.084· Q(y3|u) = 0.1293 · 1012· Q(y4|u)

() June 25, 2014 7 / 16

Functions η(x) and R(x) for µ = 17.2

x η(x)

s l

• p

e = + 1 s l

• p

e = − 1

1/e 1/e2 1 R(x) 1/µ

1

2/µ

2

3/µ

3

4/µ

4

5/µ

5

6 7 8 9 10 11 12 13 14 15 16 17 18 19

20

6/µ

η(x) = −x · ln x

() June 25, 2014 8 / 16

For y ∈ Y and u ∈ X, define the APP ϕW (u|y) = W (y|u)

• v∈X W (y|v) .

Two symbols y1, y2 ∈ Y are in the same bin, if i(u) = R(ϕW (u|y1)) = R(ϕW (u|y2)) , for all u ∈ X .

In our example earlier

y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12 ϕW (0|y) 0.21 0.21 0.20 0.21 i(0) = 6 ϕW (1|y) 0.60 0.59 0.59 0.58 i(1) = 13 ϕW (2|y) 0.19 0.20 0.21 0.21 i(2) = 6

() June 25, 2014 9 / 16

New APP

Denote all the output letters in bin z as B(z) Define the leading input and output of the bin as (u∗, y∗) = arg max

u∈X , y∈B(z)

ϕW (u|y) Define a new APP measure by ψ(u|z) = min

y∈B(z) ϕW (u|y)

for all u = u∗ and ψ(u∗|z) = 1 −

• u=u∗

ψ(u|z) αu(y) ψ(u|z) ϕW (u|y) · ϕW (u∗|y) ψ(u∗|z)

() June 25, 2014 10 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12

() June 25, 2014 11 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ϕW (1|y) 0.60 0.59 0.59 0.58 ϕW (2|y) 0.19 0.20 0.21 0.21

() June 25, 2014 11 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ϕW (1|y) 0.60 0.59 0.59 0.58 ϕW (2|y) 0.19 0.20 0.21 0.21 The leading input is u∗ = arg maxu∈{0,1,2}

• maxy∈B(z) ϕW (u|y)
• =

() June 25, 2014 11 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ϕW (1|y) 0.60 0.59 0.59 0.58 ϕW (2|y) 0.19 0.20 0.21 0.21 The leading input is u∗ = arg maxu∈{0,1,2}

• maxy∈B(z) ϕW (u|y)
• =1.

() June 25, 2014 11 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ψ(0|z) = 0.20 ϕW (1|y) 0.60 0.59 0.59 0.58 ϕW (2|y) 0.19 0.20 0.21 0.21 ψ(2|z) = 0.19 The leading input is u∗ = arg maxu∈{0,1,2}

• maxy∈B(z) ϕW (u|y)
• =1.

() June 25, 2014 11 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ψ(0|z) = 0.20 ϕW (1|y) 0.60 0.59 0.59 0.58 ψ(1|z) = 1 − (0.20 + 0.19) = 0.61 ϕW (2|y) 0.19 0.20 0.21 0.21 ψ(2|z) = 0.19 The leading input is u∗ = arg maxu∈{0,1,2}

• maxy∈B(z) ϕW (u|y)
• =1.

() June 25, 2014 11 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ψ(0|z) = 0.20 ϕW (1|y) 0.60 0.59 0.59 0.58 ψ(1|z) = 1 − (0.20 + 0.19) = 0.61 ϕW (2|y) 0.19 0.20 0.21 0.21 ψ(2|z) = 0.19 The leading input is u∗ = arg maxu∈{0,1,2}

• maxy∈B(z) ϕW (u|y)
• =1.

E.g. α0(y1) = 0.20

0.21 · 0.60 0.61 ≈ 0.936.

Thus, we subtract (≈ 0.064 · W (y1|0)) and pass over to our boost symbol κ0.

() June 25, 2014 11 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.004 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ψ(0|z) = 0.20 ϕW (1|y) 0.60 0.59 0.59 0.58 ψ(1|z) = 1 − (0.20 + 0.19) = 0.61 ϕW (2|y) 0.19 0.20 0.21 0.21 ψ(2|z) = 0.19 y1 y2 y3 y4 α0(y)

0.20 0.21 · 0.60 0.61 0.20 0.21 · 0.59 0.61 0.20 0.20 · 0.59 0.61 0.20 0.21 · 0.58 0.61

α1(y) 1 1 1 1 α2(y)

0.19 0.19 · 0.60 0.61 0.19 0.20 · 0.59 0.61 0.19 0.21 · 0.59 0.61 0.19 0.21 · 0.58 0.61

() June 25, 2014 12 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.0040 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ψ(0|z) = 0.20 ϕW (1|y) 0.60 0.59 0.59 0.58 ψ(1|z) = 1 − (0.20 + 0.19) = 0.61 ϕW (2|y) 0.19 0.20 0.21 0.21 ψ(2|z) = 0.19 y1 y2 y3 y4 Q(y|0) 0.2459 0.0038 0.0483 1.901 · 10−12 Q(y|1) 0.75 0.0118 0.1475 5.8 · 10−12 Q(y|2) 0.2336 0.0036 0.0459 1.806 · 10−12

() June 25, 2014 13 / 16

Example (Cont.)

Consider a bin B(z) = {y1, y2, y3, y4}, where y1 y2 y3 y4 W (y|0) 0.2625 0.0042 0.0500 2.1 · 10−12 W (y|1) 0.75 0.0118 0.1475 5.8 · 10−12 W (y|2) 0.2375 0.0040 0.0525 2.1 · 10−12 y1 y2 y3 y4 ϕW (0|y) 0.21 0.21 0.20 0.21 ψ(0|z) = 0.20 ϕW (1|y) 0.60 0.59 0.59 0.58 ψ(1|z) = 1 − (0.20 + 0.19) = 0.61 ϕW (2|y) 0.19 0.20 0.21 0.21 ψ(2|z) = 0.19 y1 y2 y3 y4 Q(y|0) 0.2459 0.0038 0.0483 1.901 · 10−12 ε0 ≈ 0.0185 Q(y|1) 0.75 0.0118 0.1475 5.8 · 10−12 ε1 = 0 Q(y|2) 0.2336 0.0036 0.0459 1.806 · 10−12 ε2 ≈ 0.0107

() June 25, 2014 13 / 16

Proof Outline (very broadly)

Note that

Mutual Information of U and Y

I(W ) = I(U; Y ) = ln q −

• y∈Y

pW (y) · H(U|Y = y)) Now, define

Surrogate for Mutual Information

J(U; Z) = ln q −

• z∈Z

p(z) · Hψ(U|Z = z)) = ln q −

• z∈Z

p(z)

• u∈X

η [ψ(u|z)] , where p(z)

y∈B(z) pW (y).

() June 25, 2014 14 / 16

Next, prove two bounds and sum up First, some interesting properties regarding our quantization, the APP measure ψ(·|z) and the leading input imply that |η (ϕW (u|y)) − η (ψ(u|z)) | ≤

• 1/µ

if u = u∗ ,

q−1 µ

if u = u∗ . ⇒ J(U; Z) − I(W ) = p(y) [η(ϕ(u|y)) − η(ψ(u|z))] ≤ 2 · q−1

µ

, Second, 1 q

• u∈X

εu ≤ q(q − 1) µ . I(W ′) − J(U; Z) = =

• z

(p(z) − pW ′(z))Hψ(U|Z = z) −

• κ

pW ′(κ)H(U|Z

′ = κ)

≤ ln q · 1 q ·

• u∈X

εu ≤ ln q · q(q − 1) µ .

() June 25, 2014 15 / 16

Thank You!

() June 25, 2014 16 / 16