[PPT] - Less Noisy Domination by Symmetric Channels Anuran Makur and Yury PowerPoint Presentation

SLIDE 1

Less Noisy Domination by Symmetric Channels

Anuran Makur and Yury Polyanskiy

EECS Department, Massachusetts Institute of Technology

ISIT 2017

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 1 / 28

SLIDE 2

Outline

1

Introduction Preliminaries Main Results Motivation: Strong Data Processing Inequality Main Question Less Noisy Channels in Networks

2

Equivalent Characterizations of Less Noisy Preorder

3

Conditions for Less Noisy Domination by Symmetric Channels

4

Consequences of Less Noisy Domination by Symmetric Channels

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 2 / 28

SLIDE 3

Preliminaries

probability distributions – row vectors

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 3 / 28

SLIDE 4

Preliminaries

probability distributions – row vectors channels (conditional distributions) – row stochastic matrices

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 3 / 28

SLIDE 5

Preliminaries

probability distributions – row vectors channels (conditional distributions) – row stochastic matrices

Definition (Less Noisy Preorder [K¨

rner-Marton 1977])

PY |X = W is less noisy than PZ|X = V , denoted W ln V , if and only if I(U; Y ) ≥ I(U; Z) for every joint distribution PU,X such that U → X → (Y , Z).

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 3 / 28

SLIDE 6

Preliminaries

probability distributions – row vectors channels (conditional distributions) – row stochastic matrices

Definition (Less Noisy Preorder [K¨

rner-Marton 1977])

PY |X = W is less noisy than PZ|X = V , denoted W ln V , if and only if D(PXW ||QXW ) ≥ D(PXV ||QXV ) for every pair of input distributions PX and QX.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 4 / 28

SLIDE 7

Main Results

1 Test ln using different divergence measure?

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 5 / 28

SLIDE 8

Main Results

1 Test ln using different divergence measure?

Yes, χ2-divergence

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 5 / 28

SLIDE 9

Main Results

1 Test ln using different divergence measure?

Yes, χ2-divergence

2 Sufficient conditions for ln domination by symmetric channels?

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 5 / 28

SLIDE 10

Main Results

1 Test ln using different divergence measure?

Yes, χ2-divergence

2 Sufficient conditions for ln domination by symmetric channels?

Yes degradation criterion for general channels stronger criterion for additive noise channels

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 5 / 28

SLIDE 11

Main Results

1 Test ln using different divergence measure?

Yes, χ2-divergence

2 Sufficient conditions for ln domination by symmetric channels?

Yes degradation criterion for general channels stronger criterion for additive noise channels

3 Why ln domination by symmetric channels?

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 5 / 28

SLIDE 12

Main Results

1 Test ln using different divergence measure?

Yes, χ2-divergence

2 Sufficient conditions for ln domination by symmetric channels?

Yes degradation criterion for general channels stronger criterion for additive noise channels

3 Why ln domination by symmetric channels?

just because we IT ln domination ⇒ log-Sobolev inequality secrecy capacity

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 5 / 28

SLIDE 13

Motivation: Strong Data Processing Inequality

Data Processing Inequality: For any channel V , ∀PX, QX, D(PX||QX) ≥ D(PXV ||QXV )

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 6 / 28

SLIDE 14

Motivation: Strong Data Processing Inequality

Strong Data Processing Inequality [Ahlswede-G´ acs 1976]: For any channel V , ∀PX, QX, ηKL(V ) D(PX||QX) ≥ D(PXV ||QXV ) where ηKL(V ) – contraction coefficient: ηKL(V ) sup

PX ,QX

D(PXV ||QXV ) D(PX||QX) ∈ [0, 1] .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 6 / 28

SLIDE 15

Motivation: Strong Data Processing Inequality

Strong Data Processing Inequality [Ahlswede-G´ acs 1976]: For any channel V , ∀PX, QX, ηKL(V ) D(PX||QX) ≥ D(PXV ||QXV ) where ηKL(V ) – contraction coefficient. Relation to Erasure Channels [Polyanskiy-Wu 2016]: Definition: q-ary erasure channel q-EC(1 − η) erases input w.p. 1 − η, and reproduces input w.p. η.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 6 / 28

SLIDE 16

Motivation: Strong Data Processing Inequality

Strong Data Processing Inequality [Ahlswede-G´ acs 1976]: For any channel V , ∀PX, QX, ηKL(V ) D(PX||QX) ≥ D(PXV ||QXV ) where ηKL(V ) – contraction coefficient. Relation to Erasure Channels [Polyanskiy-Wu 2016]: Definition: q-ary erasure channel q-EC(1 − η) erases input w.p. 1 − η, and reproduces input w.p. η. Prop [Polyanskiy-Wu 2016]: q-EC(1 − η) ln V ⇔ ∀PX, QX, ηD(PX||QX) ≥ D(PXV ||QXV ) .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 6 / 28

SLIDE 17

Motivation: Strong Data Processing Inequality

Strong Data Processing Inequality [Ahlswede-G´ acs 1976]: For any channel V , ∀PX, QX, ηKL(V ) D(PX||QX) ≥ D(PXV ||QXV ) where ηKL(V ) – contraction coefficient. Relation to Erasure Channels [Polyanskiy-Wu 2016]: Definition: q-ary erasure channel q-EC(1 − η) erases input w.p. 1 − η, and reproduces input w.p. η. Prop [Polyanskiy-Wu 2016]: q-EC(1 − η) ln V ⇔ ∀PX, QX, ηD(PX||QX) ≥ D(PXV ||QXV ) . SDPI ⇔ ln domination by erasure channel

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 6 / 28

SLIDE 18

Main Question

Given channel V , find q-ary symmetric channel Wδ with largest δ ∈

0, q−1

q

such that Wδ ln V ?

1 1

1 − 1 − 1 −

− 1 − 1

/( − 1) /( − 1) /( − 1) /( − 1) /( − 1) /( − 1)

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 7 / 28

SLIDE 19

Main Question

Given channel V , find q-ary symmetric channel Wδ with largest δ ∈

0, q−1

q

such that Wδ ln V ?

Definition (q-ary Symmetric Channel)

Channel matrix: Wδ       1 − δ

δ q−1

· · ·

δ q−1 δ q−1

1 − δ · · ·

δ q−1

. . . . . . ... . . .

δ q−1 δ q−1

· · · 1 − δ       where δ ∈ [0, 1] – total crossover probability.

1 1

1 − 1 − 1 −

− 1 − 1

/( − 1) /( − 1) /( − 1) /( − 1) /( − 1) /( − 1)

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 7 / 28

SLIDE 20

Main Question

Given channel V , find q-ary symmetric channel Wδ with largest δ ∈

0, q−1

q

such that Wδ ln V ?

Definition (q-ary Symmetric Channel)

Channel matrix: Wδ       1 − δ

δ q−1

· · ·

δ q−1 δ q−1

1 − δ · · ·

δ q−1

. . . . . . ... . . .

δ q−1 δ q−1

· · · 1 − δ       where δ ∈ [0, 1] – total crossover probability.

1 1

1 − 1 − 1 −

− 1 − 1

/( − 1) /( − 1) /( − 1) /( − 1) /( − 1) /( − 1)

For every channel V , W0 ln V and V ln W(q−1)/q.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 7 / 28

SLIDE 21

Main Question

Given channel V , find q-ary symmetric channel Wδ with largest δ ∈

0, q−1

q

such that Wδ ln V ?

Definition (q-ary Symmetric Channel)

Channel matrix: Wδ       1 − δ

δ q−1

· · ·

δ q−1 δ q−1

1 − δ · · ·

δ q−1

. . . . . . ... . . .

δ q−1 δ q−1

· · · 1 − δ       where δ ∈ [0, 1] – total crossover probability.

1 1

1 − 1 − 1 −

− 1 − 1

/( − 1) /( − 1) /( − 1) /( − 1) /( − 1) /( − 1)

For every channel V , W0 ln V and V ln W(q−1)/q. ∀ǫ, δ ∈ (0, 1), Wδ ln q-EC(ǫ).

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 7 / 28

SLIDE 22

Less Noisy Channels in Networks

Consider general Bayesian network:

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 8 / 28

SLIDE 23

Less Noisy Channels in Networks

Consider general Bayesian network:

Conjecture:

Replace PZ5|Z2 with less noisy channel ⇒ PY |X becomes less noisy.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 8 / 28

SLIDE 24

Less Noisy Channels in Networks

Consider general Bayesian network:

Conjecture:

Replace PZ5|Z2 with less noisy channel ⇒ PY |X becomes less noisy. Motivation: Results of [Polyanskiy-Wu 2016] on SDPIs in networks.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 8 / 28

SLIDE 25

Less Noisy Channels in Networks

Consider Bayesian network with binary r.v.s

|
|,

NOR

BSC()
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 9 / 28

SLIDE 26

Less Noisy Channels in Networks

Consider Bayesian network with binary r.v.s

|
|,

where we replace PZ|X2 with less noisy channel. Can this decrease I(X1, X2; Y )? NOR

BSC()
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 9 / 28

SLIDE 27

Less Noisy Channels in Networks

Consider Bayesian network with binary r.v.s

|
|,

where we replace PZ|X2 with less noisy channel. Can this decrease I(X1, X2; Y )? YES Example: Let X1 ∼ Ber 1

2

and X2 = 1 a.s., and let I(δ) = I(X1, X2; Y ).

NOR

BSC()
For δ > 0, BSC(0) ln BSC(δ), but h(δ/2) − h(δ)/2 = I(δ) > I(0) = 0.
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 9 / 28

SLIDE 28

Outline

1

Introduction

2

Equivalent Characterizations of Less Noisy Preorder χ2-Divergence Characterization of Less Noisy L¨

wner and Spectral Characterizations of Less Noisy

3

Conditions for Less Noisy Domination by Symmetric Channels

4

Consequences of Less Noisy Domination by Symmetric Channels

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 10 / 28

SLIDE 29

χ2-Divergence Characterization of Less Noisy

Theorem 1 (χ2-Divergence Characterization of ln)

Given channels W and V , W ln V if and only if ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) . Recall χ2-divergence between PX and QX: χ2 (PX||QX)

x∈X

(PX(x) − QX(x))2 QX(x) .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 11 / 28

SLIDE 30

χ2-Divergence Characterization of Less Noisy

Theorem 1 (χ2-Divergence Characterization of ln)

Given channels W and V , W ln V if and only if ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) . Proof: (⇒) Fix any PX, QX. Recall local approximation: lim

λ→0+

2 λ2 D (λPX + (1 − λ)QX||QX) = χ2 (PX||QX) .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 11 / 28

SLIDE 31

χ2-Divergence Characterization of Less Noisy

Theorem 1 (χ2-Divergence Characterization of ln)

Given channels W and V , W ln V if and only if ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) . Proof: (⇒) Fix any PX, QX. Recall local approximation: lim

λ→0+

2 λ2 D (λPX + (1 − λ)QX||QX) = χ2 (PX||QX) . W ln V implies D (λPXW + (1 − λ)QXW ||QXW ) ≥ D(λPXV + (1 − λ)QXV ||QXV )

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 11 / 28

SLIDE 32

χ2-Divergence Characterization of Less Noisy

Theorem 1 (χ2-Divergence Characterization of ln)

Given channels W and V , W ln V if and only if ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) . Proof: (⇒) Fix any PX, QX. Recall local approximation: lim

λ→0+

2 λ2 D (λPX + (1 − λ)QX||QX) = χ2 (PX||QX) . W ln V implies D (λPXW + (1 − λ)QXW ||QXW ) ≥ D(λPXV + (1 − λ)QXV ||QXV ) χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) after taking limits.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 11 / 28

SLIDE 33

χ2-Divergence Characterization of Less Noisy

Theorem 1 (χ2-Divergence Characterization of ln)

Given channels W and V , W ln V if and only if ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) . Proof: (⇐) Fix any PX, QX. Recall integral representation: D (PX||QX) = ∞ χ2 PX||Qt

X

dt

where Qt

X = t 1+t PX + 1 t+1QX for t ∈ [0, ∞) [Choi-Ruskai-Seneta 1994].

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 12 / 28

SLIDE 34

χ2-Divergence Characterization of Less Noisy

Theorem 1 (χ2-Divergence Characterization of ln)

Given channels W and V , W ln V if and only if ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) . Proof: (⇐) Fix any PX, QX. Recall integral representation: D (PX||QX) = ∞ χ2 PX||Qt

X

dt

where Qt

X = t 1+t PX + 1 t+1QX for t ∈ [0, ∞) [Choi-Ruskai-Seneta 1994].

χ2 PXW ||Qt

XW

≥ χ2

PXV ||Qt

XV

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 12 / 28

SLIDE 35

χ2-Divergence Characterization of Less Noisy

Theorem 1 (χ2-Divergence Characterization of ln)

Given channels W and V , W ln V if and only if ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) . Proof: (⇐) Fix any PX, QX. Recall integral representation: D (PX||QX) = ∞ χ2 PX||Qt

X

dt

where Qt

X = t 1+t PX + 1 t+1QX for t ∈ [0, ∞) [Choi-Ruskai-Seneta 1994].

χ2 PXW ||Qt

XW

≥ χ2

PXV ||Qt

XV

∞

χ2 PXW ||Qt

XW

dt ≥

∞ χ2 PXV ||Qt

XV

dt
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 12 / 28

SLIDE 36

χ2-Divergence Characterization of Less Noisy

Theorem 1 (χ2-Divergence Characterization of ln)

Given channels W and V , W ln V if and only if ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) . Proof: (⇐) Fix any PX, QX. Recall integral representation: D (PX||QX) = ∞ χ2 PX||Qt

X

dt

where Qt

X = t 1+t PX + 1 t+1QX for t ∈ [0, ∞) [Choi-Ruskai-Seneta 1994].

χ2 PXW ||Qt

XW

≥ χ2

PXV ||Qt

XV

∞

χ2 PXW ||Qt

XW

dt ≥

∞ χ2 PXV ||Qt

XV

dt

D (PXW ||QXW ) ≥ D (PXV ||QXV )

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 12 / 28

SLIDE 37

L¨

wner and Spectral Characterizations of Less Noisy

Theorem 1 (Equivalent Characterizations of ln)

Given channels W and V , W ln V ⇔ ∀PX, QX, χ2 (PXW ||QXW ) ≥ χ2 (PXV ||QXV ) ⇔ ∀PX, W diag(PXW )−1 W T PSD V diag(PXV )−1 V T ⇔ ∀PX, ρ

W diag(PXW )−1 W T†V diag(PXV )−1 V T

= 1 where PSD – L¨

wner (PSD) partial order,

A† – Moore-Penrose pseudoinverse of A, and ρ(·) – spectral radius.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 13 / 28

SLIDE 38

Outline

1

Introduction

2

Equivalent Characterizations of Less Noisy Preorder

3

Conditions for Less Noisy Domination by Symmetric Channels General Sufficient Condition via Degradation Refinements for Additive Noise Channels Proof Sketch of Additive Noise Channel Criterion

4

Consequences of Less Noisy Domination by Symmetric Channels

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 14 / 28

SLIDE 39

Condition for Degradation by Symmetric Channels

Given channel V , find q-ary symmetric channel Wδ with largest δ ∈

0, q−1

q

such that Wδ ln V ?
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 15 / 28

SLIDE 40

Condition for Degradation by Symmetric Channels

Given channel V , find q-ary symmetric channel Wδ with largest δ ∈

0, q−1

q

such that Wδ ln V ?

Definition (Degradation) [Bergmans 1973]: V is degraded version

f W , denoted W deg V , if V = WA for some channel A.
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 15 / 28

SLIDE 41

Condition for Degradation by Symmetric Channels

Given channel V , find q-ary symmetric channel Wδ with largest δ ∈

0, q−1

q

such that Wδ ln V ?

Definition (Degradation) [Bergmans 1973]: V is degraded version

f W , denoted W deg V , if V = WA for some channel A.

Prop: W deg V ⇒ W ln V .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 15 / 28

SLIDE 42

Condition for Degradation by Symmetric Channels

Given channel V , find q-ary symmetric channel Wδ with largest δ ∈

0, q−1

q

such that Wδ ln V ?

Definition (Degradation) [Bergmans 1973]: V is degraded version

f W , denoted W deg V , if V = WA for some channel A.

Prop: W deg V ⇒ W ln V .

Theorem 2 (Degradation by Symmetric Channels)

For channel V with common input and output alphabet, and minimum probability ν = min {[V ]i,j : 1 ≤ i, j ≤ q}, 0 ≤ δ ≤ ν 1 − (q − 1)ν +

ν q−1

⇒ Wδ deg V .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 15 / 28

SLIDE 43

Condition for Degradation by Symmetric Channels

Theorem 2 (Degradation by Symmetric Channels)

For channel V with common input and output alphabet, and minimum probability ν = min {[V ]i,j : 1 ≤ i, j ≤ q}, 0 ≤ δ ≤ ν 1 − (q − 1)ν +

ν q−1

⇒ Wδ deg V . Remark: Condition is tight when no further information about V known. For example, suppose V =      ν 1 − (q − 1)ν ν · · · ν 1 − (q − 1)ν ν ν · · · ν . . . . . . . . . ... . . . 1 − (q − 1)ν ν ν · · · ν      . Then, 0 ≤ δ ≤ ν/

1 − (q − 1)ν +

ν q−1

⇔ Wδ deg V .
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 16 / 28

SLIDE 44

Additive Noise Channels

Fix Abelian group (X, ⊕) with order q as alphabet.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 17 / 28

SLIDE 45

Additive Noise Channels

Fix Abelian group (X, ⊕) with order q as alphabet. Additive noise channel: Y = X ⊕ Z, X ⊥ ⊥ Z where X, Y , Z ∈ X are input, output, and noise r.v.s.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 17 / 28

SLIDE 46

Additive Noise Channels

Fix Abelian group (X, ⊕) with order q as alphabet. Additive noise channel: Y = X ⊕ Z, X ⊥ ⊥ Z where X, Y , Z ∈ X are input, output, and noise r.v.s. Channel probabilities given by noise pmf PZ: ∀x, y ∈ X, PY |X(y|x) = PZ(−x ⊕ y) .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 17 / 28

SLIDE 47

Additive Noise Channels

Fix Abelian group (X, ⊕) with order q as alphabet. Additive noise channel: Y = X ⊕ Z, X ⊥ ⊥ Z where X, Y , Z ∈ X are input, output, and noise r.v.s. Channel probabilities given by noise pmf PZ: ∀x, y ∈ X, PY |X(y|x) = PZ(−x ⊕ y) . PY is convolution of PX and PZ: ∀y ∈ X, PY (y) = (PX ⋆ PZ)(y)

x∈X

PX(x)PZ(−x ⊕ y) .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 17 / 28

SLIDE 48

Additive Noise Channels

Fix Abelian group (X, ⊕) with order q as alphabet. Additive noise channel: Y = X ⊕ Z, X ⊥ ⊥ Z where X, Y , Z ∈ X are input, output, and noise r.v.s. Channel probabilities given by noise pmf PZ: ∀x, y ∈ X, PY |X(y|x) = PZ(−x ⊕ y) . PY is convolution of PX and PZ: ∀y ∈ X, PY (y) = (PX ⋆ PZ)(y)

x∈X

PX(x)PZ(−x ⊕ y) . q-ary symmetric channel: PZ =

1 − δ,

δ q−1, . . . , δ q−1

for δ ∈ [0, 1]

(· ⋆ PZ) = Wδ

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 17 / 28

SLIDE 49

More Noisy and Degradation Regions

Fix q-ary symmetric channel Wδ with δ ∈ [0, 1].

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 18 / 28

SLIDE 50

More Noisy and Degradation Regions

Fix q-ary symmetric channel Wδ with δ ∈ [0, 1]. More noisy region of Wδ is more-noisy (Wδ) {PZ : Wδ ln (· ⋆ PZ)} .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 18 / 28

SLIDE 51

More Noisy and Degradation Regions

Fix q-ary symmetric channel Wδ with δ ∈ [0, 1]. More noisy region of Wδ is more-noisy (Wδ) {PZ : Wδ ln (· ⋆ PZ)} . Degradation region of Wδ is degrade (Wδ) {PZ : Wδ deg (· ⋆ PZ)} .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 18 / 28

SLIDE 52

Domination Structure of Additive Noise Channels

Theorem 3 (More Noisy and Degradation Regions)

For Wδ with δ ∈

0, q−1

q

and q ≥ 2,

degrade (Wδ) = co (rows of Wδ) ⊆ co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ) ⊆ {PZ : PZ − uℓ2 ≤ wδ − uℓ2} where co (·) – convex hull, γ = (1 − δ)/

1 − δ +

δ (q−1)2

, u – uniform

pmf, and wδ – first row of Wδ.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 19 / 28

SLIDE 53

Domination Structure of Additive Noise Channels

Theorem 3 (More Noisy and Degradation Regions)

For Wδ with δ ∈

0, q−1

q

and q ≥ 2,

degrade (Wδ) = co (rows of Wδ) ⊆ co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ) ⊆ {PZ : PZ − uℓ2 ≤ wδ − uℓ2} where co (·) – convex hull, γ = (1 − δ)/

1 − δ +

δ (q−1)2

, u – uniform

pmf, and wδ – first row of Wδ. Furthermore, more-noisy (Wδ) is closed, convex, and invariant under permutations of (X, ⊕).

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 19 / 28

SLIDE 54

Domination Structure of Additive Noise Channels

Illustration of the q = 3 case:

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 20 / 28

SLIDE 55

Domination Structure of Additive Noise Channels

Illustration of the q = 3 case:

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 20 / 28

SLIDE 56

Domination Structure of Additive Noise Channels

Illustration of the q = 3 case:

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 20 / 28

SLIDE 57

Domination Structure of Additive Noise Channels

Illustration of the q = 3 case:

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 20 / 28

SLIDE 58

Domination Structure of Additive Noise Channels

Illustration of the q = 3 case:

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 20 / 28

SLIDE 59

Domination Structure of Additive Noise Channels

Theorem 3 (More Noisy and Degradation Regions)

For Wδ with δ ∈

0, q−1

q

and q ≥ 2,

degrade (Wδ) = co (rows of Wδ) ⊆ co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ) ⊆ {PZ : PZ − uℓ2 ≤ wδ − uℓ2} where co (·) – convex hull, γ = (1 − δ)/

1 − δ +

δ (q−1)2

, u – uniform

pmf, and wδ – first row of Wδ. Furthermore, more-noisy (Wδ) is closed, convex, and invariant under permutations of (X, ⊕).

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 21 / 28

SLIDE 60

Proof Sketch: co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ)

more-noisy (Wδ) is convex, invariant under permutations of (X, ⊕) ⇒ suffices to prove Wδ ln Wγ.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 22 / 28

SLIDE 61

Proof Sketch: co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ)

more-noisy (Wδ) is convex, invariant under permutations of (X, ⊕) ⇒ suffices to prove Wδ ln Wγ. By Theorem 1, ∀PX, Wδ diag(PXWδ)−1 W T

δ PSD Wγ diag(PXWγ)−1 W T γ

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 22 / 28

SLIDE 62

Proof Sketch: co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ)

more-noisy (Wδ) is convex, invariant under permutations of (X, ⊕) ⇒ suffices to prove Wδ ln Wγ. By Theorem 1, ∀PX, Wδ diag(PXWδ)−1 W T

δ PSD Wγ diag(PXWγ)−1 W T γ

⇔ 1 ≥ Aop where ·op – operator norm, and A is symmetric PSD: A diag(wγ)− 1

2 WγW −1

δ

diag(wδ)W −1

δ

Wγ diag(wγ)− 1

2

with wδ – first row of Wδ, and wγ – first row of Wγ.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 22 / 28

SLIDE 63

Proof Sketch: co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ)

more-noisy (Wδ) is convex, invariant under permutations of (X, ⊕) ⇒ suffices to prove Wδ ln Wγ. By Theorem 1, ∀PX, Wδ diag(PXWδ)−1 W T

δ PSD Wγ diag(PXWγ)−1 W T γ

⇔ 1 ≥ Aop where ·op – operator norm, and A is symmetric PSD: A diag(wγ)− 1

2 WγW −1

δ

diag(wδ)W −1

δ

Wγ diag(wγ)− 1

2

with wδ – first row of Wδ, and wγ – first row of Wγ. A has left eigenvector √wγ > 0 with eigenvalue 1: √wγA = √wγ .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 22 / 28

SLIDE 64

Proof Sketch: co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ)

A ≥ 0 (entry-wise) ⇒ largest eigenvalue of A is 1 by Perron-Frobenius theorem, because √wγ > 0.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 23 / 28

SLIDE 65

Proof Sketch: co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ)

A ≥ 0 (entry-wise) ⇒ largest eigenvalue of A is 1 by Perron-Frobenius theorem, because √wγ > 0. Since A – symmetric PSD, A ≥ 0 ⇒ Aop ≤ 1. ⇒ Suffices to prove A ≥ 0.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 23 / 28

SLIDE 66

Proof Sketch: co (rows of Wδ and Wγ) ⊆ more-noisy (Wδ)

A ≥ 0 (entry-wise) ⇒ largest eigenvalue of A is 1 by Perron-Frobenius theorem, because √wγ > 0. Since A – symmetric PSD, A ≥ 0 ⇒ Aop ≤ 1. ⇒ Suffices to prove A ≥ 0. Verify that: min

i,j [A]i,j ≥ 0

⇔ δ ≤ γ ≤ 1 − δ 1 − δ +

δ (q−1)2

.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 23 / 28

SLIDE 67

Outline

1

Introduction

2

Equivalent Characterizations of Less Noisy Preorder

3

Conditions for Less Noisy Domination by Symmetric Channels

4

Consequences of Less Noisy Domination by Symmetric Channels Log-Sobolev Inequalities via Comparison of Dirichlet Forms Interpretation via Wyner’s Wiretap Channel

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 24 / 28

SLIDE 68

Log-Sobolev Inequalities

Consider irreducible Markov chain V with uniform stationary pmf u

n state space of size q.
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 25 / 28

SLIDE 69

Log-Sobolev Inequalities

Consider irreducible Markov chain V with uniform stationary pmf u

n state space of size q.

Dirichlet form EV : Rq × Rq → R+ EV (f , f ) 1 q f T

I − V + V T

2

f
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 25 / 28

SLIDE 70

Log-Sobolev Inequalities

Consider irreducible Markov chain V with uniform stationary pmf u

n state space of size q.

Dirichlet form EV : Rq × Rq → R+ EV (f , f ) 1 q f T

I − V + V T

2

f

Log-Sobolev inequality with constant α ∈ R+: For every f ∈ Rq such that f Tf = q, D

f 2u || u
= 1

q

i=1

f 2

i log

f 2

i

≤ 1

α EV (f , f ) .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 25 / 28

SLIDE 71

Log-Sobolev Inequalities

Consider irreducible Markov chain V with uniform stationary pmf u

n state space of size q.

Dirichlet form EV : Rq × Rq → R+ EV (f , f ) 1 q f T

I − V + V T

2

f

Log-Sobolev inequality with constant α ∈ R+: For every f ∈ Rq such that f Tf = q, D

f 2u || u
= 1

q

i=1

f 2

i log

f 2

i

≤ 1

α EV (f , f ) . Log-Sobolev constant – largest α satisfying log-Sobolev inequality.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 25 / 28

SLIDE 72

Comparison of Dirichlet Forms

Standard Dirichlet form: Estd (f , f ) VARu(f ) =

q

i=1

1 q f 2

i −

q

i=1

1 q fi

2
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 26 / 28

SLIDE 73

Comparison of Dirichlet Forms

For standard Dirichlet form, Estd (f , f ) VARu(f ), log-Sobolev constant known [Diaconis-Saloff-Coste 1996]: D

f 2u || u
≤ q log(q − 1)

(q − 2) Estd (f , f ) for all f ∈ Rq with f Tf = q.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 26 / 28

SLIDE 74

Comparison of Dirichlet Forms

For standard Dirichlet form, Estd (f , f ) VARu(f ), log-Sobolev constant known [Diaconis-Saloff-Coste 1996]: D

f 2u || u
≤ q log(q − 1)

(q − 2) Estd (f , f ) for all f ∈ Rq with f Tf = q.

Theorem 4 (Domination of Dirichlet Forms)

For channels Wδ and V with δ ∈

0, q−1

q

and stationary pmf u,

Wδ ln V ⇒ EV ≥ qδ q − 1 Estd pointwise .

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 26 / 28

SLIDE 75

Comparison of Dirichlet Forms

For standard Dirichlet form, Estd (f , f ) VARu(f ), log-Sobolev constant known [Diaconis-Saloff-Coste 1996]: D

f 2u || u
≤ q log(q − 1)

(q − 2) Estd (f , f ) for all f ∈ Rq with f Tf = q.

Theorem 4 (Domination of Dirichlet Forms)

For channels Wδ and V with δ ∈

0, q−1

q

and stationary pmf u,

Wδ ln V ⇒ EV ≥ qδ q − 1 Estd pointwise . Wδ ln V ⇒ log-Sobolev inequality for V , D(f 2u || u) ≤ (q − 1) log(q − 1) δ (q − 2) EV (f , f ) for every f ∈ Rq satisfying f Tf = q.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 26 / 28

SLIDE 76

Interpretation via Wyner’s Wiretap Channel

Decoder Eavesdropper Encoder

V – main channel, Wδ – eavesdropper channel
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 27 / 28

SLIDE 77

Interpretation via Wyner’s Wiretap Channel

Decoder Eavesdropper Encoder

V – main channel, Wδ – eavesdropper channel

Secrecy capacity – maximum rate to legal receiver such that P(M = ˆ M) → 0 and 1

nI(M; Z n) → 0

CS = max

PU,X

I(U; Y ) − I(U; Z) [Csisz´ ar-K¨

rner 1978]
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 27 / 28

SLIDE 78

Interpretation via Wyner’s Wiretap Channel

Decoder Eavesdropper Encoder

V – main channel, Wδ – eavesdropper channel

Secrecy capacity – maximum rate to legal receiver such that P(M = ˆ M) → 0 and 1

nI(M; Z n) → 0

CS = max

PU,X

I(U; Y ) − I(U; Z) [Csisz´ ar-K¨

rner 1978]

Prop [Csisz´ ar-K¨

rner 1978]: CS = 0 ⇔ Wδ ln V .
A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 27 / 28

SLIDE 79

Interpretation via Wyner’s Wiretap Channel

Decoder Eavesdropper Encoder

V – main channel, Wδ – eavesdropper channel

Secrecy capacity – maximum rate to legal receiver such that P(M = ˆ M) → 0 and 1

nI(M; Z n) → 0

CS = max

PU,X

I(U; Y ) − I(U; Z) [Csisz´ ar-K¨

rner 1978]

Prop [Csisz´ ar-K¨

rner 1978]: CS = 0 ⇔ Wδ ln V .

Finding maximally noisy Wδ ln V establishes minimal noise on PZ|X so that secret communication feasible.

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 27 / 28

SLIDE 80

Thank You!

A. Makur & Y. Polyanskiy (MIT)

Symmetric Channel Domination 29 June 2017 28 / 28