INFORMATION-THEORETIC SECURITY INFORMATION-THEORETIC SECURITY - - PowerPoint PPT Presentation

Matthieu Bloch December 3, 2019

INFORMATION-THEORETIC SECURITY INFORMATION-THEORETIC SECURITY

Lecture 2 - Elements of Information Theory

1

TOOLS: TOOLS: CONCENTRATION INEQUALITIES CONCENTRATION INEQUALITIES

Lemma (Markov's inequality) Let be a non-negative real-valued random variable. Then for all Lemma (Chebyshev's inequality) Let be a real-valued random variable. Then for all Proposition (Weak law of large numbers) Let be i.i.d. real-valued random variables with finite mean and finite variance . Then

X t > 0 (X ≥ t) ≤ . P [X] E t X t > 0 (|X − [X]| ≥ t) ≤ . P E Var(X) t2 {Xi}n

i=1

μ σ2 ( − μ ≥ ϵ) ≤ ( − μ ≥ ϵ) = 0. P ∣ ∣ ∣ 1 n ∑

i=1 n

Xi ∣ ∣ ∣ σ2 nϵ2 lim

n→∞ P

∣ ∣ ∣ 1 N ∑

i=1 n

Xi ∣ ∣ ∣

2

3

4

5

TOOLS: TOOLS: MORE CONCENTRATION INEQUALITIES MORE CONCENTRATION INEQUALITIES

Proposition (Hoeffding's inequality) Let be i.i.d. real-valued zero-mean random variables such that . Then for all More on concentration inequalities later in the course when we need more precise results Lemma. Let and . If then there exists such that .

{Xi}n

i=1

∈ [ ; ] Xi ai bi ϵ > 0 ( ≥ ϵ) ≤ 2 exp(− ). P ∣ ∣ ∣ 1 n ∑

i=1 n

Xi ∣ ∣ ∣ 2n2ϵ2 ( − ∑n

i=1 bi

ai)2 ϵ > 0 f : X → R+ [f(X)] ≤ ϵ EX ∈ X x∗ f( ) ≤ 2ϵ x∗

6

7

8

CHANNEL CODING CHANNEL CODING

One shot channel coding

Coding consists of encoder and decoder Objective: transmit messages with small average probability of error Lemma (Random coding for channel reliability) For and , define For a codebook

• f independently generated codewords

, we have One shot result not tight but good enough for many problems

Enc : [1; M] → X Dec : Y → [1; M] W (C) ≜ ( ≠ W|C) = (Dec(Y ) ≠ m|W = m) Pe P W ˆ 1 M ∑

m=1 M

P ∈ Δ(X) pX γ > 0 ≜ W ∘ pY pX ≜ {(x, y) ∈ X × Y : log ≥ γ} . Aγ (y|x) WY|X (y) pY C ∼ Xi pX [ (C)] ≤ ((X, Y ) ∉ ) + M . EC Pe PpXWY|X Aγ 2−γ

9

10

11

12

CHANNEL CODING CHANNEL CODING

Reliable communication over a noisy channel

Introduce coding over blocklength , so that and

• Definition. (Achievable rate)

A rate is achievable if there exists a sequence of codes with length with Proposition (Achievability) The rate is achievable. Proposition (Converse) All achievable rates must satisfy . This provides an operational meaning to the channel capacity .

n ∈ N∗ Enc : [1; M] → X n Dec : → [1; M] Yn R n log M ≥ R ( ) = 0 lim inf

n→∞

1 n lim sup

n→∞ Pe Cn

I(X; Y ) maxpX R R ≤ I(X; Y ) maxpX C ≜ I(X; Y ) maxpX

13

14



 