INFORMATION-THEORETIC SECURITY INFORMATION-THEORETIC SECURITY - PowerPoint PPT Presentation

INFORMATION-THEORETIC SECURITY INFORMATION-THEORETIC SECURITY Lecture 4 - Elements of Information Theory Matthieu Bloch December 5, 2019 1

SOURCE CODING SOURCE CODING One-shot source coding with side information Coding consists of encoder to encode source symbol and decoder Enc : X → [1; M ] to reconstruct source symbol Dec : Y × [1; M ] → X Objective: transmit messages to reconstruct from and with small average probability of error W X W Y ^ P e ( C ) ≜ P X ( ≠ X | C ) = P (Dec(Enc( X ), Y ) ≠ X ). Lemma (Random binning for source coding) For , let γ > 0 1 B γ = { ( x , y ) ∈ X × Y : log < γ } . P X | Y ( x | y ) For a codebook of independently generated , we have C = {Φ( x )} Φ( x ) ∼ U ([1; M ]) 2 γ E C P e [ ( C )] ≤ P P XY (( X , Y ) ∉ B γ ) + . M 2

SOURCE CODING SOURCE CODING One-shot source coding with side information Coding of sequences of length so that and n ∈ N ∗ X n Y n X n Enc : → [1; M ] Dec : × [1; M ] → Definition. (Achievable rate) A rate is achievable if there exists a sequence of codes with length with R n 1 lim sup log M ≤ R lim sup n →∞ P e C n ( ) = 0 n n →∞ Proposition (Achievability) The rate is achievable. H ( X | Y ) Proposition (Converse) All achievable rates must satisfy . R R ≥ H ( X | Y ) 3

CHANNEL OUTPUT APPROXIMATION CHANNEL OUTPUT APPROXIMATION One shot channel coding Coding consists of encoder to approximate output statistics Enc : [1; M ] → X D ( P Z Q Z ∥ ) Lemma (Random coding for channel output approximation) W Z | X ( z | x ) For γ > 0 C γ ≜ { ( x , z ) ∈ X × Z ) : log ≤ γ } . Q Z ( z ) For a codebook of independently generated codewords , we have C X i ∼ p X 2 γ 1 E [ D ( P Z Q Z ∥ )] ≤ P P X W Z | X (( X , Z ) ∉ C γ ) log ( + 1 ) + , μ Z M where . μ Z ≜ min z ∈supp Q Z Q Z ( z ) 4

5

   6