Information Capacity of the BSC Permutation Channel Anuran Makur - PowerPoint PPT Presentation

Information Capacity of the BSC Permutation Channel Anuran Makur EECS Department, Massachusetts Institute of Technology Allerton Conference 2018 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 1 / 21

Outline Introduction 1 Motivation: Coding for Communication Networks The Permutation Channel Model Capacity of the BSC Permutation Channel Achievability 2 Converse 3 Conclusion 4 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 2 / 21

Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: 2 L input symbols Alphabet symbols = all possible L -bit packets ⇒ A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: Alphabet symbols = all possible L -bit packets multipath routed network or evolving network topology A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: Alphabet symbols = all possible L -bit packets multipath routed network ⇒ packets received with transpositions A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: Alphabet symbols = all possible L -bit packets multipath routed network ⇒ packets received with transpositions packets are impaired (e.g. deletions, substitutions) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: Alphabet symbols = all possible L -bit packets multipath routed network ⇒ packets received with transpositions packets are impaired ⇒ model using channel probabilities A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER NETWORK A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets Random deletion channel: Delete each symbol/packet of codeword independently with probability p ∈ (0 , 1) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets Random deletion channel: Delete each symbol/packet of codeword independently with probability p ∈ (0 , 1) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets Random deletion channel: Delete each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets Random deletion channel: Delete each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION ? ? NETWORK Abstraction: n -length codeword = sequence of n packets Equivalent Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION 1 ? 3 1 2 3 3 1 ? NETWORK Abstraction: n -length codeword = sequence of n packets Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword Coding: Add sequence numbers (packet size = L + log( n ) bits, alphabet size = n 2 L ) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION 1 ? 3 1 2 3 3 1 ? NETWORK Abstraction: n -length codeword = sequence of n packets Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword Coding: Add sequence numbers and use standard coding techniques A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION 1 ? 3 1 2 3 3 1 ? NETWORK Abstraction: n -length codeword = sequence of n packets Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword Coding: Add sequence numbers and use standard coding techniques More refined coding techniques simulate sequence numbers, e.g. [Mitzenmacher 2006], [Metzner 2009] A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION ? ? NETWORK Abstraction: n -length codeword = sequence of n packets Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword How do you code in such channels without increasing alphabet size? A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

The Permutation Channel Model � � � � 𝑁 𝑌 � 𝑎 � 𝑍 𝑁 RANDOM � CHANNEL ENCODER DECODER PERMUTATION Sender sends message M ∼ Uniform( M ) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 5 / 21

The Permutation Channel Model � � � � 𝑁 𝑌 � 𝑎 � 𝑍 𝑁 RANDOM � CHANNEL ENCODER DECODER PERMUTATION Sender sends message M ∼ Uniform( M ) Possibly randomized encoder f n : M → X n produces codeword X n 1 = ( X 1 , . . . , X n ) = f n ( M ) (with block-length n ) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 5 / 21

The Permutation Channel Model � � � � 𝑁 𝑌 � 𝑎 � 𝑍 𝑁 RANDOM � CHANNEL ENCODER DECODER PERMUTATION Sender sends message M ∼ Uniform( M ) Possibly randomized encoder f n : M → X n produces codeword X n 1 = ( X 1 , . . . , X n ) = f n ( M ) (with block-length n ) Discrete memoryless channel P Z | X with input and output alphabets X and Y produces Z n 1 : n � 1 ( z n 1 | x n P Z n 1 ) = P Z | X ( z i | x i ) 1 | X n i =1 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 5 / 21

The Permutation Channel Model � � � � 𝑁 𝑌 � 𝑎 � 𝑍 𝑁 RANDOM � CHANNEL ENCODER DECODER PERMUTATION Sender sends message M ∼ Uniform( M ) Possibly randomized encoder f n : M → X n produces codeword X n 1 = ( X 1 , . . . , X n ) = f n ( M ) (with block-length n ) Discrete memoryless channel P Z | X with input and output alphabets X and Y produces Z n 1 : n � 1 ( z n 1 | x n P Z n 1 ) = P Z | X ( z i | x i ) 1 | X n i =1 Random permutation generates Y n 1 from Z n 1 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 5 / 21