Fundamental Limits of Distributed Encoding Nastaran Abadi - - PowerPoint PPT Presentation

Fundamental Limits of Distributed Encoding

Nastaran Abadi Khooshemehr Mohammad Ali Maddah-Ali

Sharif University of Technology International Symposium on Information Theory (ISIT) 2020 June 2020

Classical Coding

Channel

Shannon approach

Probabilistic errors

Hamming approach

Adversarial errors 2 Source

Some fundamental lim limit its on the parameters of codes

Hamming approach

Adversarial errors Singleton bound If 𝐵𝑟(𝑜, 𝑒) is the maximum number

• f possible codewords in a 𝑟-ary block

code of length 𝑜 and minimum distance 𝑒, then 𝐵𝑟 𝑜, 𝑒 ≤ 𝑟𝑜−𝑒+1. Gilbert–Varshamov bound If 𝐵𝑟(𝑜, 𝑒) is the maximum number of possible codewords in a 𝑟-ary block code

• f length 𝑜 and minimum distance 𝑒, then

𝐵𝑟 𝑜, 𝑒 ≥

𝑟𝑜 σ𝑘=0

𝑒−1 𝑜 𝑘

𝑟−1 𝑘 .

Griesmer bound If 𝑂(𝑙, 𝑒) is the minimum length of a binary code of dimension 𝑙 and and minimum distance 𝑒, then 𝑂 𝑙, 𝑒 ≥ σ𝑗=0

𝑙−1 𝑒 2𝑗 .

and many more … 3

Let’s focus on the

4

A closer look at encoder

Channel

In some applications, the encoder can be distributed.

5 Source

Example of applications with distributed data sources

Shard 1 Shard 2 Shard 3

⋮

IoT Blockchain In these systems, the encoding is distributed as well as the data production.

6

Source node 1 Source node 2 Source node 3

Encoder

Distributed encoding

distributed source nodes 7

Source node 1 Source node 2 Source node 3

Distributed encoding

Decoder Decoder connects to some encoding nodes. 8

Distributed encoding with adversaries

Source node 1 Source node 2 Source node 3 9

Just one adversarial source node can undermine the system.

Source node 1 Source node 2 Source node 3

More variables than equations

10

Impossible to decode

11

We study distributed encoding system, where some source nodes are controlled by an adversary. An adversarial node sends up to a finite number of different messages to the encoding nodes.

We characterize the fundamental limit of this system.

There are methods to restrain the adversaries in distributed systems.

The adversary cannot inject too many different messages into the system.

12

Why do we assume an upper limit for the number of adversarial messages?

Objective in an adversarial distributed encoding system

Decoding the messages of the honest nodes correctly.

13

We do not care about the messages of the adversaries in decoding!

Distributed encoding system with adversaries

Source node 1 Source node 2 Source node 3 Decoder

We need the decoder to decode the messages of the honest nodes correctly.

14 No information about the adversaries and their behavior. No information about and .

We don’t care about the messages of adversaries.

System Parameters

𝜸: the number of adversaries 𝑳: the number of source nodes 𝒘: the maximum number of the messages of one adversarial source node 𝑶: the number of encoding nodes 𝒖: the number of encoding nodes that decoder needs to connect to. 𝐿 = 3 𝑂 = 5 𝛾 = 1 𝑤 = 3 15 # of adversaries # of adversarial messages # of source nodes # of encoding nodes

The problem

What is the fundamental limit of 𝑢 in an (𝑂, 𝐿, 𝛾, 𝑤) distributed encoding system? 𝑢∗: fundamental limit of 𝑢

(Informally, at least how many encoding nodes does the decoder need?)

16

Fundamental limit of 𝑢

In an 𝑂, 𝐿, 𝛾, 𝑤 distributed encoding system,

• if 𝑂 ≥ 𝐿 + 𝛾 𝑤 − 1 + 1

𝑢∗ = 𝐿 + 𝛾 𝑤 − 1 + 1

• If 𝐿 ≤ 𝑂 ≤ 𝐿 + 𝛾 𝑤 − 1

𝑢∗ = 𝑂

Theorem

Recall 17

Proof

Achievability There is a coding scheme where

• the decoder can connect to any 𝑢∗ encoding nodes,
• and generate an estimate for the input messages where the messages of

the honest nodes are correctly decoded. For achievability, we need a code, decoding process, and correctness proof. Converse There is no coding scheme in which

• the decoder connects to less than 𝑢∗ encoding nodes,
• and estimates the messages of the honest nodes correctly.

18

Achievability-code

We use this nonlinear code to achieve 𝑢∗.

19

𝑔

𝑜 𝑦𝑜1, … , 𝑦𝑜𝐿 = ෍ 𝑙=1 𝐿

𝛽𝑜𝑙 𝑦𝑜1 … 𝑦𝑜𝐿 𝑦𝑜𝑙 , 1 ≤ 𝑜 ≤ 𝑂

𝛽𝑜1, … , 𝛽𝑜𝐿: chosen independently and uniformly at random from the field

Using nonlinear code

• Hard for the adversary to evaluate the contribution of its messages in the encoded symbols
• Hard for the adversary to cause confusion in the decoder
• Having a set of nonlinear equations with possibly many solutions

nice structure

Achievability-code

𝐿 − 𝛾 + 𝛾𝑤 is the number of the variables in the system. With connecting to just one more encoding node and using the equation of that node, decoder can be successful.

20

𝑔

𝑜 𝑦𝑜1, … , 𝑦𝑜𝐿 = ෍ 𝑙=1 𝐿

𝛽𝑜𝑙 𝑦𝑜1 … 𝑦𝑜𝐿 𝑦𝑜𝑙 , 1 ≤ 𝑜 ≤ 𝑂

𝑢∗ = 𝐿 + 𝛾 𝑤 − 1 + 1

Achievability-decoding

Decoder considers every possible scenario and finds feasible solutions.

21

Achievability- correctness

22 We consider a partitioning for the encoding nodes. We form a set of nonlinear equations. In some steps, we transform it to another set of nonlinear equations. We use Bezout theorem to bound the number

• f the feasible and undesirable solutions.

all options for the messages

• f source nodes

feasible and undesirable solutions

We prove every feasible solution satisfies correctness.

Converse

23

For any code, if the decoder connects to less than 𝑢∗ nodes, there is a way that adversary can mislead the decoder.

The decoder does not know the adversaries and their behavior. Decoder would be confused between two contradicting feasible solutions.

Could we achieve 𝑢∗ with a linear code?

24

Fundamental limit of 𝑢- linear regime

In an 𝑂, 𝐿, 𝛾, 𝑤 distributed encoding system where 𝑔

1, … , 𝑔 𝑂 are

linear functions,

• if 𝑂 ≥ 𝐿 + 2𝛾 𝑤 − 1

𝑢linear

∗

= 𝐿 + 2𝛾 𝑤 − 1

• If 𝐿 ≤ 𝑂 ≤ 𝐿 + 2𝛾 𝑤 − 1 − 1

𝑢linear

∗

= 𝑂

Theorem (linear code)

25

In an 𝑂, 𝐿, 𝛾, 𝑤 distributed encoding system,

• if 𝑂 ≥ 𝐿 + 𝛾 𝑤 − 1 + 1

𝑢∗ = 𝐿 + 𝛾 𝑤 − 1 + 1

• If 𝐿 ≤ 𝑂 ≤ 𝐿 + 𝛾 𝑤 − 1

𝑢∗ = 𝑂

Theorem (general code) Linear code is not good enough!

Conclusion

• We introduced the problem of distributed encoding.
• We assumed that some of the source nodes are adversaries and send inconsistent

messages to the encoding nodes.

• We characterized the fundamental limit of the distributed encoding system.
• We established matching achievability and converse.
• We introduced nonlinear coding in order to achieve the fundamental limit.
• There are many more problems to solve
• How to optimize the decoding complexity?
• What if some of encoding nodes are adversaries as well?
• What is the fundamental limit if encoding nodes use a particular coding?
• …

26

Thank you

27