Function Computation in Networked Environments Vinay A. Vaishampayan - - PowerPoint PPT Presentation

Function Computation in Networked Environments

Vinay A. Vaishampayan

City University of New York

July 25, 2018

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 1 / 57

Outline: I

While physical layer communication technologies, especially in optical and wireless communication continue to be of great importance, the internet has brought a lot of importance to application layer systems and application layer performance. Today, many application are distributed in some sense, e.g. in a data center a single machine is not sufficient to handle an application, and in a wide area network, many geographically separated nodes need to collaborate. Thus it becomes important to understand the communication needs

• f specific applications that are to be implemented in a distributed

manner. Information theoretically, a good starting point to develop this understanding is through the problems of distributed function computation and communication complexity. Problem originally formulated in late 1970’s.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 2 / 57

Outline: II

This talk will serve as an introduction, describe some applications, and some of the authors own research. Organization of the talk:

◮ Problem Description and Formulation ◮ Some Theory ◮ Some Applications ◮ Specific Research Problem: Nearest Lattice Point Search ◮ Summary Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 3 / 57

Part I: Problem Definition

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 4 / 57

Communication Complexity of Distributed Function Computation

x₁ x₂ xn x₃ Nodes+at+which+f+is+required

Given function f : X1 × X2 . . . × Xn → Z. Observations x1 ∈ Xi, i = 1, 2, . . . , n are available at physically separated locations in a network. Compute f (x1, x2, . . . , xn). Function value should be available at designated nodes.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 5 / 57

Distributed Function Computation: Problem Definition

Communication is carried out using a pre-arranged protocol, Π R(Π): Number of bits communicated by participants in Π for computing f (x1, x2, . . . , xn). Communication complexity of f : C(f ) = min

Π that compute f

R(Π)

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 6 / 57

Terminology

party ≡ sensor. n = 2: two-party. n > 2: multiparty.

◮ For 2-party: X1 = X, X2 = Y.

Where is the result required?

◮ At a single location (fusion center): Centralized. ◮ At all sensor nodes: Distributed

Nature of the protocol: Non-interactive/ Interactive How is the rate measured? Worst-case, average Exact computation or approximate computation.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 7 / 57

Interactive Communication: Problem Setup

∗ A. C. Yao, Some Complexity Questions Related to Distributive Computing, ACM 1979.

Time Enc Dec

Alice Bob

Enc Dec Enc Dec Enc Dec Enc Dec Enc Dec

X Y

m1=f(X) (R1 bits) m2=f(Y,m1) (R2 bits) m3=f(X,m1,m2) (R3 bits) mN=f(X,m1,m2,...,mN-1)

g(X,Y) g(X,Y)

Alice has x, Bob has y, both wish to compute g(x, y). Communication in rounds following a pre-decided protocol.

Definition

Two-way communication complexity is the minimum number of bits that must be exchanged for the worst-case input so that Alice and Bob can compute f (x, y)

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 8 / 57

Example: [Yao 1979]

f A B C a 1 b c 1 1 1 d 1 e 1 1

{a} {A,C} {B} {b} {c} {d,e} {A,B} {d} {e} {C} {d} {e}

Interactive protocol, distributed. Worst case inputs require 4 bits. Average rate: 9/5 bits. Assumes equally likely and independent inputs. Observation: function must be constant on every leaf of the tree.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 9 / 57

Part II: Theory

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 10 / 57

Basic Results

C(f ) ≤ min(log2 |X|, log2 |Y|) + log2 |Range(f )|

◮ Proof: Alice sends x ∈ X to Bob using log2 |X| bits. Bob sends back

f (x, y) using log2 |Range(f )| bits.

◮ This is the ‘obvious’ method. Idea is to improve on this.

C(f ) ≥ log2 |Range(f )|

◮ Proof: |Range(f )| is the number of leaves of the code tree. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 11 / 57

Rectangles

Definition (Rectangle)

R = A × B : A ⊂ X, B ⊂ Y

Definition (Monochromatic Rectangle)

Rectangle R such that f (x, y) is constant on R. Leaf nodes of a protocol Π that compute f are monochromatic rectangles. Any protocol Π that computes f induces a partition of X × Y into monochromatic rectangles.

Theorem

If any monochromatic rectangular partition of X × Y has at least T rectangles, then C(f ) ≥ log2 T.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 12 / 57

Monochromatic Rectangles due to Algorithm Π

Algorithm Π

{a} {A,C} {B} {b} {c} {d,e} {A,B} {d} {e} {C} {d} {e}

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 13 / 57

Basic Results [KN:1997]

Definition (Fooling Set S)

S ⊂ X × Y f (x, y) = z, (x, y) ∈ S. For each distinct pair (x1, y1), (x2, y2) ∈ S, either f (x1, y2) = z or f (x2, y1) = z.

Theorem

If a function f has a fooling set of size T then C(f ) ≥ log2 T.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 14 / 57

Rank Lower Bound

Mehlhorn and Schmidt, “Las Vegas is Better than Determinism in VLSI and Distributed computing,” STOC, 1982.

f A B C a 1 1 b 1 1 1 c 1 1 d

{a,b} {A,B} {C} {c,d} {A} {B,C}

    1 1 1 1 1 1 1    

• Mf

=     1 1 1 1    

• M1

+     1    

• M2

+     1 1    

• M3

rank(Mf ) ≤ rank(M1) + rank(M2) + rank(M3) ≤ T

• #leaves

Theorem

C(f ) ≥ log2 rank(Mf )

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 15 / 57

Information Theory: Distributed Function Computation

• J. Korner and K. Marton, How to Encode the Modulo-2 Sum of Binary Sources, IEEE IT 1979

Encoder 1 Decoder X Fusion Center Encoder 2 Y Rate=Rg1 bits/sample Rate=Rg2 bits/sample g(X,Y)

Pair of random variables X, Y . Function g : X × Y → R. Reproduce g(X, Y ) exactly in F. Rg1 > I(U; X|V ), Rg2 > I(V ; Y |U), Rg1 + Rg2 > I(UV , XY ). U, V auxiliary random variables that satisfy U − X − Y − V and H(g(X, Y )|UV ) = 0. A graph-theoretic characterization based on maximally independent sets of a graph determined by g is in

• A. Orlitsky and J. Roche, Coding for Computing, IEEE IT, 2001

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 16 / 57

Interactive Communication: Rate Region

• N. Ma and S. Ishwar, “Some Results on Distributed Source Coding for Interactive Function

Computation,” IEEE IT 2011.

Time Enc Dec

Alice Bob

Enc Dec Enc Dec Enc Dec Enc Dec Enc Dec

X Y

m1=f(X) (R1 bits) m2=f(Y,m1) (R2 bits) m3=f(X,m1,m2) (R3 bits) mN=f(X,m1,m2,...,mN-1)

g(X,Y) g(X,Y)

R1 ≥ I(X; U1|Y ), U1 − X − Y R2 ≥ I(Y ; U2|X, U1), U2 − (Y , U1) − X R3 ≥ I(X; U3|Y , U1, U2), U3 − (X, U1, U2) − Y . . . . . . R2n ≥ I(Y ; U2n|X, U1, . . . , U2n−1) U2n − (X, U1, U2, ..., U2n−1) − Y H(G|X, U2n

1 ) = 0, H(G|Y , U2n 1 ) = 0

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 17 / 57

Comment About the Nature of Combinatorial and Information Theoretic Results

As stated the combinatorial problem is to obtain the complexity of computing f (x, y). Information theoretic characterizations are for the number of bits Rn for computing (f (x1, y1), f (x2, y2), . . . , f (xn, yn)) with rate R = Rn/n, as n → ∞. This is referred to as the Direct Sum problem in the combinatorial literature. Thus information theory addresses the direct-sum problem, where it can use ‘hardening’ properties that result from the Law of Large Numbers. Best example: Communication complexity of ‘AND’. Single shot: 1.5

• bits. Using Information theoretic ideas, Ma and Ishwar showed this

can be reduced to 1.361 bits.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 18 / 57

Key References to Part II

[KN:1997] Kushilevitz and Nisan, Communication Complexity, Cambridge Univ. Press. 1997. [GK:2011]A. El Gamal and Y.-H. Kim, Network Information Theory, Cambridge Univ. Press, 2011.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 19 / 57

Part III: Applications

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 20 / 57

Early Application: Area-time Tradeoff in VLSI

• C. D. Thomson, Area-Time Complexity for VLSI, Caltech Conference on VLSI, Jan. 1979.

One of the early works showing that communication complexity, rather than computation complexity, had physical implications. In VLSI chip design, we would like to keep A, the chip area, as well as T, the time to complete a task, small. Problem considered: VLSI implementation of n-point DFT. AT 2 ≥ n2/16, so both cannot be small simultaneously. Wires ≡ Communication. In VLSI chips, area of wires that interconnect processing elements dominate the total area. Bound is obtained by analyzing the communication requirements of the algorithm.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 21 / 57

Current Application 1: Distributed Antennas

P1 F P2 Pn

x1 x2 xn Desired: f(x1,x2...,xn)

Base Station 1 Base Station 2 Base Station n

Actual: g(y1,y2,...,yn) y1 y2 yn Fusion Center

M1 M2 Mu

n base stations, each observes ith coordinate xi of a noisy codeword (x1, x2, . . . , xn). Objective: Calculate f (x1, x2, . . . , xn) at the fusion center, where f is the ‘most likely’ codeword. Base station -Fusion center Link: Finite capacity R.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 22 / 57

Application 2: Machine Learning in Wide Area Networks

Lewis, Noah, Sergey Plis, and Vince Calhoun. ”Cooperative learning: Decentralized data neural network.” Neural Networks (IJCNN), 2017 International Joint Conference on. IEEE, 2017.

Many small data sets; Distributed geographically and in different administrative domains. Legal, privacy, ethical hurdles to data pooling. Learning algorithms perform poorly on small data sets. Goal is to develop communication efficient learning algorithm. Distributed algorithm for training tap weights of a neural network. fMRI, MNIST... Steps taken to reduce the amount of communication by restricting the number of gradients shared. Tradeoff between communication and accuracy of the classifier. Careful mathematical/information theoretic analysis would be beneficial.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 23 / 57

Machine Learning in Data Centers

• M. Li, D. G. Andersen, A. J. Smola, and K. Yu. Communication efficient distributed machine learning with the parameter server.

In Advances in Neural Information Processing Systems, pages 19–27, 2014.

Large datasets ≈ 100TB Large models: 109–1012 parameters. Single machine is not powerful enough to carry out computations. Workload is partitioned among worker machines, which need to frequently access the central server to get parameter updates. Datacenter inter-machine connectivity bandwidth is 10-100 times smaller than memory bandwidth. Thus solving such distributed problems, even in a single datacenter creates bottlenecks and delays for itself and for other applications running in a datacenter. Frequent synchronization between machines is required. Slowest machine will hold up the computation.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 24 / 57

Security

• E. Vasilomanolakis, S. Karuppayah, M. Muhlhauser, and M. Fischer. Taxonomy and survey of collaborative intrusion detection.

ACM Computing Surveys (CSUR), 47(4):55, 2015.

• W. Wei, F. Chen, Y. Xia, and G. Jin. A rank correlation based detection against distributed redirection DOS attacks. IEEE

Communications Letters, 17(1):173-175, 2013.

Collaborative Intrusion Detection Systems (CIDS) designed to thwart distributed attacks.

◮ Distributed denial of service attacks. ◮ Covert communications to a compromised insider.

Requirements: Accuracy (False positives, false negatives), Low overhead (Computation and Communication overhead), scalability, resilience, privacy,...

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 25 / 57

Sample Security Problems

Fusion Center: Event Detection

x1 x2 x3 xn

Identify users that received more than 1000 remote login requests during a given day. Identify all destinations that received more than 10GB of traffic in a given hour and identify how much traffic they received. Is there a sudden increase in traffic that crosses a specific link? Early detection of denial of service attack. Has there been a sudden change in correlation structure? Each problem places a communication load on the network and would benefit from well-designed distributed algorithms for function computation.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 26 / 57

Games and Learning

Conitzer, Vincent, and Tuomas Sandholm. ”Communication complexity as a lower bound for learning in games.” Proceedings of the twenty-first international conference on Machine learning. ACM, 2004.

Two-player game, each player knows only her own payoff but not the payoff of the other player. Objective: Each player wishes to compute a binary function of the game, e.g. does the game have a Nash equilibrium? Key restriction: Communication may only occur through a player’s

• moves. Explicit communication is disallowed.

Role of Communication complexity: Number of rounds ≥ Lower Bound on Communication complexity Upper Bound on Communication per Round

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 27 / 57

Part IV: Communication Complexity and Lattice Decoding.

Joint work with Maiara Bollauf and her advisor Prof. Sueli Costa

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 28 / 57

Lattices in Communication

A lattice Λ is a discrete additive subgroup of Rn. Lattices are widely used in communications.

◮ Modulation codebook for communication over Gaussian channel:

Lattices are capacity achieving.

◮ Source Coding: As codebooks for lossy compression of continuous

alphabet sources, lattices are known to achieve performance close to the rate distortion function.

Most previous lattice applications have been ‘lumped’, i.e. the vector to be encoded is available at a single physical location. In many emerging applications this assumption does not hold. What is the communication cost of implementing lattice codes in a distributed setting?

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 29 / 57

Problem Studied: Communication Efficient Nearest Lattice Point Search

+ + + + + + + + + + + + + + + + + + + + +

x=(x1,x2)

x₁ x₂ Informa+on,exchange,to, find,closest,la6ce,point, at,both,loca+ons Node,1 Node,2

Given a lattice Λ ⊂ Rn. f (x1, x2, . . . , xn) is the nearest lattice vector to (x1, x2, . . . , xn). We determine upper bounds for the communication cost of the nearest lattice point problem. Motivations:

◮ compression. ◮ decoding of a message.

Broader applications:

◮ Distributed classification problems. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 30 / 57

Lattices: Voronoi Cell; Nearest Lattice Point Problem

+ + + + + + + + + + + + + + + + + + + + + v₁ v₂

A Lattice Λ is a discrete additive group in Rn. Λ = {Vu, u ∈ Zn}, where

◮ Generator matrix:

V =

• v1 v2 . . . vn
• and column vector vi ∈ Rn is the ith basis vector.

Nearest lattice point problem is NP-hard. Fast algorithms for approximate search. Here we assume that V is upper triangular.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 31 / 57

Voronoi and Babai (Partitions, Cells, Points)

1 Given lattice Λ ⊂ R2 with basis V =

1 a b

• .

2 Typical lattice vector (u1 + au2, bu2). 3 Given x = (x1, x2)

Voronoi point: λV (x) = u∗

1v1 + u∗ 2v2 where

u∗ = (u∗

1, u∗ 2) minimizes

(x2 − bu2)2 + (x1 − (u1 + au2))2. Voronoi Cell: V(λ) = {x ∈ Rn : x − λ ≤ x − λ′, λ′ = λ}. Voronoi Partition: collection of all Voronoi cells. Babai Point: λB(x) = u1v1 + u2v2. u2 = [x2/b], u1 = [x1 − au2] B(λ) = {x : λB(x) = λ}. Babai cell. Babai Partition: collection of Babai cells.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 32 / 57

Nearest Lattice point: Two-Step Process

Two step process is considered (x → λB(x) → λV (x))

1 Stage-I: Compute λB(x). At conclusion of Stage-I ◮ both nodes have λB(x). ◮ each node subtracts off its coordinate of λB(x) from x. ◮ (new) x is uniformly distributed over B(0). ◮ Pe,I = 1 − Area(B(0) V(0))/Area(B(0)) 2 Stage-II: Correct λB(x) to λV (x) by sending extra bits. ◮ Determine communication cost. ◮ Determine the residual error probability, Pe,II Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 33 / 57

Distributed Computation Models Considered

• 1. Centralized mode: result is

required at a single node distinct from the two sensor nodes.

• 2. Interactive: single round

x₁ x₂

Encoder Encoder Decoder U1.(R₁.bits) Decoder U2.(R₂.bits)

Node.1 Node.2

8me λ(x) λ(x)

1 12 or 21. 2 Nonzero Pe,II at the end of a

single round. Determine best tradeoff with rate.

• 3. Interactive infinite rounds.

x1 x2 Node 1 Node 2

time Encoder Encoder Decoder U1,1 (R11 bits) Decoder U2,1 (R21 bits) Encoder Encoder Decoder U1,m (R1m bits) Decoder U2,m (R2m bits) λv λv STOP STOP

Round 1 Round m

1 2121... 2 Zero Pe,II at conclusion. Bits

and Rounds random variable depend on x. Evaluated average number of bits and rounds.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 34 / 57

Error Probability Calculation for the Babai Point

[BVC] M. F. Bollauf, V. A. Vaishampayan and S. I. R. Costa, ”On the communication cost of determining an approximate nearest lattice point,” 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, 2017, pp. 1838-1842.

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 2.0

Pe =

1 4ρ | cos θ| sin2 θ (1 − ρ| cos θ|)

Lattice Z2: Pe = 0. Lattice A2: Hexagonal Lattice has worst error probability among 2D lattices. A2 is known to be best for both compression and coding! Pe is invariant to scaling the lattice, i.e. indep.

• f α if generator is αV , α ∈ R.

Problem with generalization to higher dimensions:

◮ Pe is harder to evaluate in higher dimensions.

n = 3, see M. Bollauf’s thesis. n > 3, bounding techniques are being developed.

◮ Parameterization of lattices in higher

dimensions is incomplete.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 35 / 57

Error Probability vs. Packing Density

Definition (Packing Density)

∆n =

Vnρn √ det Λ

0.80 0.82 0.84 0.86 0.88 0.90 Packing Density (Δ2) 0.02 0.04 0.06 0.08 Error Probability 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Packing density 0.05 0.10 0.15 0.20 0.25 0.30 Probability of error

2D 3D Z3

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 36 / 57

Rate Calculation for Babai Partition

Node n Xn Node n-1 Xn-1 Node 1 X1 Fully Connected Mesh U1 Un Un-1 Entropy Encoder Entropy Decoder

= V =       v11 v12 . . . v1n v22 . . . v2n . . . . . . vnn      

Interactive Model, One round of communication V : upper triangular. ui =

• xi − n

j=i+1 vijuj

vii

• R = (n − 1) n

i=1 H(Ui|Ui+1, Ui+2, . . . , Un)

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 37 / 57

Rate Calculation for Babai Point: Centralized Model

X1 X2 Xn

Node 1 Node 2 Node n

F λB(x) u1 u2 un

Node n-1

Xn-1 un-1

Cannot perform ui =

• xi − n

j=i+1 vijuj

vii

• , i < n

because ui+1, ..., un are needed at node i. Extra Information must be transmitted: Centralized Communication Protocol

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 38 / 57

Centralized Communication Protocol Πc

V =       v11 v12 . . . v1n v22 . . . v2n . . . . . . vnn      

Purpose: Send extra information to fusion center for Babai point computation. Let vm,l/vm,m = pm,l/qm,l where pm,l and qm,l > 0 are relatively prime. Let qm = l.c.m {qm,l, l > m}. By definition qn = 1.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 39 / 57

Protocol Πc

Protocol

(Transmission, Πc).

1 Let s(m) ∈ {0, 1, . . . , qm − 1} be the largest s for which

[xm/vm,m − s/qm] = [xm/vm,m].

2 Node m sends ˜

bm = [xm/vm,m] and s(m) to F, m = 1, 2, . . . , n (by definition s(n) = 0).

Theorem

The Babai point b can be determined at the fusion center F after running transmission protocol Πc.

Corollary

The rate required to transmit s(m), m = 1, 2, . . . , n − 1 is no larger than n−1

i=1 log2(qi) bits. Does not depend on scale α.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 40 / 57

From Babai to Voronoi: Single Round

• V. A. Vaishampayan and M. F. Bollauf, ”Communication cost of transforming a nearest plane partition to the Voronoi

partition,” 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, 2017, pp. 1843-1847.

Node 1 partitions support of x1 into intervals Ii, of length δi, i = 1, 2, . . . , N (equivalently into vertical strips). Node 1 sends index u1 = i if x1 ∈ Ii. Node 2 sub-partitions vertical strip i into ≤ three parts R−1, R0 and R1. Node 2 sends u2 = j to node 1 if x2 ∈ Rj. At the conclusion of the round, the error probability Pe,II is the sum

• f areas of the small triangles.

We optimize over δi’s and the sub-partition of each vertical strip to

• btain expressions for rates and Pe,II.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 41 / 57

Detail of the Sub-partition

Voronoi&cell&boundary

R₀ R1₁ R₁

Sum&of&areas&is& minimized&when& cut&is&midway

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 42 / 57

Babai to Voronoi: Single Round. Results

limR→∞ Pe,II2R/(1−P0) = α2

• 1 + L1

L2 α1L2 α2L1

• L1

(L1+L2) 2 (κ+H(P)) (1−P0) .

P0 = 1 − ρ cos θ Rate of decay depends on the lattice. 12 and 21 have different performance.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 43 / 57

Babai to Voronoi: Infinite Rounds, Algorithm Detail

212121... Partition after one round (magenta) and two rounds (magenta+dashes) At conclusion of each round, each node knows rectangle that x lies in. Define a rectangle to be without-error if its interior intersects boundary of Voronoi cell, with-error otherwise. For each x, STOP when each node knows x is in an error-free rectangle.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 44 / 57

Babai to Voronoi: Infinite Rounds, Detail

x1 x2 (0,0) (1,1) y1=(1-x1/L1) (0,0) (L1,L2) y2=x2/L2 y1=101101... y2=010011... stop independent iid strings

Stop after N = n rounds if bit representations derived from x1 and x2 differ for first time in nth bit. Bits strings are balanced Bernoulli, hence Pr(N = n) = 2−n. On average a finite number of bits and rounds suffice to refine a Babai partition to the Voronoi Partition. ¯ R = H(Q) + (1 − Q0)H(P) + 4(1 − P0)(1 − Q0) ¯ N = 1 + 2(1 − P0)(1 − Q0)

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 45 / 57

Performance Results

0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 Pe,II #10-3 5 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 Pe,II #10-4 2 4 (2/:)*3 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

E[R]

2 4

Figure: Variation of Pe,II with θ for the single-round interactive model, 12 (top), 21 (middle) with R = 4.0 bits. ¯ R = E[R] for the infinite-round interactive model is shown in the bottom panel for ρ = 1.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 46 / 57

A Lower Bound for min(X, Y )

An interactive protocol results in a (combinatorial) rectangular partition. We need to relate a rectangular partition P to its communication cost C(P). minP C(P) is the desired lower bound.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 47 / 57

Simpler Problem: I

f(x)=0 f(x)=1

X = (X1, X2) uniform on the unit square. f (x) as shown above. Simple bit-exchange protocol results in average communication cost

• f 1.5 bits.

To find a lower bound, consider the constraints on the probabilities of a rectangular partition of the sets Q = f −1(0) and P = f −1(1).

◮ For Q: m

j=1 pij ≤ 1/4, m = 1, 2, . . . , for any subsequence ij.

◮ For P: qi ≤ 1/2, i = 1, 2, . . ., m

j=1 qij ≤ 3/4, m = 2, . . . , for any

increasing subsequence ij.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 48 / 57

Simpler Problem: II

f(x)=0 f(x)=1

Entropy is minimized at the vertices of the constraint set. Vertices of Q partition are (1/4, 0, 0, . . .) and any of its permutations. Vertices of P partition are (1/2, 1/4, 0, 0, . . .) and any of its permutations. Thus minimum entropy is H(1/4, 1/2, 1/4) = 3/2 bits. Since the bit-exchange protocol achieves 3/2 bits, this lower bound is tight!

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 49 / 57

A Lower Bound for min(X, Y )

f(x)=0 f(x)=1

Complication: Slanted boundary. Try to follow same approach as in previous example.

Theorem

The partition probabilities of a zero-error partition (P, Q) satisfy the following constraints:

m

• j=1

pij ≤ m 2(m + 1) , m = 1, 2, . . . (1)

m

• j=1

qij ≤ m 2(m + 1) , m = 1, 2, . . . (2)

• i

pi = 1/2, (3)

• i

qi = 1/2, (4) for any increasing subsequence of positive integers {ij}.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 50 / 57

A Lower Bound for min(X, Y )

f(x)=0 f(x)=1

Not every vertex is achievable by a rectangular partition, e.g. (1/4, 1/12, 1/24, ...) Tighter characterization of constraint set appears to be complicated. However, we can sidestep characterization of the constraint set through majorization.

Definition

Let p = (p1, p2, . . . , ) and q = (q1, q2, . . . , ) be two probability vectors with probabilities in nonincreasing order. Then p majorizes q, written p q, if k

i=1 pi ≥ k i=1 qi, for k = 1, 2, . . ..

Lemma

If p q then H(p) ≤ H(q).

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 51 / 57

Using Majorization to Characterize an Optimal Partition

Theorem

If a partition minimizes the entropy it contains a rectangle with vertices (1, 0) and (v, v) and another rectangle with vertices (0, 1) and (u, u), for some 0 < u, v < 1.

Proof.

Given any partition P, let R be rectangle with largest probability. Grow R until one vertex touches boundary. New partition: Q.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 52 / 57

Using Majorization...

v 0.2 0.4 0.6 0.8 1 H([v2,2v(1-v), (1-v)2])/(2v(1-v)) 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5

Theorem

The minimum single-shot interactive communication cost of the Πmin2 problem is four bits.

Proof.

Self-Similarity of partition implies H(P, Q|C = 1) = H([v2, 2v(1 − v), (1 − v)2]) 2v(1 − v) (5) whose unique minimum value of 3 bits occurs when u = v = 1/2.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 53 / 57

Summary of Communication Complexity for Nearest Lattice Point Problem

Two distributed function computation models—central and distributed. Information theoretic bounds are available in both cases, but not easily computable. Derived bounds for communication complexity of Nearest Lattice Point problem. Two stages: Babai Partition. Refine to Voronoi Partition. Surprise: On average a finite number of bits suffice to recompute the partition.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 54 / 57

Part V: Summary, Conclusions, Future Work

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 55 / 57

Future Work

Current interest is strong and will continue to grow, driven by trends in collaborative systems for machine learning, AI, security, and wireless. Applications are diverse and cover a broad range: from social architectures to fundamental physics. Constructions, especially for multi-party settings are scarce, but are needed. Many theoretical results are order of magnitude results; not sufficient crutch for a code designer. Solutions are extremely problem-dependent. Learning and exploiting structure of each problem requires a significant effort. Good potential for collaborative efforts. Information theoretic lower bounds: very little is known in multi-party settings. Rather than focus on a single function for accomplishing a given task, study a class of solutions or algorithms. Which algorithm accomplishes the best tradeoff between effectiveness and communication complexity? Systems-level work: Integration with existing collaborative systems.

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 56 / 57

Thanks!

Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 57 / 57