Asymptotics, asynchrony, and asymmetry in distributed consensus - - PowerPoint PPT Presentation

▶
asymptotics asynchrony and asymmetry in distributed
SMART_READER_LITE
LIVE PREVIEW

Asymptotics, asynchrony, and asymmetry in distributed consensus - - PowerPoint PPT Presentation

Feb 01, 2024 780 likes •2.12k views

DANCES Seminar 1 / 45 Asymptotics, asynchrony, and asymmetry in distributed consensus Anand D. Sarwate Information Theory and Applications Center University of California, San Diego 9 March 2011 Joint work with Alex G. Dimakis, Tuncer Can

slide-1
SLIDE 1

DANCES Seminar 1 / 45

Asymptotics, asynchrony, and asymmetry in distributed consensus

Anand D. Sarwate

Information Theory and Applications Center University of California, San Diego

9 March 2011

Joint work with Alex G. Dimakis, Tuncer Can Aysal, Mehmet Ercan Yildiz, Martin Wainwright, and Anna Scaglione, and Tara Javidi

UCSD Sarwate

slide-2
SLIDE 2

DANCES Seminar > Introduction 2 / 45

Rapprochement, consensus, accord

UCSD Sarwate

slide-3
SLIDE 3

DANCES Seminar > Introduction 2 / 45

Rapprochement, consensus, accord

UCSD Sarwate

slide-4
SLIDE 4

DANCES Seminar > Introduction 2 / 45

Rapprochement, consensus, accord

UCSD Sarwate

slide-5
SLIDE 5

DANCES Seminar > Introduction 2 / 45

Rapprochement, consensus, accord

UCSD Sarwate

slide-6
SLIDE 6

DANCES Seminar > Introduction 3 / 45

Consensus is an important task

  • Calibration
  • Dissemination
  • Coordination

UCSD Sarwate

slide-7
SLIDE 7

DANCES Seminar > Introduction 4 / 45

Abstracting the task

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

UCSD Sarwate

slide-8
SLIDE 8

DANCES Seminar > Introduction 4 / 45

Abstracting the task

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Network of agents, each with an observation

UCSD Sarwate

slide-9
SLIDE 9

DANCES Seminar > Introduction 4 / 45

Abstracting the task

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Network of agents, each with an observation
  • Communicate locally – exchange messages about observations

UCSD Sarwate

slide-10
SLIDE 10

DANCES Seminar > Introduction 4 / 45

Abstracting the task

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Network of agents, each with an observation
  • Communicate locally – exchange messages about observations
  • Compute locally – estimate a function of all values

UCSD Sarwate

slide-11
SLIDE 11

DANCES Seminar > Introduction 5 / 45

There are many aspects to consider

UCSD Sarwate

slide-12
SLIDE 12

DANCES Seminar > Introduction 5 / 45

There are many aspects to consider

  • What are observations?
  • continuous or discrete?
  • scalar or vector?

UCSD Sarwate

slide-13
SLIDE 13

DANCES Seminar > Introduction 5 / 45

There are many aspects to consider

  • What are observations?
  • continuous or discrete?
  • scalar or vector?
  • How can we communicate?
  • point-to-point or broadcast?
  • low resolution or high resolution?

UCSD Sarwate

slide-14
SLIDE 14

DANCES Seminar > Introduction 5 / 45

There are many aspects to consider

  • What are observations?
  • continuous or discrete?
  • scalar or vector?
  • How can we communicate?
  • point-to-point or broadcast?
  • low resolution or high resolution?
  • What do we compute?
  • averages
  • medians, quantiles
  • convex optimization

UCSD Sarwate

slide-15
SLIDE 15

DANCES Seminar > Introduction 6 / 45

The goal(s) for today

W1,i x1(t) x2(t) x3(t) x4(t) x5(t) xi(t) W2,i W3,i W4,i W5,i

1 The basic mathematical model for consensus 2 Routing and mobility can speed up convergence 3 Broadcasting can trade off accuracy for speed 4 The discreet charm of discrete messages

UCSD Sarwate

slide-16
SLIDE 16

DANCES Seminar > A simple mathematical model 7 / 45

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

Building a mathematical model

UCSD Sarwate

slide-17
SLIDE 17

DANCES Seminar > A simple mathematical model 8 / 45

The data model

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

UCSD Sarwate

slide-18
SLIDE 18

DANCES Seminar > A simple mathematical model 8 / 45

The data model

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Set of n agents

UCSD Sarwate

slide-19
SLIDE 19

DANCES Seminar > A simple mathematical model 8 / 45

The data model

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Set of n agents
  • Agent i observes initial value xi(0) ∈ R for i = 1, 2 . . . n

UCSD Sarwate

slide-20
SLIDE 20

DANCES Seminar > A simple mathematical model 8 / 45

The data model

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Set of n agents
  • Agent i observes initial value xi(0) ∈ R for i = 1, 2 . . . n
  • Assume data is bounded : xi(0) ∈ [0, 10], for example

UCSD Sarwate

slide-21
SLIDE 21

DANCES Seminar > A simple mathematical model 9 / 45

The communication graph

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

UCSD Sarwate

slide-22
SLIDE 22

DANCES Seminar > A simple mathematical model 9 / 45

The communication graph

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Agents are arranged in a graph G = (V, E).

UCSD Sarwate

slide-23
SLIDE 23

DANCES Seminar > A simple mathematical model 9 / 45

The communication graph

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Agents are arranged in a graph G = (V, E).
  • Agents i can communicate with j if there is an edge (i, j) (e.g.

j ∈ Ni).

UCSD Sarwate

slide-24
SLIDE 24

DANCES Seminar > A simple mathematical model 9 / 45

The communication graph

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Agents are arranged in a graph G = (V, E).
  • Agents i can communicate with j if there is an edge (i, j) (e.g.

j ∈ Ni).

  • Bidirectional communication : agents exchange messages.

UCSD Sarwate

slide-25
SLIDE 25

DANCES Seminar > A simple mathematical model 10 / 45

Constraints on the communication

UCSD Sarwate

slide-26
SLIDE 26

DANCES Seminar > A simple mathematical model 10 / 45

Constraints on the communication

  • Time is slotted : only one transmission per slot.

UCSD Sarwate

slide-27
SLIDE 27

DANCES Seminar > A simple mathematical model 10 / 45

Constraints on the communication

  • Time is slotted : only one transmission per slot.
  • Synchronous : use many edges, then update.

UCSD Sarwate

slide-28
SLIDE 28

DANCES Seminar > A simple mathematical model 10 / 45

Constraints on the communication

  • Time is slotted : only one transmission per slot.
  • Synchronous : use many edges, then update.
  • Asynchronous : edges chosen randomly in each slot.

UCSD Sarwate

slide-29
SLIDE 29

DANCES Seminar > A simple mathematical model 11 / 45

Measuring performance

The goal is to pass messages between agents such that they can estimate the average of the initial observations: x(t) →

  • i

xi(0)

  • · 1

Averaging time Tave(n, ǫ) is time when x(t) is within ǫ of the average: Tave(n, ǫ) = sup

x(0)

inf

t

  • PAlg

x(t) − xave · 1 x(0) ≥ ǫ

  • ≤ ǫ
  • UCSD

Sarwate

slide-30
SLIDE 30

DANCES Seminar > A simple mathematical model 12 / 45

A centralized solution

3 5 7 2 8 1 6 4 8 7 6 2 3 5 4 9

Simple centralized algorithm:

UCSD Sarwate

slide-31
SLIDE 31

DANCES Seminar > A simple mathematical model 12 / 45

A centralized solution

3 5 7 2 8 1 6 4 8 7 6 2 3 5 4 9

Simple centralized algorithm:

1 Build a spanning tree

UCSD Sarwate

slide-32
SLIDE 32

DANCES Seminar > A simple mathematical model 12 / 45

A centralized solution

3 5 7 2 8 1 6 4 8 7 6 2 3 5 4 9

Simple centralized algorithm:

1 Build a spanning tree 2 Gather all the values at

root

UCSD Sarwate

slide-33
SLIDE 33

DANCES Seminar > A simple mathematical model 12 / 45

A centralized solution

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

Simple centralized algorithm:

1 Build a spanning tree 2 Gather all the values at

root

3 Compute and disseminate

average

UCSD Sarwate

slide-34
SLIDE 34

DANCES Seminar > A simple mathematical model 12 / 45

A centralized solution

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

Simple centralized algorithm:

1 Build a spanning tree 2 Gather all the values at

root

3 Compute and disseminate

average Pro: requires Θ(n) messages Con: completely centralized

UCSD Sarwate

slide-35
SLIDE 35

DANCES Seminar > A simple mathematical model 13 / 45

Distributed synchronous consensus

Suppose each agent linearly combines itself and its neighbors: xi(t + 1) = Wiixi(t) +

  • j∈Ni

Wijxj(t)

  • j

Wij = 1 ∀i Wij = Wji Synchronous algorithm where the update after each slot is given by: x(t + 1) = Wx(t) where W is a doubly stochastic matrix.

UCSD Sarwate

slide-36
SLIDE 36

DANCES Seminar > A simple mathematical model 14 / 45

A simple result

Theorem For synchronous consensus with update matrix W, Tave(n, ǫ) = Θ

  • |E| · Trelax(W) · log ǫ−1

where Trelax(W) is the relaxation time of the matrix W: Trelax(W) = 1 1 − λ2(W) . Proof : W is the transition matrix of a Markov chain – consensus is the convergence of the chain to its stationary distribution.

UCSD Sarwate

slide-37
SLIDE 37

DANCES Seminar > A simple mathematical model 15 / 45

A theme with variations

Survey article by Dimakis et al. in Proc. IEEE.

  • Synchronous DeGroot (1974), Tsitsiklis (1984)
  • Time-varying topologies Chatterjee-Seneta (1977), Tsitsiklis et al. (1986),

Jadbabaie et al. (2003), Ren-Beard (2005), Gao-Cheng (2006), Fagnani-Zampieri (2008)

  • Asynchronous Boyd et al. (2006)
  • Quantization Kashyap et al. (2007), Nedic et al. (2009), Yildiz-Scaglione

(2008), Aysal et. al (2009), Kar-Moura (2010), Carli et al. (2010), Lavaie-Murray (2010)

  • Discrete values Benezit et al. (2010)
  • Many others!

UCSD Sarwate

slide-38
SLIDE 38

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

UCSD Sarwate

slide-39
SLIDE 39

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.

UCSD Sarwate

slide-40
SLIDE 40

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.

UCSD Sarwate

slide-41
SLIDE 41

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 7 3 5 9 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-42
SLIDE 42

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 7 3 7 7 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-43
SLIDE 43

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 7 3 7 7 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-44
SLIDE 44

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 7 3 7 7 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-45
SLIDE 45

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 5 5 7 7 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-46
SLIDE 46

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 5 5 7 7 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-47
SLIDE 47

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 5 5 7 7 8 4 1 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-48
SLIDE 48

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 5 3 7 7 8 4 3 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-49
SLIDE 49

DANCES Seminar > A simple mathematical model 16 / 45

Asynchronous updates = “gossip”

3 5 5 3 7 7 8 4 3 7 4 5 8 2 7 6 2 3 5 4 9

  • Node i wakes up at random, chooses neighbor j at random.
  • Nodes i and j exchange xi(t) and xj(t) and compute average.
  • Set xi(t + 1) = xj(t + 1) = 1

2(xi(t) + xj(t)).

UCSD Sarwate

slide-50
SLIDE 50

DANCES Seminar > A simple mathematical model 17 / 45

Gossip uses random linear updates

At each time a random pair (i, j) ∈ E averages: xi(t + 1) = xj(t + 1) = xi(t) + xj(t) 2 . Each update is linear : x(t + 1) = W (i,j)(t)x(t). Theorem Let ¯ W = E[W (i,j)] over the edge selection process. Then Tave(n, ǫ) = Θ

  • Trelax( ¯

W) · log ǫ−1

UCSD Sarwate

slide-51
SLIDE 51

DANCES Seminar > A simple mathematical model 18 / 45

The implication for big graphs

For the grid with uniform selection, gossip takes Θ(n2) transmissions! Selecting edges at random is inefficient! Local exchange is inefficient!

UCSD Sarwate

slide-52
SLIDE 52

DANCES Seminar > Shrinking the graph 19 / 45

Network properties can accelerate convergence

Joint work with Alex Dimakis and Martin Wainwright

UCSD Sarwate

slide-53
SLIDE 53

DANCES Seminar > Shrinking the graph 20 / 45

Geographic gossip with routing

UCSD Sarwate

slide-54
SLIDE 54

DANCES Seminar > Shrinking the graph 20 / 45

Geographic gossip with routing

  • Assume that packets can be routed between any two nodes.

UCSD Sarwate

slide-55
SLIDE 55

DANCES Seminar > Shrinking the graph 20 / 45

Geographic gossip with routing

  • Assume that packets can be routed between any two nodes.
  • Now select “neighbor” uniformly from all nodes and route

message.

UCSD Sarwate

slide-56
SLIDE 56

DANCES Seminar > Shrinking the graph 20 / 45

Geographic gossip with routing

  • Assume that packets can be routed between any two nodes.
  • Now select “neighbor” uniformly from all nodes and route

message.

  • “Effective graph” is now the complete graph.

UCSD Sarwate

slide-57
SLIDE 57

DANCES Seminar > Shrinking the graph 21 / 45

Example : the grid

algorithm Trelax( ¯ W)

UCSD Sarwate

slide-58
SLIDE 58

DANCES Seminar > Shrinking the graph 21 / 45

Example : the grid

algorithm Trelax( ¯ W) Local Θ(n2)

UCSD Sarwate

slide-59
SLIDE 59

DANCES Seminar > Shrinking the graph 21 / 45

Example : the grid

algorithm Trelax( ¯ W) Local Θ(n2) With routing Θ(n)

UCSD Sarwate

slide-60
SLIDE 60

DANCES Seminar > Shrinking the graph 21 / 45

Example : the grid

algorithm Trelax( ¯ W) Local Θ(n2) With routing Θ(n) This is unfair, since routing costs in number of hops.

UCSD Sarwate

slide-61
SLIDE 61

DANCES Seminar > Shrinking the graph 22 / 45

One-hop transmissions to reach consensus

Count number of hops (power) to get within ǫ of the average: algorithm

  • ne-hop transmission

UCSD Sarwate

slide-62
SLIDE 62

DANCES Seminar > Shrinking the graph 22 / 45

One-hop transmissions to reach consensus

Count number of hops (power) to get within ǫ of the average: algorithm

  • ne-hop transmission

Local Θ(n2)

Boyd et al.

UCSD Sarwate

slide-63
SLIDE 63

DANCES Seminar > Shrinking the graph 22 / 45

One-hop transmissions to reach consensus

Count number of hops (power) to get within ǫ of the average: algorithm

  • ne-hop transmission

Local Θ(n2)

Boyd et al.

With routing Θ(n3/2)

Dimakis,Sarwate, Wainwright

UCSD Sarwate

slide-64
SLIDE 64

DANCES Seminar > Shrinking the graph 22 / 45

One-hop transmissions to reach consensus

Count number of hops (power) to get within ǫ of the average: algorithm

  • ne-hop transmission

Local Θ(n2)

Boyd et al.

With routing Θ(n3/2)

Dimakis,Sarwate, Wainwright

Average on the way Θ(n)

Benezit et al.

UCSD Sarwate

slide-65
SLIDE 65

DANCES Seminar > Shrinking the graph 23 / 45

Gossip with mobility

UCSD Sarwate

slide-66
SLIDE 66

DANCES Seminar > Shrinking the graph 23 / 45

Gossip with mobility

  • Start with a grid of static nodes.

UCSD Sarwate

slide-67
SLIDE 67

DANCES Seminar > Shrinking the graph 23 / 45

Gossip with mobility

  • Start with a grid of static nodes.
  • Add m fully mobile nodes.

UCSD Sarwate

slide-68
SLIDE 68

DANCES Seminar > Shrinking the graph 23 / 45

Gossip with mobility

  • Start with a grid of static nodes.
  • Add m fully mobile nodes.

UCSD Sarwate

slide-69
SLIDE 69

DANCES Seminar > Shrinking the graph 23 / 45

Gossip with mobility

  • Start with a grid of static nodes.
  • Add m fully mobile nodes.
  • At each time, m mobile nodes choose new locations uniformly at

random.

UCSD Sarwate

slide-70
SLIDE 70

DANCES Seminar > Shrinking the graph 23 / 45

Gossip with mobility

  • Start with a grid of static nodes.
  • Add m fully mobile nodes.
  • At each time, m mobile nodes choose new locations uniformly at

random.

UCSD Sarwate

slide-71
SLIDE 71

DANCES Seminar > Shrinking the graph 23 / 45

Gossip with mobility

  • Start with a grid of static nodes.
  • Add m fully mobile nodes.
  • At each time, m mobile nodes choose new locations uniformly at

random.

UCSD Sarwate

slide-72
SLIDE 72

DANCES Seminar > Shrinking the graph 24 / 45

Gossip with mobility

UCSD Sarwate

slide-73
SLIDE 73

DANCES Seminar > Shrinking the graph 24 / 45

Gossip with mobility

  • Same local transmission model.

UCSD Sarwate

slide-74
SLIDE 74

DANCES Seminar > Shrinking the graph 24 / 45

Gossip with mobility

  • Same local transmission model.
  • Mobile nodes reduce effective diameter to 2.

UCSD Sarwate

slide-75
SLIDE 75

DANCES Seminar > Shrinking the graph 24 / 45

Gossip with mobility

  • Same local transmission model.
  • Mobile nodes reduce effective diameter to 2.
  • Mobile nodes are accessed rarely.

UCSD Sarwate

slide-76
SLIDE 76

DANCES Seminar > Shrinking the graph 25 / 45

Lower bounds on Trelax( ¯ W)

UCSD Sarwate

slide-77
SLIDE 77

DANCES Seminar > Shrinking the graph 25 / 45

Lower bounds on Trelax( ¯ W)

  • Merge all mobile nodes into a “super node.”

UCSD Sarwate

slide-78
SLIDE 78

DANCES Seminar > Shrinking the graph 25 / 45

Lower bounds on Trelax( ¯ W)

  • Merge all mobile nodes into a “super node.”
  • Trelax for induced chain ≤ Trelax for original chain.

UCSD Sarwate

slide-79
SLIDE 79

DANCES Seminar > Shrinking the graph 25 / 45

Lower bounds on Trelax( ¯ W)

  • Merge all mobile nodes into a “super node.”
  • Trelax for induced chain ≤ Trelax for original chain.
  • At most a m-factor improvement.

UCSD Sarwate

slide-80
SLIDE 80

DANCES Seminar > Shrinking the graph 26 / 45

Upper bounds on Trelax( ¯ W)

π(i) π(j)

π(i)Wik

Use a “flow” argument and the Poincar´ e inequality:

UCSD Sarwate

slide-81
SLIDE 81

DANCES Seminar > Shrinking the graph 26 / 45

Upper bounds on Trelax( ¯ W)

π(i) π(j)

π(i)Wik

Use a “flow” argument and the Poincar´ e inequality:

  • Demands Dij = π(i)π(j) = n−2 between each pair of nodes.

UCSD Sarwate

slide-82
SLIDE 82

DANCES Seminar > Shrinking the graph 26 / 45

Upper bounds on Trelax( ¯ W)

π(i) π(j)

π(i)Wik

Use a “flow” argument and the Poincar´ e inequality:

  • Demands Dij = π(i)π(j) = n−2 between each pair of nodes.
  • Capacity Cik = π(i) ¯

Wik = n−1 ¯ Wik between each edge.

UCSD Sarwate

slide-83
SLIDE 83

DANCES Seminar > Shrinking the graph 26 / 45

Upper bounds on Trelax( ¯ W)

Dij Dij

Cik

Use a “flow” argument and the Poincar´ e inequality:

  • Demands Dij = π(i)π(j) = n−2 between each pair of nodes.
  • Capacity Cik = π(i) ¯

Wik = n−1 ¯ Wik between each edge.

UCSD Sarwate

slide-84
SLIDE 84

DANCES Seminar > Shrinking the graph 26 / 45

Upper bounds on Trelax( ¯ W)

Dij Dij

Cik

Use a “flow” argument and the Poincar´ e inequality:

  • Demands Dij = π(i)π(j) = n−2 between each pair of nodes.
  • Capacity Cik = π(i) ¯

Wik = n−1 ¯ Wik between each edge.

  • Route flows i → j to minimize overload on each edge.

UCSD Sarwate

slide-85
SLIDE 85

DANCES Seminar > Shrinking the graph 26 / 45

Upper bounds on Trelax( ¯ W)

Dij Dij

Cik

Use a “flow” argument and the Poincar´ e inequality:

  • Demands Dij = π(i)π(j) = n−2 between each pair of nodes.
  • Capacity Cik = π(i) ¯

Wik = n−1 ¯ Wik between each edge.

  • Route flows i → j to minimize overload on each edge.

UCSD Sarwate

slide-86
SLIDE 86

DANCES Seminar > Shrinking the graph 27 / 45

Network effects on convergence

algorithm transmissions

UCSD Sarwate

slide-87
SLIDE 87

DANCES Seminar > Shrinking the graph 27 / 45

Network effects on convergence

algorithm transmissions Local Θ(n2)

Boyd et al.

UCSD Sarwate

slide-88
SLIDE 88

DANCES Seminar > Shrinking the graph 27 / 45

Network effects on convergence

algorithm transmissions Local Θ(n2)

Boyd et al.

With routing Θ(n3/2)

Dimakis-Sarwate-Wainwright

UCSD Sarwate

slide-89
SLIDE 89

DANCES Seminar > Shrinking the graph 27 / 45

Network effects on convergence

algorithm transmissions Local Θ(n2)

Boyd et al.

With routing Θ(n3/2)

Dimakis-Sarwate-Wainwright

Average on the way Θ(n)

Benezit et al.

UCSD Sarwate

slide-90
SLIDE 90

DANCES Seminar > Shrinking the graph 27 / 45

Network effects on convergence

algorithm transmissions Local Θ(n2)

Boyd et al.

With routing Θ(n3/2)

Dimakis-Sarwate-Wainwright

Average on the way Θ(n)

Benezit et al.

Add m mobile Θ

  • n2

m

  • Sarwate-Dimakis

UCSD Sarwate

slide-91
SLIDE 91

DANCES Seminar > Shrinking the graph 27 / 45

Network effects on convergence

algorithm transmissions Local Θ(n2)

Boyd et al.

With routing Θ(n3/2)

Dimakis-Sarwate-Wainwright

Average on the way Θ(n)

Benezit et al.

Add m mobile Θ

  • n2

m

  • Sarwate-Dimakis

k-local O

  • n2

k2

  • Sarwate-Dimakis

UCSD Sarwate

slide-92
SLIDE 92

DANCES Seminar > Trading accuracy for speed 28 / 45

3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

Asymmetric gossip using broadcasting

Joint work with T.C. Aysal, M.E. Yildiz and A. Scaglione

UCSD Sarwate

slide-93
SLIDE 93

DANCES Seminar > Trading accuracy for speed 29 / 45

Wireless is inherently broadcast

UCSD Sarwate

slide-94
SLIDE 94

DANCES Seminar > Trading accuracy for speed 29 / 45

Wireless is inherently broadcast

  • In a wireless network, all neighbors can hear a transmission.

UCSD Sarwate

slide-95
SLIDE 95

DANCES Seminar > Trading accuracy for speed 29 / 45

Wireless is inherently broadcast

  • In a wireless network, all neighbors can hear a transmission.
  • Can perform multiple computations per slot.

UCSD Sarwate

slide-96
SLIDE 96

DANCES Seminar > Trading accuracy for speed 29 / 45

Wireless is inherently broadcast

  • In a wireless network, all neighbors can hear a transmission.
  • Can perform multiple computations per slot.
  • When graph is well-connected, can get performance gains.

UCSD Sarwate

slide-97
SLIDE 97

DANCES Seminar > Trading accuracy for speed 30 / 45

Gossip in one direction

2 4 6 6

4

7 8 8 9 9 2 2 3 4 6 7 3 8 1 8 4 5 6 1 2 2 4 4 2 2 7 2 1 9

UCSD Sarwate

slide-98
SLIDE 98

DANCES Seminar > Trading accuracy for speed 30 / 45

Gossip in one direction

2 4 6 6

4

7 8 8 9 9 2 2 3 4 6 7 3 8 1 8 4 5 6 1 2 2 4 4 2 2 7 2 1 9

  • All neighbors j ∈ Ni of

node i can hear transmission.

UCSD Sarwate

slide-99
SLIDE 99

DANCES Seminar > Trading accuracy for speed 30 / 45

Gossip in one direction

3 4 2 5 5

4

3 4 2 5 5

4

7 8 8 9 9 2 2 3 4 6 7 3 8 1 8 4 5 6 1 2 2 4 4 2 2 7 2 1 9

  • All neighbors j ∈ Ni of

node i can hear transmission.

  • Can do a simultaneous

update xj(t + 1) = γxj(t) + (1 − γ)xi(t).

UCSD Sarwate

slide-100
SLIDE 100

DANCES Seminar > Trading accuracy for speed 30 / 45

Gossip in one direction

3 4 2 5 5

4

7 8 8 9 9 2 2 3 4 6 7 3 8 1 8 4 5 6 1 2 2 4 4 2 2 7 2 1 9

  • All neighbors j ∈ Ni of

node i can hear transmission.

  • Can do a simultaneous

update xj(t + 1) = γxj(t) + (1 − γ)xi(t).

UCSD Sarwate

slide-101
SLIDE 101

DANCES Seminar > Trading accuracy for speed 30 / 45

Gossip in one direction

7 8 8 9 3 4 2 5 5

4

9 2 2 3 4 6 7 3 8 1 8 4 5 6 1 2 2 4 4 2 2 7 2 1 9

  • All neighbors j ∈ Ni of

node i can hear transmission.

  • Can do a simultaneous

update xj(t + 1) = γxj(t) + (1 − γ)xi(t).

UCSD Sarwate

slide-102
SLIDE 102

DANCES Seminar > Trading accuracy for speed 30 / 45

Gossip in one direction

7 8 8 9 3 4 2 5 5

4

9 2 2 3 4 6 7 3 8 1 8 4 5 6 1 2 2 4 4 2 2 7 2 1 9

  • All neighbors j ∈ Ni of

node i can hear transmission.

  • Can do a simultaneous

update xj(t + 1) = γxj(t) + (1 − γ)xi(t).

  • No information exchange

– can get consensus (agreement) but not the true average.

UCSD Sarwate

slide-103
SLIDE 103

DANCES Seminar > Trading accuracy for speed 31 / 45

Analyzing the broadcast gossip algorithm

2 4 6 6 4 7 8 8 9 9 2 2 3 4 6 7 3 8 1 8 4 5 6 1 2 2 4 4 2 2 7 2 1 9

Again, update given by a matrix multiplication: x(T) = T

  • t=1

W (it)

  • x(0)

For all t we have W (it)1 = 1, so consensus is stable.

UCSD Sarwate

slide-104
SLIDE 104

DANCES Seminar > Trading accuracy for speed 32 / 45

Benefits and challenges of broadcast

2 4 6 6 4 7 8 8 9 9 2 2 3 4 6 7 3 8 1 8 4 5 6 1 2 2 4 4 2 2 7 2 1 9

  • No coordination to exchange data.
  • Exploits potential long-range connections from shadowing/fading.
  • No convergence to true average, but to consensus.
  • Important to control the MSE of the consensus.

UCSD Sarwate

slide-105
SLIDE 105

DANCES Seminar > Trading accuracy for speed 33 / 45

Main results

Algorithm reaches consensus almost surely: P

  • lim

t→∞ x(t) = c1

  • = 1.

The expected consensus value is the true average: E[c] = ¯ x Moreover, there is a closed form for the limiting MSE.

UCSD Sarwate

slide-106
SLIDE 106

DANCES Seminar > Trading accuracy for speed 34 / 45

Simulations : MSE

1000 2000 3000 4000 5000 10

−6

10

−5

10

−4

10

−3

10

−2

10

−1

10 Number of Radio Transmissions Per Node MSE Standard N=50 Geographic N=50 Broadcast N=50 UCSD Sarwate

slide-107
SLIDE 107

DANCES Seminar > Trading accuracy for speed 34 / 45

Simulations : MSE

1000 2000 3000 4000 5000 10

−3

10

−2

10

−1

10 Number of Radio Transmissions Per Node MSE Randomized N=100 Geographic N=100 Broadcast N=100 UCSD Sarwate

slide-108
SLIDE 108

DANCES Seminar > Trading accuracy for speed 35 / 45

Extensions

UCSD Sarwate

slide-109
SLIDE 109

DANCES Seminar > Trading accuracy for speed 35 / 45

Extensions

  • Can look at effect of the wireless medium as well.

UCSD Sarwate

slide-110
SLIDE 110

DANCES Seminar > Trading accuracy for speed 35 / 45

Extensions

  • Can look at effect of the wireless medium as well.
  • Fading allows long-distance connections.

UCSD Sarwate

slide-111
SLIDE 111

DANCES Seminar > Trading accuracy for speed 35 / 45

Extensions

  • Can look at effect of the wireless medium as well.
  • Fading allows long-distance connections.
  • Initial results suggest significant improvement when path loss is

not too severe.

UCSD Sarwate

slide-112
SLIDE 112

DANCES Seminar > Trading accuracy for speed 36 / 45

Implications

UCSD Sarwate

slide-113
SLIDE 113

DANCES Seminar > Trading accuracy for speed 36 / 45

Implications

  • Broadcasting is simpler than standard gossip – no exchange.

UCSD Sarwate

slide-114
SLIDE 114

DANCES Seminar > Trading accuracy for speed 36 / 45

Implications

  • Broadcasting is simpler than standard gossip – no exchange.
  • More robust to packet drops which may occur in wireless.

UCSD Sarwate

slide-115
SLIDE 115

DANCES Seminar > Trading accuracy for speed 36 / 45

Implications

  • Broadcasting is simpler than standard gossip – no exchange.
  • More robust to packet drops which may occur in wireless.
  • Faster convergence in small-to-medium networks.

UCSD Sarwate

slide-116
SLIDE 116

DANCES Seminar > Discrete consensus 37 / 45

k 2R k + 1 2R

Reaching consensus discretely

Joint work with Tara Javidi

UCSD Sarwate

slide-117
SLIDE 117

DANCES Seminar > Discrete consensus 38 / 45

Typical assumptions are unrealistic

3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

Existing work doesn’t “look practical”:

UCSD Sarwate

slide-118
SLIDE 118

DANCES Seminar > Discrete consensus 38 / 45

Typical assumptions are unrealistic

3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

Existing work doesn’t “look practical”:

  • Transmit and receive real numbers

UCSD Sarwate

slide-119
SLIDE 119

DANCES Seminar > Discrete consensus 38 / 45

Typical assumptions are unrealistic

3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

Existing work doesn’t “look practical”:

  • Transmit and receive real numbers
  • Consensus is the only goal of the network

UCSD Sarwate

slide-120
SLIDE 120

DANCES Seminar > Discrete consensus 38 / 45

Typical assumptions are unrealistic

3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

Existing work doesn’t “look practical”:

  • Transmit and receive real numbers
  • Consensus is the only goal of the network
  • Asymptotics and universality

UCSD Sarwate

slide-121
SLIDE 121

DANCES Seminar > Discrete consensus 39 / 45

Synchronous quantized communication

R R R R R R R R R R R R R R R R 3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

UCSD Sarwate

slide-122
SLIDE 122

DANCES Seminar > Discrete consensus 39 / 45

Synchronous quantized communication

R R R R R R R R R R R R R R R R 3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

  • At each time t all neighbors (i, j) exchange quantized values

ˆ xj(t).

UCSD Sarwate

slide-123
SLIDE 123

DANCES Seminar > Discrete consensus 39 / 45

Synchronous quantized communication

R R R R R R R R R R R R R R R R 3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

  • At each time t all neighbors (i, j) exchange quantized values

ˆ xj(t).

  • Messages i → j and j → i must take no more than R bits.

UCSD Sarwate

slide-124
SLIDE 124

DANCES Seminar > Discrete consensus 39 / 45

Synchronous quantized communication

R R R R R R R R R R R R R R R R 3 2 5 4 4 6 1 5 6 3 8 8 8 4 3 7 5 8 7 6 9 3

  • At each time t all neighbors (i, j) exchange quantized values

ˆ xj(t).

  • Messages i → j and j → i must take no more than R bits.
  • Update xi(t + 1) as a function of xi(t) and messages {ˆ

xj(t)}.

UCSD Sarwate

slide-125
SLIDE 125

DANCES Seminar > Discrete consensus 40 / 45

A simple protocol

W1,i x1(t) x2(t) x3(t) x4(t) x5(t) xi(t) W2,i W3,i W4,i W5,i

xi(t + 1) = (xi(t) − ˆ xi(t)) +

  • j∈Ni∪{i}

Wij ˆ xj(t).

  • Quantization error plus weighted sum of messages
  • Iterations preserve sum

i xi(t)

UCSD Sarwate

slide-126
SLIDE 126

DANCES Seminar > Discrete consensus 41 / 45

So how well does it work?

UCSD Sarwate

slide-127
SLIDE 127

DANCES Seminar > Discrete consensus 41 / 45

So how well does it work? Random topology, 49 nodes, good connectivity

5 10 15 20 25 30 35 40 45 50 −2 2 4 6 8 10

Log error Iterations

UCSD Sarwate

slide-128
SLIDE 128

DANCES Seminar > Discrete consensus 41 / 45

So how well does it work? Random topology, 49 nodes, poor connectivity

5 10 15 20 25 30 35 40 45 50 −2 2 4 6 8 10

Log error Iterations

UCSD Sarwate

slide-129
SLIDE 129

DANCES Seminar > Discrete consensus 41 / 45

So how well does it work? Grid, 100 nodes

10 20 30 40 50 60 70 80 90 100 −2 2 4 6 8 10

Log error Iterations

UCSD Sarwate

slide-130
SLIDE 130

DANCES Seminar > Discrete consensus 42 / 45

Observations

t + 1 t

  • Quantization is important for practical applications.
  • Average consensus to within reasonable resolution can be fast.
  • Overhead can be reduced by piggybacking on existing traffic.

UCSD Sarwate

slide-131
SLIDE 131

DANCES Seminar > Conclusions 43 / 45

Conclusions

W1,i x1(t) x2(t) x3(t) x4(t) x5(t) xi(t) W2,i W3,i W4,i W5,i

  • Algorithm can use network resources to accelerate convergence.
  • Reaching consensus may be faster than computing averages.
  • Lower-resolution averages can be fast and require less overhead.

UCSD Sarwate

slide-132
SLIDE 132

DANCES Seminar > Conclusions 44 / 45

Some challenges for the future

IEEE 802.xyz.pdq

  • Implementing consensus in protocols for applications.
  • Extending to other distributed computation problems.
  • Quantifying robustness in rate, connectivity, etc.

UCSD Sarwate

slide-133
SLIDE 133

DANCES Seminar > Conclusions 45 / 45

Thank you!

UCSD Sarwate