Asymptotics, asynchrony, and asymmetry in distributed consensus - PowerPoint PPT Presentation

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 1 5 7 4 3 7 2 0 9 6 4 5 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 1 5 7 4 3 7 2 0 9 6 4 5 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 1 5 7 4 3 7 2 0 7 6 4 7 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 1 5 7 4 3 7 2 0 7 6 4 7 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 1 5 7 4 3 7 2 0 7 6 4 7 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 1 5 7 4 5 5 2 0 7 6 4 7 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 1 5 7 4 5 5 2 0 7 6 4 7 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 1 5 7 4 5 5 2 0 7 6 4 7 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 3 5 7 4 3 5 2 0 7 6 4 7 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 16 / 45 Asynchronous updates = “gossip” 3 5 7 4 3 5 2 0 7 6 4 7 7 8 3 2 8 5 3 0 4 9 5 • Node i wakes up at random, chooses neighbor j at random. • Nodes i and j exchange x i ( t ) and x j ( t ) and compute average. • Set x i ( t + 1) = x j ( t + 1) = 1 2 ( x i ( t ) + x j ( t )) . UCSD Sarwate

DANCES Seminar > A simple mathematical model 17 / 45 Gossip uses random linear updates At each time a random pair ( i, j ) ∈ E averages: x i ( t + 1) = x j ( t + 1) = x i ( t ) + x j ( t ) . 2 Each update is linear : x ( t + 1) = W ( i,j ) ( t ) x ( t ) . Theorem Let ¯ W = E [ W ( i,j ) ] over the edge selection process. Then T relax ( ¯ W ) · log ǫ − 1 � � T ave ( n, ǫ ) = Θ UCSD Sarwate

DANCES Seminar > A simple mathematical model 18 / 45 The implication for big graphs For the grid with uniform selection, gossip takes Θ( n 2 ) transmissions! Selecting edges at random is inefficient! Local exchange is inefficient! UCSD Sarwate

DANCES Seminar > Shrinking the graph 19 / 45 Network properties can accelerate convergence Joint work with Alex Dimakis and Martin Wainwright UCSD Sarwate

DANCES Seminar > Shrinking the graph 20 / 45 Geographic gossip with routing UCSD Sarwate

DANCES Seminar > Shrinking the graph 20 / 45 Geographic gossip with routing • Assume that packets can be routed between any two nodes. UCSD Sarwate

DANCES Seminar > Shrinking the graph 20 / 45 Geographic gossip with routing • Assume that packets can be routed between any two nodes. • Now select “neighbor” uniformly from all nodes and route message. UCSD Sarwate

DANCES Seminar > Shrinking the graph 20 / 45 Geographic gossip with routing • Assume that packets can be routed between any two nodes. • Now select “neighbor” uniformly from all nodes and route message. • “Effective graph” is now the complete graph. UCSD Sarwate

DANCES Seminar > Shrinking the graph 21 / 45 Example : the grid T relax ( ¯ algorithm W ) UCSD Sarwate

DANCES Seminar > Shrinking the graph 21 / 45 Example : the grid T relax ( ¯ algorithm W ) Θ( n 2 ) Local UCSD Sarwate

DANCES Seminar > Shrinking the graph 21 / 45 Example : the grid T relax ( ¯ algorithm W ) Θ( n 2 ) Local With routing Θ( n ) UCSD Sarwate

DANCES Seminar > Shrinking the graph 21 / 45 Example : the grid T relax ( ¯ algorithm W ) Θ( n 2 ) Local With routing Θ( n ) This is unfair, since routing costs in number of hops. UCSD Sarwate

DANCES Seminar > Shrinking the graph 22 / 45 One-hop transmissions to reach consensus Count number of hops (power) to get within ǫ of the average: algorithm one-hop transmission UCSD Sarwate

DANCES Seminar > Shrinking the graph 22 / 45 One-hop transmissions to reach consensus Count number of hops (power) to get within ǫ of the average: algorithm one-hop transmission Θ( n 2 ) Local Boyd et al. UCSD Sarwate

DANCES Seminar > Shrinking the graph 22 / 45 One-hop transmissions to reach consensus Count number of hops (power) to get within ǫ of the average: algorithm one-hop transmission Θ( n 2 ) Local Boyd et al. Θ( n 3 / 2 ) With routing Dimakis,Sarwate, Wainwright UCSD Sarwate

DANCES Seminar > Shrinking the graph 22 / 45 One-hop transmissions to reach consensus Count number of hops (power) to get within ǫ of the average: algorithm one-hop transmission Θ( n 2 ) Local Boyd et al. Θ( n 3 / 2 ) With routing Dimakis,Sarwate, Wainwright Average on the way Θ( n ) Benezit et al. UCSD Sarwate

DANCES Seminar > Shrinking the graph 23 / 45 Gossip with mobility UCSD Sarwate

DANCES Seminar > Shrinking the graph 23 / 45 Gossip with mobility • Start with a grid of static nodes. UCSD Sarwate

DANCES Seminar > Shrinking the graph 23 / 45 Gossip with mobility • Start with a grid of static nodes. • Add m fully mobile nodes . UCSD Sarwate

DANCES Seminar > Shrinking the graph 23 / 45 Gossip with mobility • Start with a grid of static nodes. • Add m fully mobile nodes . UCSD Sarwate

DANCES Seminar > Shrinking the graph 23 / 45 Gossip with mobility • Start with a grid of static nodes. • Add m fully mobile nodes . • At each time, m mobile nodes choose new locations uniformly at random. UCSD Sarwate

DANCES Seminar > Shrinking the graph 23 / 45 Gossip with mobility • Start with a grid of static nodes. • Add m fully mobile nodes . • At each time, m mobile nodes choose new locations uniformly at random. UCSD Sarwate

DANCES Seminar > Shrinking the graph 23 / 45 Gossip with mobility • Start with a grid of static nodes. • Add m fully mobile nodes . • At each time, m mobile nodes choose new locations uniformly at random. UCSD Sarwate

DANCES Seminar > Shrinking the graph 24 / 45 Gossip with mobility UCSD Sarwate

DANCES Seminar > Shrinking the graph 24 / 45 Gossip with mobility • Same local transmission model. UCSD Sarwate

DANCES Seminar > Shrinking the graph 24 / 45 Gossip with mobility • Same local transmission model. • Mobile nodes reduce effective diameter to 2. UCSD Sarwate

DANCES Seminar > Shrinking the graph 24 / 45 Gossip with mobility • Same local transmission model. • Mobile nodes reduce effective diameter to 2. • Mobile nodes are accessed rarely. UCSD Sarwate

DANCES Seminar > Shrinking the graph 25 / 45 Lower bounds on T relax ( ¯ W ) UCSD Sarwate

DANCES Seminar > Shrinking the graph 25 / 45 Lower bounds on T relax ( ¯ W ) • Merge all mobile nodes into a “super node.” UCSD Sarwate

DANCES Seminar > Shrinking the graph 25 / 45 Lower bounds on T relax ( ¯ W ) • Merge all mobile nodes into a “super node.” • T relax for induced chain ≤ T relax for original chain. UCSD Sarwate

DANCES Seminar > Shrinking the graph 25 / 45 Lower bounds on T relax ( ¯ W ) • Merge all mobile nodes into a “super node.” • T relax for induced chain ≤ T relax for original chain. • At most a m -factor improvement. UCSD Sarwate

DANCES Seminar > Shrinking the graph 26 / 45 Upper bounds on T relax ( ¯ W ) π ( i ) π ( i ) W ik π ( j ) Use a “flow” argument and the Poincar´ e inequality: UCSD Sarwate

DANCES Seminar > Shrinking the graph 26 / 45 Upper bounds on T relax ( ¯ W ) π ( i ) π ( i ) W ik π ( j ) Use a “flow” argument and the Poincar´ e inequality: • Demands D ij = π ( i ) π ( j ) = n − 2 between each pair of nodes. UCSD Sarwate

DANCES Seminar > Shrinking the graph 26 / 45 Upper bounds on T relax ( ¯ W ) π ( i ) π ( i ) W ik π ( j ) Use a “flow” argument and the Poincar´ e inequality: • Demands D ij = π ( i ) π ( j ) = n − 2 between each pair of nodes. W ik = n − 1 ¯ • Capacity C ik = π ( i ) ¯ W ik between each edge. UCSD Sarwate

DANCES Seminar > Shrinking the graph 26 / 45 Upper bounds on T relax ( ¯ W ) D ij C ik D ij Use a “flow” argument and the Poincar´ e inequality: • Demands D ij = π ( i ) π ( j ) = n − 2 between each pair of nodes. W ik = n − 1 ¯ • Capacity C ik = π ( i ) ¯ W ik between each edge. UCSD Sarwate

DANCES Seminar > Shrinking the graph 26 / 45 Upper bounds on T relax ( ¯ W ) D ij C ik D ij Use a “flow” argument and the Poincar´ e inequality: • Demands D ij = π ( i ) π ( j ) = n − 2 between each pair of nodes. W ik = n − 1 ¯ • Capacity C ik = π ( i ) ¯ W ik between each edge. • Route flows i → j to minimize overload on each edge. UCSD Sarwate

DANCES Seminar > Shrinking the graph 26 / 45 Upper bounds on T relax ( ¯ W ) D ij C ik D ij Use a “flow” argument and the Poincar´ e inequality: • Demands D ij = π ( i ) π ( j ) = n − 2 between each pair of nodes. W ik = n − 1 ¯ • Capacity C ik = π ( i ) ¯ W ik between each edge. • Route flows i → j to minimize overload on each edge. UCSD Sarwate

DANCES Seminar > Shrinking the graph 27 / 45 Network effects on convergence algorithm transmissions UCSD Sarwate

DANCES Seminar > Shrinking the graph 27 / 45 Network effects on convergence algorithm transmissions Θ( n 2 ) Local Boyd et al. UCSD Sarwate

DANCES Seminar > Shrinking the graph 27 / 45 Network effects on convergence algorithm transmissions Θ( n 2 ) Local Boyd et al. Θ( n 3 / 2 ) With routing Dimakis-Sarwate-Wainwright UCSD Sarwate

DANCES Seminar > Shrinking the graph 27 / 45 Network effects on convergence algorithm transmissions Θ( n 2 ) Local Boyd et al. Θ( n 3 / 2 ) With routing Dimakis-Sarwate-Wainwright Θ( n ) Average on the way Benezit et al. UCSD Sarwate

DANCES Seminar > Shrinking the graph 27 / 45 Network effects on convergence algorithm transmissions Θ( n 2 ) Local Boyd et al. Θ( n 3 / 2 ) With routing Dimakis-Sarwate-Wainwright Θ( n ) Average on the way Benezit et al. � � n 2 Add m mobile Θ Sarwate-Dimakis m UCSD Sarwate

DANCES Seminar > Shrinking the graph 27 / 45 Network effects on convergence algorithm transmissions Θ( n 2 ) Local Boyd et al. Θ( n 3 / 2 ) With routing Dimakis-Sarwate-Wainwright Θ( n ) Average on the way Benezit et al. � � n 2 Add m mobile Θ Sarwate-Dimakis m � � n 2 k -local O Sarwate-Dimakis k 2 UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 28 / 45 6 8 5 7 4 5 3 8 6 5 8 4 3 1 3 8 4 2 0 6 9 3 7 Asymmetric gossip using broadcasting Joint work with T.C. Aysal, M.E. Yildiz and A. Scaglione UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 29 / 45 Wireless is inherently broadcast UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 29 / 45 Wireless is inherently broadcast • In a wireless network, all neighbors can hear a transmission. UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 29 / 45 Wireless is inherently broadcast • In a wireless network, all neighbors can hear a transmission. • Can perform multiple computations per slot. UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 29 / 45 Wireless is inherently broadcast • In a wireless network, all neighbors can hear a transmission. • Can perform multiple computations per slot. • When graph is well-connected, can get performance gains. UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 30 / 45 Gossip in one direction 6 8 6 4 0 9 5 1 2 7 3 2 7 3 2 9 8 4 4 2 4 6 1 0 6 2 7 2 2 8 4 9 2 1 4 8 UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 30 / 45 Gossip in one direction 6 8 6 4 0 • All neighbors j ∈ N i of 9 5 1 node i can hear 2 7 3 transmission. 2 7 3 2 9 8 4 4 2 4 6 1 0 6 2 7 2 2 8 4 9 2 1 4 8 UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 30 / 45 Gossip in one direction 6 8 6 4 0 • All neighbors j ∈ N i of 9 5 1 node i can hear 2 7 3 transmission. 2 7 3 • Can do a simultaneous 3 3 9 8 update x j ( t + 1) = 4 4 4 4 2 γx j ( t ) + (1 − γ ) x i ( t ) . 4 5 5 1 2 2 5 5 2 7 2 2 8 4 9 2 1 4 8 UCSD Sarwate

DANCES Seminar > Trading accuracy for speed 30 / 45 Gossip in one direction 6 8 6 4 0 • All neighbors j ∈ N i of 9 5 1 node i can hear 2 7 3 transmission. 2 7 3 • Can do a simultaneous 3 9 8 update x j ( t + 1) = 4 4 2 γx j ( t ) + (1 − γ ) x i ( t ) . 4 5 1 2 5 2 7 2 2 8 4 9 2 1 4 8 UCSD Sarwate