Balanced Label Propagation for Partitioning Massive Graphs Johan - PowerPoint PPT Presentation

Balanced Label Propagation for Partitioning Massive Graphs Johan Ugander, Cornell University Lars Backstrom, Facebook WSDM ‘ 13

Goal: partition a really big graph

Motivation: distributed computation ▪ Distributing graph calculations ( ‘sharding a graph’) makes traversal/aggregation very expensive. ...

Motivation: distributed computation ▪ Distributing graph calculations ( ‘sharding a graph’) makes traversal/aggregation very expensive. ▪ Naive sharding : ID%n == 0 ID%n == 1 ID%n == 2 ID%n == 3 ... Pr( colocation ) = 1 / n ID%n == n-4 ID%n == n-3 ID%n == n-2 ID%n == n-1

Motivation: distributed computation ▪ Distributing graph calculations ( ‘sharding a graph’) makes traversal/aggregation very expensive. ▪ Naive sharding : ID%n == 0 ID%n == 1 ID%n == 2 ID%n == 3 ... Pr( colocation ) = 1 / n ID%n == n-4 ID%n == n-3 ID%n == n-2 ID%n == n-1 ▪ Intelligent sharding : specify a shard map f() that colocates users with friends f(ID) == 0 f(ID) == 1 f(ID) == 2 f(ID) == 3 ... f(ID) == n-4 f(ID) == n-3 f(ID) == n-2 f(ID) == n-1

Motivation: distributed computation ▪ Distributing graph calculations ( ‘sharding a graph’) makes traversal/aggregation very expensive. ▪ Naive sharding : ID%n == 0 ID%n == 1 ID%n == 2 ID%n == 3 ... Pr( colocation ) = 1 / n ID%n == n-4 ID%n == n-3 ID%n == n-2 ID%n == n-1 ▪ Intelligent sharding : specify a shard map f() that colocates users with friends f(ID) == 0 f(ID) == 1 f(ID) == 2 f(ID) == 3 Oh... and the algorithm ... better be FAST. f(ID) == n-4 f(ID) == n-3 f(ID) == n-2 f(ID) == n-1

Partitioning a really big graph: How? ▪ Garey, Johnson, Stockmeyer 1976 : Minimum bisection is NP-hard ▪ Karypsis and Kumar 1998 : METIS ▪ Feige and Krautgamer 2000 : O(n 1 / 2 log n)-factor approximation

Partitioning a really big graph: How? ▪ Garey, Johnson, Stockmeyer 1976 : Minimum bisection is NP-hard ▪ Karypsis and Kumar 1998 : METIS ▪ Feige and Krautgamer 2000 : O(n 1 / 2 log n)-factor approximation ▪ METIS does not scale to 100 B+ edges. ▪ Need a principled approach, ideally one that can be Hadoop-ified.

Basic idea: Label propagation ▪ Iteratively move nodes to be with the plurality of their neighbors: 5 7 4 3 1 8 6 2 9 - Zhu, Ghahramani. CMU Tech Report 2002. - Raghavan, Albert, Kumara. Phys Rev E 2007. - Barber, Clark. Phys Rev E 2009.

Basic idea: Label propagation ▪ Iteratively move nodes to be with the plurality of their neighbors: 3 3 4 3 4 3 3 4 3 - Zhu, Ghahramani. CMU Tech Report 2002. - Raghavan, Albert, Kumara. Phys Rev E 2007. - Barber, Clark. Phys Rev E 2009.

Basic idea: Label propagation ▪ Iteratively move nodes to be with the plurality of their neighbors: ▪ But how to maintain balance? 3 3 4 3 4 3 3 4 3 - Zhu, Ghahramani. CMU Tech Report 2002. - Raghavan, Albert, Kumara. Phys Rev E 2007. - Barber, Clark. Phys Rev E 2009.

Basic idea: Label propagation ▪ Iteratively move nodes to be with the plurality of their neighbors: ▪ But how to maintain balance? ▪ Label = machine machine V1 V2 V3 machine machine V4 V5 machine machine

Basic idea: Label propagation ▪ Iteratively move nodes to be with the plurality of their neighbors: ▪ But how to maintain balance? ▪ Label = machine machine Modification! Figure out who wants to move. V1 P 13 if people want to move from 1 to 3 . allow only people move, s.t. flow balance. x 13 V2 V3 machine machine V4 V5 machine machine

Basic idea: Label propagation ▪ Iteratively move nodes to be with the plurality of their neighbors: ▪ But how to maintain balance? ▪ Label = machine machine Modification! Figure out who wants to move. V1 P 13 if people want to move from 1 to 3 . Constraints allow only people move, s.t. flow balance. x 13 V2 V3 machine machine V4 V5 machine machine

Basic idea: Label propagation ▪ Iteratively move nodes to be with the plurality of their neighbors: ▪ But how to maintain balance? ▪ Label = machine machine Modification! Figure out who wants to move. V1 P 13 if people want to move from 1 to 3 . Constraints allow only people move, s.t. flow balance. x 13 V2 V3 machine machine How do we solve for ? x ij V4 V5 machine machine

Balance via Linear Program ▪ Greedily maximize edge locality with constraints (max/min sizes S i ,T i ): x ij Solution: number of people to move from i to j. f ij ( x ) Cumulative gain from moving people (ordered by co-location gain) . x f ij ( x ij ) (piecewise-linear concave!) x ij

Balance via Linear Program ▪ Greedily maximize edge locality with constraints (max/min sizes S i ,T i ): x ij Solution: number of people to move from i to j. f ij ( x ) Cumulative gain from moving people (ordered by co-location gain) . x ⇢ S i − | V i | P j 6 = i ( x ij − x ji ) T i − | V i | , ∀ i ≤ ≤ X max f ij ( x ij ) s.t. 0 ≤ x ij ≤ P ij , ∀ i, j X i,j (Maximize the co-location gain of all (Subject to balance) machine swaps) (and the number of people available to move)

Balance via Linear Program ▪ Greedily maximize edge locality with constraints (max/min sizes S i ,T i ): x ij Solution: number of people to move from i to j. f ij ( x ) Cumulative gain from moving people (ordered by co-location gain) . x ⇢ S i − | V i | P j 6 = i ( x ij − x ji ) T i − | V i | , ∀ i ≤ ≤ X max f ij ( x ij ) s.t. 0 ≤ x ij ≤ P ij , ∀ i, j X i,j (Maximize the co-location gain of all (Subject to balance) machine swaps) (and the number of people available to move) ▪ Linear Program : n= 78 machines => 12 k variables / 400 k constraints 8 S i − | V i | P j 6 = i ( x ij − x ji ) T i − | V i | , ∀ i ≤ ≤ < X max s.t. z ij 0 ≤ x ij ≤ P ij , ∀ i, j X,Z − a ijk x ij + z ij : ≤ b ijk , ∀ i, j, k i,j

Balance via Linear Program ▪ Summary of algorithm: ▪ Step 1 : Figure out who wants to move ▪ Step 2 : Solve LP to decide who can move without breaking balance ▪ Step 3 : Move those people

Balance via Linear Program ▪ Summary of algorithm: ▪ Step 1 : Figure out who wants to move ▪ Step 2 : Solve LP to decide who can move without breaking balance ▪ Step 3 : Move those people Step 2 is the contribution compared to ordinary Label Prop.

What about geography?

Initialization using geography ▪ Possible to do much better than random with Facebook, using geography .

Initialization using geography ▪ Possible to do much better than random with Facebook, using geography . ▪ Spatial model of small-world networks (for routing): Kleinberg 2000 ▪ Validation : Liben-Nowell et al. 2005 ; Backstrom, Sun, Marlow 2010 . ▪ Friendship probability as a function of rank-distance: Number of Friends at Different Ranks 0.1 Total connections at ranks (rank+104) -0.95 Best Fit 0.01 0.001 Count 0.0001 1e-05 1e-06 10 100 1000 10000 100000 1e+06 Rank – Backstrom, Sun, Marlow 2010

Initialization using geography ▪ IP data reveals geographic location of users: ▪ 1 , 000 , 000 , 000 users mapped to 700 , 000 cities

Initialization using geography ▪ IP data reveals geographic location of users: ▪ 1 , 000 , 000 , 000 users mapped to 700 , 000 cities ▪ Grow equi-population balloons around population centers.

Initialization using geography ▪ IP data reveals geographic location of users: ▪ 1 , 000 , 000 , 000 users mapped to 700 , 000 cities ▪ Grow equi-population balloons around population centers.

Results: Iteration convergence ▪ Geographic initialization ‘converges’ within 1 step Facebook ( n= 800 m, |E|= 68 b ) 1.0 Geo Random Fraction of edges local * = restrained 0.8 0.6 0.4 0.2 * * 0.0 0 1 2 3 4 5 6 7 8 Iteration

Results: Iteration convergence ▪ Geographic initialization ‘converges’ within 1 step ▪ Random initialization slow to start when: avg degree > # partitions Use ‘ restraint ’: only move big gainers (*s below) Facebook ( n= 800 m, |E|= 68 b ) 1.0 1.0 Geo Geo Fraction of nodes moving Random Random Fraction of edges local * = restrained * = restrained 0.8 0.8 * * 0.6 0.6 0.4 0.4 0.2 0.2 * * 0.0 0.0 0 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 Iteration Iteration

Results: Iteration convergence ▪ Geographic initialization ‘converges’ within 1 step ▪ Random initialization slow to start when: avg degree > # partitions Use ‘ restraint ’: only move big gainers (*s below) ▪ LJ partitioning quality not so dependent on # partitions: BLP exploiting primarily local structure. Facebook ( n= 800 m, |E|= 68 b ) LiveJournal (n= 4 . 8 m, |E|= 42 . 8 m) 1.0 1.0 1.0 1.0 Geo Geo 20 shards 20 shards Fraction of nodes moving Fraction of nodes moving Random Random 40 40 Fraction of edges local Fraction of edges local * = restrained * = restrained 0.8 0.8 0.8 0.8 60 60 80 80 * * 0.6 0.6 0.6 0.6 100 100 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 * * 0.0 0.0 0.0 0.0 0 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 0 2 4 6 8 10 12 1 3 5 7 9 11 Iteration Iteration Iteration Iteration