SLIDE 17 CS535 Big Data 4/7/2020 Week 11-A Sangmi Lee Pallickara http://www.cs.colostate.edu/~cs535 Spring 2020 Colorado State University, page 17
Example use of mrTriplets
A B E D C F 42 23 30 75 19 16 A mapF( )=1 B Source property 42 Target property 23 Message to vertex B V id Property A B 2 C 1 D 1 E F 3 Resulting vertices
Compute the number of older followers for each user in a social network
val graph: Graph[User, Double] def mapUDF(t: Triplet[User, Double]) = if (t.src.age > t.dst.age) 1 else 0 def reduceUDF(a: Int, b: Int): Int = a + b val seniors: Collection[(Id, Int)] = graph.mrTriplets(mapUDF, reduceUDF)
CS535 Big Data | Computer Science | Colorado State University
Implementation of the Pregel abstraction using GraphX
- Initializes the vertex properties
with an additional field to track active vertices
messages are computed using the mrTriplets operator
- Edge-parallel map operation
- Message computation
- Commutative associated
aggregation
def Pregel(g: Graph[V, E], vprog: (Id, V, M) => V, sendMsg: (Triplet) => M, gather: (M, M) => M): Collection[V] = { // Set all vertices as active g = g.mapV((id, v) => (v, halt=false)) // Loop until convergence while (g.vertices.exists(v => !v.halt)) { // Compute the messages val msgs: Collection[(Id, M)] = // Restrict to edges with active source g.subgraph(ePred=(s,d,sP,eP,dP)=>!sP.halt) // Compute messages .mrTriplets(sendMsg, gather) // Receive messages and run vertex program g = g.leftJoinV(msgs).mapV(vprog) } return g.vertices }
CS535 Big Data | Computer Science | Colorado State University
