Kineograph Raymond Cheng (University of Washinton, Microsoft - PowerPoint PPT Presentation

Kineograph Raymond Cheng (University of Washinton, Microsoft Research) et al.

The challenge ● Social networks (Facebook, Twitter) generate a lot of information ● Let's analyze it! ● Simple data-mining won't do: ○ too much data ○ constant influx of new data ○ long computation time

A solution ● Process live stream of data (i.e. tweets) ● Aggregate them as a dynamic graph ● Snapshot regularly ● Run distributed graph-mining on snapshots ○ support incremental computation

Kineograph architecture

Data influx (ingest node) @Alice: @Bob , check out these #kittens ! Ingest node node(Alice) T @Alice @Alice -> #kittens @Alice -> @Bob Transaction T @Bob #kittens T T @Bob -> @Alice #kittens -> @Alice node(Bob) node(kittens) after receiving ACKs, report T Progress table

Data influx (ingest nodes) ● Parse data and convert them to graph updates (i.e. sets of edges) ● Send transaction to affected graph nodes ○ at this point, it's just stored in the queue ● Report submitted transaction to global vector clock

Snapshot creation

Snapshot creation ● Snapshooter initiates the process ○ in practice, every 10 seconds ● Snapshooter copies current progress table and sends it to graph nodes ● Graph nodes commit transactions up to times specified in progress table ○ new updates are coming in parallel

Computation overview ● Ran on snapshots ● Algorithm-specific data stored in vertices ● Alternating phases of computation and propagation

Example: TunkRank ● similar to PageRank: ● vertex value - single real number ● add ranks received from neighbours ● when rank increases by ε, push update to neighbours ● repeat until stable Bonus: it's incremental between snapshots!

Example: Shortest Paths ● Bellman-Ford with landmarks ○ landmarks - top vertices from TunkRank ○ calculate only paths passing through landmarks ● vertex data - distances to landmarks ● shorten distances by relaxing edges ● push new distances to neighbours ● repeat until stable

Evaluation ● 17,000 lines of C# code ● 50 Windows servers ○ Intel Xeon (quad-core, 2.8 GHz) with 8 GB RAM ● 100k tweets per second (10 times peak Twitter rate)

Degree distribution

Graph growth Decaying can help

Throughput & timeliness

Throughput

Timeliness

Incrementality helps! Tunk-rank:

Incrementality helps!

Scalability (TunkRank)

Fault tolerance ● Centralized services (progress table & snapshooter): ○ simple replication ○ Paxos-based consensus ● Ingest nodes: ○ input data is cached until it is committed to a snapshot ○ if ingest node fails, all its transactions are discarded ○ another machine processes data from cache

Replication of graph nodes ● quorum-based: 3 replicas of each node ● Update must be acknowledged by 2 replicas ● If replica misses update, it retrieves it from other replicas ● If replica fails and is replaced, it waits for the next snapshot and starts working normally from there ● For computation failures: rollback and redo

Incremental expansion ● Ingest nodes - trivial, just add a node ● Storage nodes: ○ maintain more logical partitions than nodes ○ to add nodes, migrate some logical partitions to it ○ splitting logical partitions is possible too ○ new node starts working from the next snapshot - just as in failure recovery

Failure recovery

Thank you!