Kineograph Raymond Cheng (University of Washinton, Microsoft - - PowerPoint PPT Presentation

Kineograph

Raymond Cheng (University of Washinton, Microsoft Research) et al.

The challenge

• Social networks (Facebook, Twitter)

generate a lot of information

• Let's analyze it!
• Simple data-mining won't do:

○ too much data ○ constant influx of new data ○ long computation time

A solution

• Process live stream of data (i.e. tweets)
• Aggregate them as a dynamic graph
• Snapshot regularly
• Run distributed graph-mining on snapshots

○ support incremental computation

Kineograph architecture

Data influx (ingest node)

@Alice: @Bob, check out these #kittens ! Transaction T @Alice @Bob #kittens Ingest node node(Alice)

T @Alice -> #kittens @Alice -> @Bob T #kittens -> @Alice

node(kittens)

T @Bob -> @Alice

node(Bob) Progress table after receiving ACKs, report T

Data influx (ingest nodes)

• Parse data and convert them to graph

updates (i.e. sets of edges)

• Send transaction to affected graph nodes

○ at this point, it's just stored in the queue

• Report submitted transaction to global vector

clock

Snapshot creation

Snapshot creation

• Snapshooter initiates the process

○ in practice, every 10 seconds

• Snapshooter copies current progress table

and sends it to graph nodes

• Graph nodes commit transactions up to

times specified in progress table

○ new updates are coming in parallel

Computation overview

• Ran on snapshots
• Algorithm-specific data stored in vertices
• Alternating phases of computation and

propagation

Example: TunkRank

• similar to PageRank:
• vertex value - single real number
• add ranks received from neighbours
• when rank increases by ε, push update to

neighbours

• repeat until stable

Bonus: it's incremental between snapshots!

Example: Shortest Paths

• Bellman-Ford with landmarks

○ landmarks - top vertices from TunkRank ○ calculate only paths passing through landmarks

• vertex data - distances to landmarks
• shorten distances by relaxing edges
• push new distances to neighbours
• repeat until stable

Evaluation

• 17,000 lines of C# code
• 50 Windows servers

○ Intel Xeon (quad-core, 2.8 GHz) with 8 GB RAM

• 100k tweets per second (10 times peak

Twitter rate)

Degree distribution

Graph growth

Decaying can help

Throughput & timeliness

Throughput

Timeliness

Incrementality helps!

Tunk-rank:

Incrementality helps!

Scalability (TunkRank)

Fault tolerance

• Centralized services (progress table &

snapshooter):

○ simple replication ○ Paxos-based consensus

• Ingest nodes:

○ input data is cached until it is committed to a snapshot ○ if ingest node fails, all its transactions are discarded ○ another machine processes data from cache

Replication of graph nodes

• quorum-based: 3 replicas of each node
• Update must be acknowledged by 2 replicas
• If replica misses update, it retrieves it from
• ther replicas
• If replica fails and is replaced, it waits for the

next snapshot and starts working normally from there

• For computation failures: rollback and redo

Incremental expansion

• Ingest nodes - trivial, just add a node
• Storage nodes:

○ maintain more logical partitions than nodes ○ to add nodes, migrate some logical partitions to it ○ splitting logical partitions is possible too ○ new node starts working from the next snapshot - just as in failure recovery

Failure recovery

Thank you!