Naiad: A Timely Dataflow System Derek G. Murray Frank McSherry - - PowerPoint PPT Presentation

▶
naiad a timely dataflow system
SMART_READER_LITE
LIVE PREVIEW

Naiad: A Timely Dataflow System Derek G. Murray Frank McSherry - - PowerPoint PPT Presentation

Jun 28, 2023 213 likes •967 views

Naiad: A Timely Dataflow System Derek G. Murray Frank McSherry Rebecca Isaacs Michael Isard Paul Barham Martn Abadi MSR Silicon Valley Presented by Jesse Mu (jlm95) Background: dataflow programming Batch processing Batch processing

slide-1
SLIDE 1

Naiad: A Timely Dataflow System

Derek G. Murray Frank McSherry Rebecca Isaacs Michael Isard Paul Barham Martín Abadi MSR Silicon Valley Presented by Jesse Mu (jlm95)

slide-2
SLIDE 2

Background: dataflow programming

slide-3
SLIDE 3

Batch processing

slide-4
SLIDE 4

Batch processing

slide-5
SLIDE 5

Batch processing

Count most popular hashtags at a given time

slide-6
SLIDE 6

Batch processing

Count most popular hashtags at a given time ...

slide-7
SLIDE 7

Batch processing

slide-8
SLIDE 8

Batch processing

Must wait for all inputs to be completed (= latency)

slide-9
SLIDE 9

Stream processing (asynchronous)

slide-10
SLIDE 10

Stream processing (asynchronous)

Pick out key words/mentions/relevant topics

slide-11
SLIDE 11

Stream processing (asynchronous)

Real-time access Pick out key words/mentions/relevant topics

slide-12
SLIDE 12

Background: types of data processing systems

  • Batch processing (e.g. Pregel, CIEL)

○ High throughput, aggregate summaries of data ○ Waiting for batches introduces latency

  • Stream processing (e.g. Storm, MillWheel)

○ Low-latency, near-realtime access to results ○ No synchronization/aggregate computation

  • Iterative (graph-centric) computation

○ e.g. network data, ML

slide-13
SLIDE 13

Background: types of data processing systems

Timely Dataflow

One-size-fits-all

  • Batch processing (e.g. Pregel, CIEL)

○ High throughput, aggregate summaries of data ○ Waiting for batches introduces latency

  • Stream processing (e.g. Storm, MillWheel)

○ Low-latency, near-realtime access to results ○ No synchronization/aggregate computation

  • Iterative (graph-centric) computation

○ e.g. network data, ML

slide-14
SLIDE 14

Background: types of data processing systems

Timely Dataflow

One-size-fits-all

slide-15
SLIDE 15

Contributions

1. Timely dataflow, a dataflow computing model which supports batch, stream, and graph-centric iterative processing

a. Supports common high-level programming interfaces (e.g. LINQ)

2. Naiad, a high-performance distributed implementation of the model

a. Faster than SOTA batch/streaming frameworks

slide-16
SLIDE 16

Timely Dataflow supports Batch and Stream

Async event-based model Nodes are always active. Send and receive messages via A.SendBy(edge, message, time) B.OnRecv(edge, message, time) Request and operate on notifications for batches C.NotifyAt(time) C.OnNotify(time)

A B C

slide-17
SLIDE 17

Timely Dataflow supports Batch and Stream

Async event-based model Nodes are always active. Send and receive messages via A.SendBy(edge, message, time) B.OnRecv(edge, message, time) Request and operate on notifications for batches C.NotifyAt(time) C.OnNotify(time)

A B C

Stream processing Batch processing

slide-18
SLIDE 18

A B

realtime output batched output

a_out rt_out b_out

slide-19
SLIDE 19

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

slide-20
SLIDE 20

Pass through even numbers

  • nly

A

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

slide-21
SLIDE 21

Pass through even numbers

  • nly

A B

Pass through all numbers; compute min of each time

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

slide-22
SLIDE 22

function OnRecv(input_edge, msg, time) { if (msg % 2 == 0) this.SendBy(a_out, msg, time)}

Pass through even numbers

  • nly

A B

Pass through all numbers; compute min of each time

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

slide-23
SLIDE 23

function OnRecv(input_edge, msg, time) { if (msg % 2 == 0) this.SendBy(a_out, msg, time)}

Pass through even numbers

  • nly

A B

Pass through all numbers; compute min of each time

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

state = {} // times -> running mins

slide-24
SLIDE 24

function OnRecv(input_edge, msg, time) { if (msg % 2 == 0) this.SendBy(a_out, msg, time)}

Pass through even numbers

  • nly

A B

Pass through all numbers; compute min of each time

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

state = {} // times -> running mins function OnRecv(input_edge, msg, time) {

slide-25
SLIDE 25

function OnRecv(input_edge, msg, time) { if (msg % 2 == 0) this.SendBy(a_out, msg, time)}

Pass through even numbers

  • nly

A B

Pass through all numbers; compute min of each time

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

state = {} // times -> running mins function OnRecv(input_edge, msg, time) {

this.SendBy(rt_out, msg, time)

slide-26
SLIDE 26

function OnRecv(input_edge, msg, time) { if (msg % 2 == 0) this.SendBy(a_out, msg, time)}

Pass through even numbers

  • nly

A B

Pass through all numbers; compute min of each time

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

state = {} // times -> running mins function OnRecv(input_edge, msg, time) {

this.SendBy(rt_out, msg, time) // Streaming if (time not in state) // New time state[time] = msg this.NotifyAt(time)

slide-27
SLIDE 27

function OnRecv(input_edge, msg, time) { if (msg % 2 == 0) this.SendBy(a_out, msg, time)}

Pass through even numbers

  • nly

A B

Pass through all numbers; compute min of each time

state = {} // times -> running mins function OnRecv(input_edge, msg, time) {

this.SendBy(rt_out, msg, time) // Streaming if (time not in state) // New time state[time] = msg this.NotifyAt(time) if (msg < state[time]) // New min state[time] = msg

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

slide-28
SLIDE 28

state = {} // times -> running mins function OnRecv(input_edge, msg, time) {

this.SendBy(rt_out, msg, time) // Streaming if (time not in state) // New time state[time] = msg this.NotifyAt(time) if (msg < state[time]) // New min state[time] = msg

function OnRecv(input_edge, msg, time) { if (msg % 2 == 0) this.SendBy(a_out, msg, time)}

Pass through even numbers

  • nly

A B

realtime output batched output

A B

Pass through all numbers; compute min of each time

a_out rt_out b_out

function OnNotify(time) { this.SendBy(batch_out, state[time], time)}

Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

slide-29
SLIDE 29

state = {} // times -> running mins function OnRecv(input_edge, msg, time) {

this.SendBy(rt_out, msg, time) // Streaming if (time not in state) // New time state[time] = msg this.NotifyAt(time) if (msg < state[time]) // New min state[time] = msg

function OnRecv(input_edge, msg, time) { if (msg % 2 == 0) this.SendBy(a_out, msg, time)}

Pass through even numbers

  • nly

A B

realtime output batched output

A B

Pass through all numbers; compute min of each time

a_out rt_out b_out

function OnNotify(time) { this.SendBy(batch_out, state[time], time)}

Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ... Node B, you’ve seen all messages for time 1

slide-30
SLIDE 30

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ...

slide-31
SLIDE 31

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ... All messages for time 1 delivered

slide-32
SLIDE 32

A B

realtime output batched output

a_out rt_out b_out Input time numbers 1 9, 3, 2, 5, ... 2 3, 2, 7, 12, ... ... All messages for time 1 delivered

???

slide-33
SLIDE 33

Progress tracking

slide-34
SLIDE 34

Progress tracking

SendBy(_, _, 1)

slide-35
SLIDE 35

Progress tracking

NotifyAt(1) SendBy(_, _, 1)

slide-36
SLIDE 36

Progress tracking

NotifyAt(1) SendBy(_, _, 1) SendBy(_, _, (1, 1))

slide-37
SLIDE 37

Progress tracking

NotifyAt(1) SendBy(_, _, 1) SendBy(_, _, (1, 1)) SendBy(_, _, (1, 2)) NotifyAt((1, 2))

slide-38
SLIDE 38

Progress tracking

NotifyAt(1) SendBy(_, _, 1) SendBy(_, _, (1, 1)) SendBy(_, _, (1, 2)) NotifyAt((1, 2))

slide-39
SLIDE 39

Sort by could-result-in order

NotifyAt(1) SendBy(_, _, 1) SendBy(_, _, (1, 1)) SendBy(_, _, (1, 2)) NotifyAt((1, 2))

slide-40
SLIDE 40

Sort by could-result-in order

NotifyAt(1) SendBy(_, _, 1) SendBy(_, _, (1, 1)) SendBy(_, _, (1, 2)) NotifyAt((1, 2))

slide-41
SLIDE 41

Sort by could-result-in order

NotifyAt(1) SendBy(_, _, (1, 1)) SendBy(_, _, (1, 2)) NotifyAt((1, 2))

slide-42
SLIDE 42

Sort by could-result-in order

NotifyAt(1) SendBy(_, _, (1, 1)) SendBy(_, _, (1, 2)) NotifyAt((1, 2))

slide-43
SLIDE 43

Sort by could-result-in order

NotifyAt(1) SendBy(_, _, (1, 2)) NotifyAt((1, 2))

slide-44
SLIDE 44

Sort by could-result-in order

NotifyAt(1) SendBy(_, _, (1, 2)) NotifyAt((1, 2))

slide-45
SLIDE 45

Sort by could-result-in order

NotifyAt(1) NotifyAt((1, 2))

slide-46
SLIDE 46

Sort by could-result-in order

NotifyAt(1) NotifyAt((1, 2))

slide-47
SLIDE 47

Sort by could-result-in order

NotifyAt(1) NotifyAt((1, 2)) Send notification!

slide-48
SLIDE 48

Sort by could-result-in order

NotifyAt(1)

slide-49
SLIDE 49

Sort by could-result-in order

NotifyAt(1) Send notification!

slide-50
SLIDE 50

Sort by could-result-in order

slide-51
SLIDE 51

Sort by could-result-in order

...a notification can be delivered only when no possible predecessors of a timestamp exist

slide-52
SLIDE 52

Sort by could-result-in order

...a notification can be delivered only when no possible predecessors of a timestamp exist (based on timestamps + graph structure)

slide-53
SLIDE 53

Low vs High Level Interfaces

slide-54
SLIDE 54

Low vs High Level Interfaces

SendBy(edge, message, time) OnRecv(edge, message, time) NotifyAt(time) OnNotify(time)

Event-based system

slide-55
SLIDE 55

Low vs High Level Interfaces

SendBy(edge, message, time) OnRecv(edge, message, time) NotifyAt(time) OnNotify(time)

// 1a. Define input stages for the dataflow. var input = controller.NewInput<string>(); // 1b. Define the timely dataflow graph. // Here, we use LINQ to implement MapReduce. var result = input.SelectMany(y => map(y)) .GroupBy(y => key(y), (k, vs) => reduce(k, vs)); // 1c. Define output callbacks for each epoch result.Subscribe(result => { ... }); // 2. Supply input data to the query. input.OnNext(/* 1st epoch data */); input.OnNext(/* 2nd epoch data */); input.OnCompleted();

Event-based system Common dataflow interfaces (LINQ, Pregel)

slide-56
SLIDE 56

Contributions

1. Timely dataflow, a dataflow computing model which supports batch, stream, and graph-centric iterative processing

a. Supports common high-level programming interfaces (e.g. LINQ)

2. Naiad, a high-performance distributed implementation of the model

a. Faster than SOTA batch/streaming frameworks

slide-57
SLIDE 57

Implementation: Naiad

slide-58
SLIDE 58

Implementation: Naiad

slide-59
SLIDE 59

Distributed Progress Tracking

slide-60
SLIDE 60

Each node has its own local progress tracker, must be conservative Updates other nodes over network as events finish

Distributed Progress Tracking

slide-61
SLIDE 61

Optimizations

slide-62
SLIDE 62

Reduce small delays micro-stragglers Tweak TCP configuration GC less often Reduce backoff time to 1ms after concurrent access to shared memory

Optimizations

slide-63
SLIDE 63

Fault Tolerance

slide-64
SLIDE 64

Since vertices have dynamic state, one failure -> all nodes have to reset from checkpoint System-wide synchronized checkpoints Tradeoff between how often to log checkpoints and performance

Fault Tolerance

slide-65
SLIDE 65

Evaluation

slide-66
SLIDE 66

Evaluation

slide-67
SLIDE 67

Contributions

1. Timely dataflow, a dataflow computing model which supports batch, stream, and graph-centric iterative processing

a. Supports common high-level programming interfaces (e.g. LINQ)

2. Naiad, a high-performance distributed implementation of the model

a. Faster than SOTA batch/streaming frameworks

slide-68
SLIDE 68

My opinion

slide-69
SLIDE 69

My opinion

  • Computational model is theoretically sound

○ Iterative computation without modifying graph in e.g. CIEL (which has overhead)

  • Evaluation good too, though dramatic speedups likely better than real-world

applications

  • Fine-grained control over logging for fault tolerance/throughput tradeoff

seems annoying

  • But…
slide-70
SLIDE 70

What problem does Naiad solve?

slide-71
SLIDE 71

What problem does Naiad solve?

“While it might be possible to assemble the application in Figure 1 by combining multiple existing systems, applications built on a single platform are typically more efficient, succinct, and maintainable.”

slide-72
SLIDE 72

What problem does Naiad solve?

“While it might be possible to assemble the application in Figure 1 by combining multiple existing systems, applications built on a single platform are typically more efficient, succinct, and maintainable.”

slide-73
SLIDE 73

My opinion

  • Computational model is theoretically sound

○ Iterative computation without modifying graph in e.g. CIEL (which has overhead)

  • Evaluation good too, though dramatic speedups likely better than real-world

applications

  • Fine-grained control over logging for fault tolerance/throughput tradeoff

seems annoying

  • But…

○ For all but especially complex systems requiring graph + stream + batch, existing systems probably work just fine + have better infrastructure

slide-74
SLIDE 74

Naiad: A Timely Dataflow System

Derek G. Murray Frank McSherry Rebecca Isaacs Michael Isard Paul Barham Martín Abadi MSR Silicon Valley Presented by Jesse Mu (jlm95)