Minimizing Latency in Fault-Tolerant Distributed Stream Processing - - PowerPoint PPT Presentation

▶
minimizing latency in fault tolerant distributed stream
SMART_READER_LITE
LIVE PREVIEW

Minimizing Latency in Fault-Tolerant Distributed Stream Processing - - PowerPoint PPT Presentation

Aug 08, 2023 100 likes •663 views

Department of Computer Science Institute for Systems Architecture, Systems Engineering Group Minimizing Latency in Fault-Tolerant Distributed Stream Processing Systems Andrey Brito 1 , Christof Fetzer 1 , Pascal Felber 2 1 Technische Universitt

slide-1
SLIDE 1

Minimizing Latency in Fault-Tolerant Distributed Stream Processing Systems

Andrey Brito1, Christof Fetzer1, Pascal Felber2

1 Technische Universität Dresden, Germany 2 Université de Neuchâtel, Switzerland

Department of Computer Science Institute for Systems Architecture, Systems Engineering Group

ICDCS'09, June 23rd, 2009

slide-2
SLIDE 2

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 2 of 55

Goal

Minimize the cost of logging/checkpointing in event stream processing systems Contribution: Usage of an speculation framework based on transactional memory to overlap logging and processing

slide-3
SLIDE 3

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 3 of 55

Motivation (1)

  • Event stream applications

– Directed acyclic graph of operators – Some operators don't keep state

  • Trivially parallelizable

– Some do keep state

  • Not trivially parallelizable

– Sometimes they are order sensitive

  • Need to process events sequentially, maybe even waiting for

the order to be restored

slide-4
SLIDE 4

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 4 of 55

Application example

A0 B0 A2 A1 B1 B3 A4 B2 B5 B7 A5 B6

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

slide-5
SLIDE 5

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 5 of 55

Application example

A0 B0 A2 A1 B1 B3 A4 B2 B5 B7 A5 B6

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

Events based on non-deterministic decision Events are out!

slide-6
SLIDE 6

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 6 of 55

Application example

B5 B7 A5 B6 B1 B3 A4 B2 A0 B0 A2 A1

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

slide-7
SLIDE 7

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 7 of 55

Application example

Restore checkpoint.

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

slide-8
SLIDE 8

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 8 of 55

Application example

Ask upstream node to replay missing ones.

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

slide-9
SLIDE 9

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 9 of 55

Application example

Processing some events again.

B5 B7 A5 B6 B1 B2 B3 A4

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

slide-10
SLIDE 10

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 10 of 55

Application example

B5 B7 A5 B6 B1 B2 B3 A4

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

Incomplete log of non-deterministic decisions no repeatability →

Events reflect different decisions.

What are you talking about?

slide-11
SLIDE 11

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 11 of 55

Motivation (2)

  • Fault-tolerant event stream applications

– Precise recovery – Even if order does not matter, repeatability does – Non-determinism

  • Input order from different streams
  • Non-determinism in processing (multi-threading, time,

random numbers)

– Log or checkpoint before each output

slide-12
SLIDE 12

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 12 of 55

Logging is expensive

slide-13
SLIDE 13

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 13 of 55

My solution

  • Speculate...
  • … to parallelize stateful components
  • … to not have to wait for events
  • … to not have to wait for logging
slide-14
SLIDE 14

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 14 of 55

Outline

  • How the speculation works
  • Logging algorithm
  • Experiments
  • Final remarks
slide-15
SLIDE 15

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 15 of 55

How the speculation works

  • Base: TinySTM

– Some extra features added – But same basic rule: “it appears to be atomic”

  • Goal: track accesses to shared memory

– Instrumentation

  • Reads and writes are intercepted
  • Hold back writes, validate reads until all dependencies

satisfied

slide-16
SLIDE 16

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 16 of 55

Speculative execution: parallelization

Processor 1 Processor 2

NEXT = 9

9 12 11 6 8 7

slide-17
SLIDE 17

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 17 of 55

Speculative execution: parallelization

Processor 1 Processor 2

NEXT = 9 11 9

6 8 7 12 14 13

slide-18
SLIDE 18

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 18 of 55

Speculative execution: parallelization

Processor 1 Processor 2

NEXT = 9 11 9

6 8 7 12 14 13

slide-19
SLIDE 19

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 19 of 55

Speculative execution: parallelization

Processor 1 Processor 2

NEXT = 9 11 9

12 14 13 6 8 7

slide-20
SLIDE 20

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 20 of 55

Speculative execution: parallelization

Processor 1 Processor 2

NEXT = 10 11

7 9 8 12 14 13

slide-21
SLIDE 21

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 21 of 55

Speculative execution: parallelization

Processor 1 Processor 2

NEXT = 10 11

7 9 8 12 14 13

slide-22
SLIDE 22

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 22 of 55

Logging algorithm

  • Operator enqueues all events & decisions
  • N+1 threads for N disks

– One groups requests in a buffers – The others write their buffers to disk

slide-23
SLIDE 23

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 23 of 55

Logging algorithm

Operator

E

slide-24
SLIDE 24

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 24 of 55

Logging algorithm

Operator

E

slide-25
SLIDE 25

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 25 of 55

Logging algorithm

Operator

slide-26
SLIDE 26

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 26 of 55

Logging algorithm

Operator

NDDs

slide-27
SLIDE 27

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 27 of 55

Logging algorithm

Operator

E

slide-28
SLIDE 28

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 28 of 55

Logging algorithm

Operator

E is here waiting.

slide-29
SLIDE 29

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 29 of 55

Logging algorithm

Operator

slide-30
SLIDE 30

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 30 of 55

Logging algorithm

Operator

slide-31
SLIDE 31

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 31 of 55

Logging algorithm

Operator

update(E)

slide-32
SLIDE 32

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 32 of 55

Logging algorithm

Operator

E

slide-33
SLIDE 33

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 33 of 55

Logging algorithm

A0 B0 A2 A1 B1 B3 A4 B2 B5 B7 A5 B6

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

slide-34
SLIDE 34

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 34 of 55

Logging algorithm

A0 B0 A2 A1 B1 B3 A4 B2 B5 B7 A5 B6

STATE

Processor1

Filter n

B8

Output Adapter

Publisher B

Filter n

A6 Publisher A

STATE

Processor2

Events based on non-deterministic decision Events are out!

slide-35
SLIDE 35

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 35 of 55

Logging algorithm

B5 B7 A5 B6

STATE

Processor1

Filter n

B8

Filter 1

A6 Checkpoint/Logging

slide-36
SLIDE 36

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 36 of 55

Logging algorithm

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

slide-37
SLIDE 37

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 37 of 55

Logging algorithm

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

1

slide-38
SLIDE 38

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 38 of 55

Logging algorithm

2

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

1

slide-39
SLIDE 39

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 39 of 55

Logging algorithm

2 3

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

1

slide-40
SLIDE 40

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 40 of 55

Logging algorithm

2 4 3

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

1

slide-41
SLIDE 41

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 41 of 55

Logging algorithm

2 4 3

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

5 1

slide-42
SLIDE 42

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 42 of 55

Logging algorithm

2 4 3 6

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

5 1

slide-43
SLIDE 43

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 43 of 55

Logging algorithm

2 4 3 6

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

5 1 7

slide-44
SLIDE 44

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 44 of 55

Speculative processing + Logging

  • From the original node's viewpoint

– Emit outputs as speculative – When logging requests are acknowledged, emit final

  • The next downstream node

– If speculative event modifies some state, keep track

  • Outputs that consider that part of the state are speculative
  • Speculative status is contagious
slide-45
SLIDE 45

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 45 of 55

Speculation + Logging

2 4 3 6

STATE

Processor1

Filter n Filter 1

Checkpoint/Logging

5 1 3' 7

slide-46
SLIDE 46

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 46 of 55

Experiments

  • Parallelization: benefits & STM's
  • verheads
  • Optimism control
  • Overlapping processing and logging
slide-47
SLIDE 47

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 47 of 55

Speculation costs & speed-ups Hardware: Sun T1000 Speculation creation- commit-disposal

  • verheads.

Few shared-memory accesses. Amdahl's law influence. Hardware: Sun T1000 Speculation creation- commit-disposal

  • verheads.

Few shared-memory accesses. Amdahl's law influence.

slide-48
SLIDE 48

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 48 of 55

Processor 1 Processor 2

NEXT = 9

Controlling optimism

slide-49
SLIDE 49

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 49 of 55

Controlling optimism

Processor 1 Processor 2

NEXT = 9

Processor 3 Processor 4 Processor 5 Processor 6 Processor 7 Processor 8 Processor 1 Processor 2

NEXT = 9

Processor 3 Processor 4 Processor 5 Processor 6 Processor 7 Processor 8

slide-50
SLIDE 50

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 50 of 55

Hardware: SUN T1000 State size varies between 1 and 20. Size=1: concurrent executions will conflict. Size=20: considerable parallelism. Hardware: SUN T1000 State size varies between 1 and 20. Size=1: concurrent executions will conflict. Size=20: considerable parallelism. Controlling optimism

slide-51
SLIDE 51

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 51 of 55

2 components do logging. Non-speculative:

  • nly stable events

are sent. Speculative: send events before logging is finished. 2 components do logging. Non-speculative:

  • nly stable events

are sent. Speculative: send events before logging is finished. Motivation for distributed speculation: logging costs

slide-52
SLIDE 52

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS

X axis: number of components logging. Even in a SAN/WAN, the shapes would look similar. For deterministic components: add “fixed” latency. X axis: number of components logging. Even in a SAN/WAN, the shapes would look similar. For deterministic components: add “fixed” latency. Accumulated gains

slide-53
SLIDE 53

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 53 of 55

Final remarks - Parallelization

  • Parallelization through speculation

– Easier, less bugs

  • Programmer does not need to fight with locks
  • Keeps sequential semantics

– Waste of resources reduced with optimism control

  • Overhead can be much lower with

hardware support for TM (e.g., ASF)

slide-54
SLIDE 54

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 54 of 55

Final remarks - Logging

  • Overlap logging with processing

– Independent of available parallelism – Distributed speculation possible due to less aborts – But do not let speculative results get out of the system

  • In combination with speculative

parallelization may even reduce logging

slide-55
SLIDE 55

ICDCS'09, 23.06.09 Minimizing latency in fault-tolerant DSMS Slide 55 of 55

Thank you! http://streammine.inf.tu-dresden.de http://wwwse.inf.tu-dresden.de