Applying the Saga Pattern Caitie McCafffrey Caitie McCaffrey - - PowerPoint PPT Presentation

▶
applying the saga pattern
SMART_READER_LITE
LIVE PREVIEW

Applying the Saga Pattern Caitie McCafffrey Caitie McCaffrey - - PowerPoint PPT Presentation

Jun 19, 2023 606 likes •1.53k views

Applying the Saga Pattern Caitie McCafffrey Caitie McCaffrey Distributed Systems Engineer @Caitie CaitieM.com Why Sagas? Sagas Paper Distributed Sagas Sagas in Halo 4 Systems Used to Be Simple Serializability & ACID

slide-1
SLIDE 1

Applying the Saga Pattern

Caitie McCafffrey

slide-2
SLIDE 2

Caitie McCaffrey

Distributed Systems Engineer @Caitie CaitieM.com

slide-3
SLIDE 3
slide-4
SLIDE 4
  • Why Sagas?
  • Sagas Paper
  • Distributed Sagas
  • Sagas in Halo 4
slide-5
SLIDE 5

Systems Used to Be Simple

slide-6
SLIDE 6

Serializability & ACID

slide-7
SLIDE 7

SOA/Microservices

slide-8
SLIDE 8

Two Phase Commit

slide-9
SLIDE 9

2PC: Prepare Phase

slide-10
SLIDE 10

2PC: Commit Phase

slide-11
SLIDE 11

2PC Doesn’t Scale

  • O(n^2) Messages
  • Coordinator: Single Point of Failure
  • Reduced Throughput
slide-12
SLIDE 12

Spanner

Google’s Globally-Distributed Database Corbett et. al.

slide-13
SLIDE 13

–Corbett et al. “Spanner is Google’s scalable, multi-version,

globally distributed, and synchronously-

replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed

transactions.”

slide-14
SLIDE 14

“The key enabler of these properties is a new TrueTime API and its implementation…using multiple modern clock references (GPS and atomic clocks).”

–Corbett et al.

slide-15
SLIDE 15

Spanner is Expensive & Proprietary

  • Custom Hardware & Data-Centers
  • Synchronization Not Solved
slide-16
SLIDE 16

Distributed Transactions are Hard & Expensive

slide-17
SLIDE 17

Can We Do Better?

slide-18
SLIDE 18

Can We Do Better?

slide-19
SLIDE 19

Sagas

Hector Garcia-Molina, Kenneth Salem Princeton University 1987

slide-20
SLIDE 20

Sagas are Long Lived Transactions

slide-21
SLIDE 21

“A Saga is a Long Lived Transaction that can be written as a sequence of transactions that can be interleaved. All transactions in the sequence complete successfully or compensating transactions are ran to amend a partial execution.”

slide-22
SLIDE 22

A Saga is a Collection of Sub-Transactions

T1, T2 … Tn

slide-23
SLIDE 23

Each Sub-Transaction has a Compensating Transaction

C1, C2 … Cn

slide-24
SLIDE 24

Cn Semantically Undoes Tn

slide-25
SLIDE 25

Saga Guarantee

Either

  • T1, T2 … Tn or
  • T1, T2 … Tj, Cj, … C2, C1
slide-26
SLIDE 26

Trade-Off: Atomicity for Availability

slide-27
SLIDE 27

Sagas are a Failure Management Pattern

slide-28
SLIDE 28

Large Single Transaction

slide-29
SLIDE 29
  • Book Hotel (T1)
  • Book Car (T2)
  • Book Flight (T3)
  • Cancel Hotel (C1)
  • Cancel Car (C2)
  • Cancel Flight (C3)

Sagas

slide-30
SLIDE 30

Saga Execution Coordinator (SEC)

slide-31
SLIDE 31

Saga Log

  • Begin Saga
  • End Saga
  • Abort Saga
  • Begin Ti
  • End Ti
  • Begin Ci
  • End Ci
slide-32
SLIDE 32

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) End Book Car Rental (T2) Start Book Flight (T3) End Book Flight (T3) End Saga

Successful Saga

slide-33
SLIDE 33

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) End Book Car Rental (T2) Start Book Flight (T3) End Book Flight (T3) End Saga

Successful Saga

slide-34
SLIDE 34

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) End Book Car Rental (T2) Start Book Flight (T3) End Book Flight (T3) End Saga

Successful Saga

slide-35
SLIDE 35

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) End Book Car Rental (T2) Start Book Flight (T3) End Book Flight (T3) End Saga

Successful Saga

slide-36
SLIDE 36

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) End Book Car Rental (T2) Start Book Flight (T3) End Book Flight (T3) End Saga

Successful Saga

slide-37
SLIDE 37

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) End Book Car Rental (T2) Start Book Flight (T3) End Book Flight (T3) End Saga

Successful Saga

slide-38
SLIDE 38

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) End Book Car Rental (T2) Start Book Flight (T3) End Book Flight (T3) End Saga

Successful Saga

slide-39
SLIDE 39

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) End Book Car Rental (T2) Start Book Flight (T3) End Book Flight (T3) End Saga

Successful Saga

slide-40
SLIDE 40

Unsuccessful Saga

Backwards Recovery

slide-41
SLIDE 41

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-42
SLIDE 42

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-43
SLIDE 43

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-44
SLIDE 44

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-45
SLIDE 45

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-46
SLIDE 46

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-47
SLIDE 47

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-48
SLIDE 48

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-49
SLIDE 49

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-50
SLIDE 50

Begin Saga Start Book Hotel (T1) End Book Hotel (T1) Start Book Car Rental (T2) Abort Saga Start Compensate Car Rental (C2) End Compensate Car Rental (C2) Start Compensate Book Hotel (C1) End Compensate Book Hotel (C1) End Saga

Unsuccessful Saga

slide-51
SLIDE 51

–Molina et. al

“Due to space limitations, we only discuss Sagas in a centralized System, although clearly they can be implemented in a distributed database system.”

Sagas in Distributed Systems

slide-52
SLIDE 52

SOA/Microservices

slide-53
SLIDE 53

\

  • Book Hotel (T1)
  • Book Car (T2)
  • Book Flight (T3)
  • Cancel Hotel (C1)
  • Cancel Car (C2)
  • Cancel Flight (C3)

Requests instead of Transactions

slide-54
SLIDE 54

A Distributed Saga is a Collection of Sub-Requests

Each Sub-Request has a Compensating Request

T1, T2 … Tn C1, C2 … Cn

slide-55
SLIDE 55

Begin Saga Start Book Hotel Request (T1) End Book Hotel Request (T1) Start Book Car Rental Request (T2) End Book Car Rental Request (T2) Start Book Flight Request (T3) End Book Flight Request (T3) End Saga

Successful Distributed Saga

slide-56
SLIDE 56

Saga Log

Durable & Distributed

slide-57
SLIDE 57

Saga Execution Coordinator (SEC)

  • Interprets & Writes to Saga Log
  • Applies Saga Sub-Requests
  • Applies Saga Compensating Requests

when Necessary

slide-58
SLIDE 58
slide-59
SLIDE 59
slide-60
SLIDE 60
slide-61
SLIDE 61
slide-62
SLIDE 62
slide-63
SLIDE 63
slide-64
SLIDE 64
slide-65
SLIDE 65
slide-66
SLIDE 66
slide-67
SLIDE 67

Apply Compensating Requests

  • Aborted Saga Response
  • Start Request Fails
  • SEC Crashes (non-safe state)
slide-68
SLIDE 68
slide-69
SLIDE 69
slide-70
SLIDE 70
slide-71
SLIDE 71

What Happens when Compensating Requests Fail?

slide-72
SLIDE 72

Compensating Requests Must Be Idempotent

slide-73
SLIDE 73

What Happens when SEC Fails?

slide-74
SLIDE 74

Safe States

  • All Executed Sub-Requests are Complete

(Start Ti & End Ti both logged)

  • Saga has been Aborted, Proceed with

Compensating Transactions

slide-75
SLIDE 75

Un-Safe State

  • Start Ti logged, no End Ti logged

Abort Saga Start Compensating Requests

slide-76
SLIDE 76

Request Messaging Semantics

  • Sub-Requests (Ti): At Most Once
  • Compensating Requests (Ci): At Least Once
slide-77
SLIDE 77

Distributed Saga Guarantee

Either

  • T1, T2 … Tn or
  • T1, T2 … Tj, Cj, … C2, C1
slide-78
SLIDE 78

Distributed Sagas

  • Distributed/Durable Saga Log
  • SEC Process
  • Compensating Requests: Idempotent
slide-79
SLIDE 79
slide-80
SLIDE 80

Halo Statistics

slide-81
SLIDE 81

Halo Statistics Service

slide-82
SLIDE 82

Halo Statistics Service with Sagas

slide-83
SLIDE 83

Forward-Recovery

slide-84
SLIDE 84

Store Stats Player One Store Stats Player Two Store Stats Player Three Store Stats Player Four Game Grain / SEC

slide-85
SLIDE 85

Forward Recovery

Sub-Requests Must Also Be Idempotent

slide-86
SLIDE 86

Store Stats Player One Store Stats Player Two Store Stats Player Three Store Stats Player Four Game Grain / SEC

slide-87
SLIDE 87

Store Stats Player One Store Stats Player Two Store Stats Player Three Store Stats Player Four Game Grain / SEC

slide-88
SLIDE 88

Sagas

  • Long Lived / Distributed Transactions
  • Trade Atomicity for Availability
  • Failure Management Pattern
slide-89
SLIDE 89

Thank You

  • @pbailis
  • @randommood
  • @tsantero
  • @aphyr
  • @jmhodges
  • @clemnsv
slide-90
SLIDE 90

Questions?

Please remember to evaluate via the GOTO Guide App

@Caitie