applying the saga pattern
play

Applying the Saga Pattern Caitie McCafffrey Caitie McCaffrey - PowerPoint PPT Presentation

Jun 19, 2023 •606 likes •1.53k views

Applying the Saga Pattern Caitie McCafffrey Caitie McCaffrey Distributed Systems Engineer @Caitie CaitieM.com Why Sagas? Sagas Paper Distributed Sagas Sagas in Halo 4 Systems Used to Be Simple Serializability & ACID

  1. Applying the Saga Pattern Caitie McCafffrey

  2. Caitie McCaffrey � Distributed Systems Engineer @Caitie CaitieM.com

  4. • Why Sagas? • Sagas Paper • Distributed Sagas • Sagas in Halo 4

  5. Systems Used to Be Simple

  6. Serializability & ACID

  7. SOA/Microservices

  8. Two Phase Commit

  9. 2PC: Prepare Phase

  10. 2PC: Commit Phase

  11. 2PC Doesn’t Scale • O(n^2) Messages • Coordinator: Single Point of Failure • Reduced Throughput

  12. Spanner Google’s Globally-Distributed Database Corbett et. al.

  13. “Spanner is Google’s scalable, multi-version, globally distributed , and synchronously- replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed transactions .” –Corbett et al.

  14. “The key enabler of these properties is a new TrueTime API and its implementation…using multiple modern clock references ( GPS and atomic clocks ).” –Corbett et al.

  15. Spanner is Expensive & Proprietary • Custom Hardware & Data-Centers • Synchronization Not Solved

  16. Distributed Transactions are Hard & Expensive

  17. Can We Do Better?

  18. Can We Do Better?

  19. Sagas Hector Garcia-Molina, Kenneth Salem Princeton University 1987

  20. Sagas are Long Lived Transactions

  21. “A Saga is a Long Lived Transaction that can be written as a sequence of transactions that can be interleaved. All transactions in the sequence complete successfully or compensating transactions are ran to amend a partial execution.”

  22. A Saga is a Collection of Sub-Transactions T 1 , T 2 … T n

  23. Each Sub-Transaction has a Compensating Transaction C 1 , C 2 … C n

  24. C n Semantically Undoes T n

  25. Saga Guarantee Either • T 1 , T 2 … T n or • T 1 , T 2 … T j, C j , … C 2 , C 1

  26. Trade-Off: Atomicity for Availability

  27. Sagas are a Failure Management Pattern

  28. Large Single Transaction

  29. Sagas • Book Hotel (T 1 ) • Cancel Hotel (C 1 ) • Book Car (T 2 ) • Cancel Car (C 2 ) • Book Flight (T 3 ) • Cancel Flight (C 3 )

  30. Saga Execution Coordinator (SEC)

  31. Saga Log • Begin T i • Begin Saga • End T i • End Saga • Begin C i • Abort Saga • End C i

  32. Successful Saga Begin Saga � Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  33. Successful Saga Begin Saga Start Book Hotel (T 1 ) � End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  34. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) � Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  35. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) � End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  36. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) � Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  37. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) � End Book Flight (T 3 ) End Saga

  38. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) � End Saga

  39. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  40. Unsuccessful Saga Backwards Recovery

  41. Unsuccessful Saga Begin Saga � Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  42. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) � End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  43. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) � Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  44. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) � Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  45. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga � Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  46. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) � End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  47. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) � Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  48. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) � End Compensate Book Hotel (C 1 ) End Saga

  49. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) � End Saga

  50. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  51. Sagas in Distributed Systems “Due to space limitations, we only discuss Sagas in a centralized System, although clearly they can be implemented in a distributed database system.” –Molina et. al

  52. SOA/Microservices

  53. Requests instead of Transactions \ • Book Hotel (T 1 ) • Cancel Hotel (C 1 ) • Book Car (T 2 ) • Cancel Car (C 2 ) • Book Flight (T 3 ) • Cancel Flight (C 3 )

  54. A Distributed Saga is a Collection of Sub-Requests T 1 , T 2 … T n Each Sub-Request has a Compensating Request C 1 , C 2 … C n

  55. Successful Distributed Saga Begin Saga Start Book Hotel Request (T 1 ) End Book Hotel Request (T 1 ) Start Book Car Rental Request (T 2 ) End Book Car Rental Request (T 2 ) Start Book Flight Request (T 3 ) End Book Flight Request (T 3 ) End Saga

  56. Saga Log Durable & Distributed

  57. Saga Execution Coordinator (SEC) • Interprets & Writes to Saga Log • Applies Saga Sub-Requests • Applies Saga Compensating Requests when Necessary

  67. Apply Compensating Requests • Aborted Saga Response • Start Request Fails • SEC Crashes (non-safe state)

  71. What Happens when Compensating Requests Fail?

  72. Compensating Requests Must Be Idempotent

  73. What Happens when SEC Fails?

  74. Safe States • All Executed Sub-Requests are Complete (Start T i & End T i both logged) • Saga has been Aborted, Proceed with Compensating Transactions

  75. Un-Safe State • Start T i logged, no End T i logged Abort Saga Start Compensating Requests

  76. Request Messaging Semantics • Sub-Requests (T i ): At Most Once • Compensating Requests (C i ): At Least Once

  77. Distributed Saga Guarantee Either • T 1 , T 2 … T n or • T 1 , T 2 … T j, C j , … C 2 , C 1

  78. Distributed Sagas • Distributed/Durable Saga Log • SEC Process • Compensating Requests: Idempotent

  80. Halo Statistics

  81. Halo Statistics Service

  82. Halo Statistics Service with Sagas

  83. Forward-Recovery

  84. Game Grain / SEC Store Stats Store Stats Store Stats Store Stats Player One Player Two Player Three Player Four

  85. Forward Recovery Sub-Requests Must Also Be Idempotent

  86. Game Grain / SEC Store Stats Store Stats Store Stats Store Stats Player One Player Two Player Three Player Four

  87. Game Grain / SEC Store Stats Store Stats Store Stats Store Stats Player One Player Two Player Three Player Four

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Introduction to the SAGA API Outline SAGA Standardization API Structure and Scope (C++)

Introduction to the SAGA API Outline SAGA Standardization API Structure and Scope (C++)

Introduction to the SAGA API Outline SAGA Standardization API Structure and Scope (C++) API Walkthrough SAGA SoftwareComponents Command Line Tools Python API bindings C++ API bindings [ Java API bindings ] SAGA:

1.26k views • 87 slides
Interoperabilty: The SAGA Approach and Experience Shantenu Jha, Andre Merzky, Ole Weidner &

Interoperabilty: The SAGA Approach and Experience Shantenu Jha, Andre Merzky, Ole Weidner &

Interoperabilty: The SAGA Approach and Experience Shantenu Jha, Andre Merzky, Ole Weidner & * Collaborators http://saga.cct.lsu.edu Outline Introduction to SAGA: Why SAGA for Interoperability? Use of a standards-based approach for

390 views • 27 slides
C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS

C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS

C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS @ardalis | ardalis.com | weeklydevtips.com t h s What problems does proxy solve? Objectives How is the proxy pattern structured? Apply the

779 views • 23 slides
Applying Architectural Patterns for the Cloud: Lessons Learned During Pattern Mining and

Applying Architectural Patterns for the Cloud: Lessons Learned During Pattern Mining and

Applying Architectural Patterns for the Cloud: Lessons Learned During Pattern Mining and Application Ralph Retter (Daimler TSS GmbH) ralph.retter@daimler.com Christoph Fehling (University of Stuttgart, Germany) Problem We need to do Cloud

282 views • 25 slides
Today you are going to: Watch a video Learn what a mini saga is and how to write one

Today you are going to: Watch a video Learn what a mini saga is and how to write one

Today you are going to: Watch a video Learn what a mini saga is and how to write one Generate ideas to write about Create a character, setting and plot Write a mini saga for Young Writers Ridiculous Writers mini saga

528 views • 13 slides
A Pattern Pattern Taxonomy Creational Behavioral Structural Pattern Pattern Pattern

A Pattern Pattern Taxonomy Creational Behavioral Structural Pattern Pattern Pattern

Podcast Ch08-15 Title : Template Design Pattern Description : Generic class diagram; examples; template vs. inheritance Participants : Barry Kurtz (instructor); Brandon Winters, Sara Hyde, Cheng Vue, Dan Baehr (students) Textbook

315 views • 3 slides
Projet Raw-data La Saga le dveloppement de cet t Le Rsultat Readers Analyst &

Projet Raw-data La Saga le dveloppement de cet t Le Rsultat Readers Analyst &

Projet Raw-data La Saga le dveloppement de cet t Le Rsultat Readers Analyst & Xcalibur 15/12/2008 La Saga La source : Les API des constructeurs L'interface : OLE/COM Le software : Dveloppement d'un reader

522 views • 26 slides
Building and Installing All material from this tutorial can be found at:

Building and Installing All material from this tutorial can be found at:

A Brief Introduction to SAGA Building and Installing All material from this tutorial can be found at: https://svn.cct.lsu.edu/repos/saga-projects/tutorial/EGI-2011 General Information and Documentation General information http://saga.

220 views • 9 slides
Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern

Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern

CSC209 Fall 2001 What is AWK? Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern matches, do something Many details handled automatically very easy to one off (write and throw away)

673 views • 12 slides
saga PRODUCTION UNIT RAW MATERIAL PRODUCT APPLICATIONS PRODUCTION PROCESS BALES

saga PRODUCTION UNIT RAW MATERIAL PRODUCT APPLICATIONS PRODUCTION PROCESS BALES

SICEM SAGA CMP FOR PAPER INDUSTRY www. sicemsaga.com sicem saga PRODUCTION UNIT RAW MATERIAL PRODUCT APPLICATIONS PRODUCTION PROCESS BALES AND TRASPORT UNITS QUALITY CONTROL AND R&D ENERGY MANAGEMENT AND

746 views • 20 slides
Dr. Michelle Black Map of Japan I lived on Kyushu island in Saga Prefecture ( ,

Dr. Michelle Black Map of Japan I lived on Kyushu island in Saga Prefecture ( ,

Dr. Michelle Black Map of Japan I lived on Kyushu island in Saga Prefecture ( , Saga-ken ) for 3 years Saga Prefecture is a prefecture in the northwest part of the island of Kyushu, Japan. [ It touches both the Sea of Japan and

367 views • 36 slides
The SAGA Project: Satellites Around Milky Way-like Galaxies Marla Geha (Yale) Risa Wechsler

The SAGA Project: Satellites Around Milky Way-like Galaxies Marla Geha (Yale) Risa Wechsler

The SAGA Project: Satellites Around Milky Way-like Galaxies Marla Geha (Yale) Risa Wechsler (Stanford), Yao-Yuan Mao (UPitt/Rutgers), Erik Tollerud (STScI), Ben Weiner (UA), Nitya Kallivayalil (UA) and the SAGA team Satellites Around a Milky

335 views • 17 slides
Travel Insurance Niall Palmer Saga Insurance Overview About Saga Types of Travel

Travel Insurance Niall Palmer Saga Insurance Overview About Saga Types of Travel

Travel Insurance Niall Palmer Saga Insurance Overview About Saga Types of Travel Insurance What Travel Insurance covers you for Preparing to travel Where to buy Travel Insurance Dos and don'ts Case

662 views • 25 slides
Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern

Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern

Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern Matching 1 Outline and Reading Strings (11.1) Pattern matching algorithms Brute-force algorithm (11.2.1) Boyer-Moore algorithm (11.2.2)

612 views • 14 slides
Pattern Review Pattern Name and Classification: A descriptive and unique name that helps in

Pattern Review Pattern Name and Classification: A descriptive and unique name that helps in

Pattern Review Pattern Name and Classification: A descriptive and unique name that helps in identifying and referring to the pattern. Intent: A description of the goal behind the pattern and the reason for using it. Also Known As: Other names for

352 views • 4 slides
Today you are going to: Watch a video Learn what a mini saga is and Learn what a

Today you are going to: Watch a video Learn what a mini saga is and Learn what a

Today you are going to: Watch a video Learn what a mini saga is and Learn what a mini saga is and is and how to write one is and how to write one Take part in a group discussion discussion Generate ideas to write about

604 views • 16 slides
Using Deep Learning to rank and tag 6,30 5,90 millions of hotel images 15/11/2018 - PyParis

Using Deep Learning to rank and tag 6,30 5,90 millions of hotel images 15/11/2018 - PyParis

0,00 9,80 16,10 16,10 7,40 6,94 Using Deep Learning to rank and tag 6,30 5,90 millions of hotel images 15/11/2018 - PyParis 2018 0,00 Christopher Lennan (Senior Data Scientist) @chris_lennan Tanuj Jain (Data Scientist) @tjainn

1.02k views • 87 slides
Modern Display Technology - Rendering Challenges - Guest Lecturer: Hyeonseung Yu Philipp

Modern Display Technology - Rendering Challenges - Guest Lecturer: Hyeonseung Yu Philipp

Modern Display Technology - Rendering Challenges - Guest Lecturer: Hyeonseung Yu Philipp Slusallek Karol Myszkowski Gurprit Singh Realistic Image Synthesis SS18 Modern Display Technologies Karol Myszkowski Outline Binocular 3D

1.03k views • 68 slides
Montanas Statewide Longitudinal Data System (SLDS) Student Achievement Fact Data Available

Montanas Statewide Longitudinal Data System (SLDS) Student Achievement Fact Data Available

Montanas Statewide Longitudinal Data System (SLDS) Student Achievement Fact Data Available Today MontCAS (Criterion-referenced Test) Results Adequate Yearly Progress Iowa Test of Basic Skills English Language Proficiency

363 views • 11 slides
Coronavirus (COVID-19) Update for Employers: Planning for the Potential Impact on the

Coronavirus (COVID-19) Update for Employers: Planning for the Potential Impact on the

Coronavirus (COVID-19) Update for Employers: Planning for the Potential Impact on the Construction Industry Presented By: Ashley Cuttino Ashley.Cuttino@ogletreedeakins.com Overview Prepare and Communicate Review/Revise Policies

664 views • 43 slides
Authorization for IoT using OAuth draft-seitz-ace-oauth-authz-00 Ludwig Seitz (ludwig@sics.se)

Authorization for IoT using OAuth draft-seitz-ace-oauth-authz-00 Ludwig Seitz (ludwig@sics.se)

Authorization for IoT using OAuth draft-seitz-ace-oauth-authz-00 Ludwig Seitz (ludwig@sics.se) Gran Selander (goran.selander@ericsson.com) Erik Wahlstrm (erik.wahlstrom@nexusgroup.com) Samuel Erdtman (samuel.erdtman@nexusgroup.com) Hannes

147 views • 10 slides
Immer Immersion sion Ph Phase se MSTP Introduction Phase Directors Lourdes Estrada, Ph.D.

Immer Immersion sion Ph Phase se MSTP Introduction Phase Directors Lourdes Estrada, Ph.D.

Immer Immersion sion Ph Phase se MSTP Introduction Phase Directors Lourdes Estrada, Ph.D. Kendra Parekh, M.D. Program Manager Brenna Hansen Program Coordinator Program Assistant LaToya Ford Bethanie McCrary

1.45k views • 71 slides
UTILIZING ACE REPORTS JULY 19, 2017 Presented by: Beth Pride, President, BPE Global Julie Gibbs,

UTILIZING ACE REPORTS JULY 19, 2017 Presented by: Beth Pride, President, BPE Global Julie Gibbs,

UNDERSTANDING & UTILIZING ACE REPORTS JULY 19, 2017 Presented by: Beth Pride, President, BPE Global Julie Gibbs, Director, BPE Global Presented for NCBFAA Educational Institute Innovative | Efficient | Effective Todays Speakers Julie

795 views • 51 slides
Arizona Adverse Childhood Experiences Consortium QUARTERLY MEETING Agenda 9:30 Welcome &

Arizona Adverse Childhood Experiences Consortium QUARTERLY MEETING Agenda 9:30 Welcome &

Arizona Adverse Childhood Experiences Consortium QUARTERLY MEETING Agenda 9:30 Welcome & Board Update 9:45 Workgroup Updates 10:15 State Targeted Response to the Opioid Crisis Grant Update 10:30 Historical Trauma and the COVID-19

390 views • 25 slides

More recommend