distributed systems @berndruecker With thoughts from - - PowerPoint PPT Presentation

▶
distributed systems
SMART_READER_LITE
LIVE PREVIEW

distributed systems @berndruecker With thoughts from - - PowerPoint PPT Presentation

Jul 24, 2023 164 likes •1.12k views

Complex event flows in distributed systems @berndruecker With thoughts from http://flowing.io @berndruecker | @martinschimak 3 common hypotheses I check today: # Events decrease coupling # Orchestration needs to be avoided # Workflow engines

slide-1
SLIDE 1

Complex event flows in distributed systems

@berndruecker

With thoughts from http://flowing.io @berndruecker | @martinschimak

slide-2
SLIDE 2

3 common hypotheses I check today: # Events decrease coupling # Orchestration needs to be avoided # Workflow engines are painful

slide-3
SLIDE 3

Berlin, Germany

bernd.ruecker@camunda.com @berndruecker

Bernd Ruecker

Co-founder and Developer Advocate of Camunda

slide-4
SLIDE 4
slide-5
SLIDE 5

Simplified example: dash button

Photo by 0xF2, available under Creative Commons BY-ND 2.0

  • license. https://www.flickr.com/photos/0xf2/29873149904/
slide-6
SLIDE 6

Three steps…

slide-7
SLIDE 7

Who is involved? Some bounded contexts…

Checkout Payment Inventory Shipment

slide-8
SLIDE 8

(Micro-)services

Checkout Payment Inventory Shipment

slide-9
SLIDE 9

Autonomous (micro-)services

Checkout Payment Inventory Shipment

Dedicated Application Processes Dedicated infrastructure Dedicated Development Teams

slide-10
SLIDE 10

Events decrease coupling

slide-11
SLIDE 11

Example

Checkout Payment Inventory Shipment

The button blinks if we can ship within 24 hours

slide-12
SLIDE 12

Request/response: temporal coupling

Checkout Payment Inventory Shipment

Request Response

The button blinks if we can ship within 24 hours

slide-13
SLIDE 13

T emporal decoupling with events and read models

Checkout Payment Inventory Shipment

Good Stored Read Model Good Fetched

The button blinks if we can ship within 24 hours *Events are facts about what happened (in the past)

slide-14
SLIDE 14

Events can decrease coupling*

*e.g. decentral data-management, read models, extract cross-cutting aspects

slide-15
SLIDE 15

Peer-to-peer event chains

Checkout Payment Inventory Shipment

Order placed Payment received Goods shipped Goods fetched

slide-16
SLIDE 16

Peer-to-peer event chains

Checkout Payment Inventory Shipment

Order placed Payment received Goods shipped Goods fetched

slide-17
SLIDE 17

The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years.

https://martinfowler.com/articles/201701-event-driven.html

slide-18
SLIDE 18

The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years.

https://martinfowler.com/articles/201701-event-driven.html

slide-19
SLIDE 19

The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years.

https://martinfowler.com/articles/201701-event-driven.html

slide-20
SLIDE 20

Peer-to-peer event chains

Checkout Payment Inventory Shipment

Order placed Payment received Goods shipped Goods fetched

Fetch the goods before the payment

slide-21
SLIDE 21

Peer-to-peer event chains

Checkout Payment Inventory Shipment

Fetch the goods before the payment

Goods fetched Order placed Payment received Goods shipped

slide-22
SLIDE 22

Peer-to-peer event chains

Checkout Payment Inventory Shipment

Fetch the goods before the payment Customers can pay via invoice

Goods fetched Order placed Payment received Goods shipped

…

slide-23
SLIDE 23

Photo by born1945, available under Creative Commons BY 2.0 license.

slide-24
SLIDE 24

Order

Extract the end-to-end responsibility

Checkout Payment Inventory Shipment

*Commands have an intent about what needs to happen in the future

Order placed Payment received Retrieve payment

slide-25
SLIDE 25

Commands help to avoid (complex) peer-to-peer event chains

slide-26
SLIDE 26

Orchestration needs to be avoided

slide-27
SLIDE 27

Smart ESB-like middleware

Checkout Payment Inventory Shipment Order

Order placed Payment received Good fetched Good shipped

slide-28
SLIDE 28

Dumb pipes

Checkout Payment Inventory Shipment Order

Smart endpoints and dumb pipes

Martin Fowler

slide-29
SLIDE 29

Danger of god services?

Checkout Order

A few smart god services tell anemic CRUD services what to do

Sam Newmann

Payment Inventory Shipment

slide-30
SLIDE 30

Danger of god services?

Checkout

Payment Inventory Shipment

Order

A few smart god services tell anemic CRUD services what to do

Sam Newmann

slide-31
SLIDE 31

A god service is only created by bad API design!

slide-32
SLIDE 32

Example

Order Payment

Retrieve Payment

slide-33
SLIDE 33

Example

Order Payment Credit Card

Retrieve Payment

slide-34
SLIDE 34

Example

Order Payment Credit Card

Retrieve Payment Rejected

slide-35
SLIDE 35

Example

Order Payment

Client of dumb endpoints easily become a god services. If the credit card was rejected, the customer can provide new details

Credit Card

Retrieve Payment Rejected Rejected

slide-36
SLIDE 36

Payment failed

Who is responsible to deal with problems?

Order Payment

If the credit card was rejected, the customer can provide new details

Credit Card

Retrieve Payment Rejected Payment received

slide-37
SLIDE 37

Payment failed

Who is responsible to deal with problems?

Order Payment

Clients of smart endpoints remains lean. If the credit card was rejected, the customer can provide new details

Credit Card

Retrieve Payment Rejected Payment received

Smart endpoints are potentially long-running

slide-38
SLIDE 38

Photo by Tookapic, available under Creative Commons CC0 1.0 license.

slide-39
SLIDE 39

„There was an error while sending your boarding pass“

slide-40
SLIDE 40

Check-in Web-UI

Me

Current situation

slide-41
SLIDE 41

Check-in

Barcode Generator

Web-UI

Me

Output Mgmt

Current situation

slide-42
SLIDE 42

Check-in

Barcode Generator

Web-UI

Me

Output Mgmt

Current situation

slide-43
SLIDE 43

Check-in

Barcode Generator

Web-UI

Me

Output Mgmt

Current situation – the bad part

slide-44
SLIDE 44

Check-in

Barcode Generator

Web-UI

Me

Output Mgmt

Current situation – the bad part

Stateful Retry

slide-45
SLIDE 45
slide-46
SLIDE 46

We are having some technical difficulties and cannot present you your boarding pass right away. But we do actively retry ourselves, so lean back, relax and we will send it

  • n time.
slide-47
SLIDE 47

Check-in

Barcode Generator

Web-UI

Me

Output Mgmt

Possible situation – much better!

slide-48
SLIDE 48

Check-in

Barcode Generator

Web-UI

Me

Output Mgmt

Possible situation – much better!

Stateful Retry

slide-49
SLIDE 49

Check-in

Barcode Generator

Web-UI

Me

Output Mgmt

Stateful Retry

Possible situation – much better!

The failure never leaves this scope!

slide-50
SLIDE 50

Persist thing (Entity, Actor, …) State machine or workflow engine T ypical concerns DIY = effort, accidental complexity

Scheduling, Versioning,

  • perating, visibility,

scalability, …

Handling State

slide-51
SLIDE 51

Workflow engines are painful

Complex, proprietary, heavyweight, central, developer adverse, …

slide-52
SLIDE 52

Avoid the wrong tools!

Death by properties panel

Low-code is great! (You can get rid

  • f your developers!)

Complex, proprietary, heavyweight, central, developer adverse, …

slide-53
SLIDE 53

Workflow engines, state machines

It is

relevant

in modern architectures

slide-54
SLIDE 54

CADENCE

Silicon valley has recognized

Workflow engines, state machines

slide-55
SLIDE 55

CADENCE

Workflow engines, state machines

slide-56
SLIDE 56

CADENCE

also at scale

Workflow engines, state machines

slide-57
SLIDE 57

CADENCE

for todays demo

Workflow engines, state machines

slide-58
SLIDE 58

public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") // .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

What do I mean by „leightweight?“

slide-59
SLIDE 59

public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") // .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

Build engine in one line of code (using in- memory H2)

slide-60
SLIDE 60

public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

Define flow e.g. in Java DSL

slide-61
SLIDE 61

public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

Define flow e.g. in Java DSL

slide-62
SLIDE 62

BPMN

Business Process Model and Notation ISO Standard

slide-63
SLIDE 63

public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

We can attach code…

slide-64
SLIDE 64

public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

…that is called when workflow instances pass through

slide-65
SLIDE 65

public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

Start instances

slide-66
SLIDE 66

Payment

Now you have a state machine!

slide-67
SLIDE 67

Payment

Easy to handle time

slide-68
SLIDE 68

Stateful retries

Payment Credit Card

REST

slide-69
SLIDE 69

Fallbacks increase resilience

Payment Credit Card

REST

slide-70
SLIDE 70

Distributed systems

slide-71
SLIDE 71

It is impossible to differentiate certain failure scenarios.

Independant of communication style!

Service Provider Client

slide-72
SLIDE 72

Distributed systems introduce complexity you have to tackle!

Credit Card Payment

REST

slide-73
SLIDE 73

Distributed systems introduce complexity you have to tackle!

Credit Card Payment

REST

slide-74
SLIDE 74

Distributed systems

slide-75
SLIDE 75

Distributed transaction with compensations*

slide-76
SLIDE 76

Living documentation for long-running behaviour

slide-77
SLIDE 77

Visual HTML reports for test cases

slide-78
SLIDE 78

Proper Operations

Visibility + Context

slide-79
SLIDE 79

Before mapping processes explicitly with BPMN, the truth was buried in the code and nobody knew what was going on.

Jimmy Floyd, 24 Hour Fitnesse

„

slide-80
SLIDE 80

Workflows live inside service boundaries

slide-81
SLIDE 81

Explicit flows help separate domain and infrastructure

Infrastructure

Aggregates, Domain Events, Domain Services, etc … + the flow Workflow Engine Payment

Application Domain

slide-82
SLIDE 82

Manigfold architecture options

https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

slide-83
SLIDE 83

Manigfold architecture options

https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

slide-84
SLIDE 84

Manigfold architecture options

https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

slide-85
SLIDE 85

Manigfold architecture options

https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

slide-86
SLIDE 86

…

Manigfold architecture options

https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

slide-87
SLIDE 87

Lightweight workflow engines are great – don‘t DIY*

*e.g. enabling potentially long-running services, solving hard developer problems, can run decentralized

slide-88
SLIDE 88
slide-89
SLIDE 89

Zalando Sales-Order and Order-Fulfillment via Camunda for every order worldwide Orders Q2-2017: 22,2 Mio. Sales 2016: 3,6 Mrd. EUR Growth 2016: 23%

slide-90
SLIDE 90

Code example & live demo

Inventory Payment Order Shipping Checkout Monitor

https://github.com/flowing/flowing-retail/

Human T asks

H2 H2
slide-91
SLIDE 91

# Events decrease coupling: sometimes

read-models, but no complex peer-to-peer event chains!

# Orchestration needs to be avoided: sometimes

no ESB, smart endpoints/dumb pipes, important capabilities need a home

# Workflow engines are painful: some of them

lightweight engines are easy to use and can run decentralized, they solve hard developer problems, don‘t DIY

slide-92
SLIDE 92

Thank you! Meet me at

Meet the experts

Now!

slide-93
SLIDE 93

bernd.ruecker@camunda.com @berndruecker https://bernd-ruecker.com https://blog.bernd-ruecker.com https://github.com/flowing https://www.infoq.com/articles/ events-workflow-automation

With thoughts from http://flowing.io @berndruecker | @martinschimak

Contact: Slides: Blog: Code:

https://www.infoworld.com/article/3254777/ application-development/ 3-common-pitfalls-of-microservices- integrationand-how-to-avoid-them.html