distributed systems @berndruecker With thoughts from - PowerPoint PPT Presentation

Complex event flows in distributed systems @berndruecker With thoughts from http://flowing.io @berndruecker | @martinschimak

3 common hypotheses I check today: # Events decrease coupling # Orchestration needs to be avoided # Workflow engines are painful

Bernd Ruecker Co-founder and Developer Advocate of Camunda Berlin, Germany bernd.ruecker@camunda.com @berndruecker

Simplified example: dash button Photo by 0xF2, available under Creative Commons BY-ND 2.0 license. https://www.flickr.com/photos/0xf2/29873149904/

Three steps …

Who is involved? Some bounded contexts … Checkout Shipment Payment Inventory

(Micro-)services Checkout Shipment Payment Inventory

Autonomous (micro-)services Checkout Dedicated Application Processes Dedicated infrastructure Shipment Payment Dedicated Development Teams Inventory

Events decrease coupling

Example The button blinks if we can ship within 24 hours Checkout Shipment Payment Inventory

Request/response: temporal coupling The button blinks if we can ship within 24 hours Checkout Response Request Shipment Payment Inventory

T emporal decoupling with events and read models The button blinks if we can ship within 24 hours Checkout Read Model Good Good Fetched Stored Shipment Payment Inventory *Events are facts about what happened (in the past)

Events can decrease coupling* *e.g. decentral data-management, read models, extract cross-cutting aspects

Peer-to-peer event chains Order placed Checkout Shipment Payment Inventory Goods shipped Payment Goods received fetched

Peer-to-peer event chains Order placed Checkout Shipment Payment Inventory Goods shipped Payment Goods received fetched

The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html

The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html

The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html

Peer-to-peer event chains Fetch the goods before the Order payment placed Checkout Shipment Payment Inventory Goods shipped Payment Goods received fetched

Peer-to-peer event chains Fetch the goods before the Order payment placed Checkout Shipment Payment Inventory Goods shipped Payment Goods received fetched

Peer-to-peer event chains Fetch the goods before the Order payment placed Customers can Checkout pay via invoice … Shipment Payment Inventory Goods shipped Payment Goods received fetched

Photo by born1945, available under Creative Commons BY 2.0 license.

Extract the end-to-end responsibility Order placed Checkout Retrieve Order payment Payment Shipment Payment received Inventory *Commands have an intent about what needs to happen in the future

Commands help to avoid (complex) peer-to-peer event chains

Orchestration needs to be avoided

Smart ESB-like middleware Order Checkout Order placed Payment received Good Shipment fetched Payment Inventory Good shipped

Dumb pipes Checkout Order Martin Fowler Smart endpoints Shipment Payment and dumb pipes Inventory

Danger of god services? Checkout Order Sam Newmann A few Shipment smart god services Payment tell Inventory anemic CRUD services what to do

Danger of god services? Checkout Order Sam Newmann A few Shipment Payment smart god services Inventory tell anemic CRUD services what to do

A god service is only created by bad API design!

Example Retrieve Payment Order Payment

Example Retrieve Payment Credit Order Payment Card

Example Retrieve Payment Credit Order Payment Card Rejected

Example Retrieve Payment Credit Order Payment Card Rejected If the credit Rejected card was rejected, the customer can provide new details Client of dumb endpoints easily become a god services.

Who is responsible to deal with problems? Retrieve Payment Credit Order Payment Card If the credit Rejected card was rejected, the customer can provide new Payment details Payment received failed

Who is responsible to deal with problems? Retrieve Payment Credit Order Payment Card If the credit Rejected card was Smart endpoints are rejected, the potentially long-running customer can provide new Payment details Payment received failed Clients of smart endpoints remains lean.

Photo by Tookapic, available under Creative Commons CC0 1.0 license.

„ There was an error while sending your boarding pass“

Current situation Web-UI Me Check-in

Current situation Web-UI Me Check-in Barcode Output Generator Mgmt

Current situation Web-UI Me Check-in Barcode Output Generator Mgmt

Current situation – the bad part Web-UI Me Check-in Barcode Output Generator Mgmt

Current situation – the bad part Web-UI Me Check-in Barcode Output Generator Stateful Mgmt Retry

We are having some technical difficulties and cannot present you your boarding pass right away. But we do actively retry ourselves, so lean back, relax and we will send it on time.

Possible situation – much better! Web-UI Me Check-in Barcode Output Generator Mgmt

Possible situation – much better! Web-UI Stateful Retry Me Check-in Barcode Output Generator Mgmt

Possible situation – much better! Web-UI Stateful The failure Retry never leaves Me Check-in this scope! Barcode Output Generator Mgmt

Handling State Persist thing State machine or (Entity, Actor, …) workflow engine DIY = effort, T ypical Scheduling, Versioning, accidental concerns operating, visibility, complexity scalability , …

Workflow engines are painful Complex, proprietary, heavyweight, central, developer adverse, …

Avoid the wrong tools! Low-code is great! (You can get rid of your developers!) Death by properties panel Complex, proprietary, heavyweight, central, developer adverse, …

Workflow engines, state machines It is relevant in modern architectures

Workflow engines, CADENCE state machines Silicon valley has recognized

Workflow engines, CADENCE state machines

Workflow engines, CADENCE state machines also at scale

Workflow engines, CADENCE state machines for todays demo

public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") // .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) What do I mean by .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() „ leightweight ?“ ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

public static void main(String[] args) { Build engine ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() in one line of .buildProcessEngine(); code engine.getRepositoryService().createDeployment() // (using in- .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") // memory H2) .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }