Events First Microservices Jonas Bonr @jboner So, you want to do - - PowerPoint PPT Presentation

▶
events first
SMART_READER_LITE
LIVE PREVIEW

Events First Microservices Jonas Bonr @jboner So, you want to do - - PowerPoint PPT Presentation

Aug 08, 2023 164 likes •1.68k views

Designing Events First Microservices Jonas Bonr @jboner So, you want to do microservices? Make sure you dont end up with Microliths Make sure you dont end up with Microliths We can do better than this Events First Domain Driven

slide-1
SLIDE 1

Designing

Events First

Microservices

Jonas Bonér

@jboner

slide-2
SLIDE 2

So, you want to do

microservices?

slide-3
SLIDE 3

Make sure you don’t end up with

Microliths

slide-4
SLIDE 4

Make sure you don’t end up with

Microliths

We can do better than this

slide-5
SLIDE 5

Events First

Domain Driven

Design

slide-6
SLIDE 6

“When you start modeling events, it forces you to think about the behaviour

  • f the system. As opposed to thinking

about the structure of the system.”

  • Greg Young

A Decade of DDD, CQRS, Event Sourcing, Greg Young (Presentation from 2016)

slide-7
SLIDE 7

✴ Don’t focus on the things

The Nouns The Domain Objects

slide-8
SLIDE 8

✴ Don’t focus on the things

The Nouns The Domain Objects

✴ Focus on what happens

The Verbs The Events

slide-9
SLIDE 9

What is an

Event?

slide-10
SLIDE 10

The Nature of Events

slide-11
SLIDE 11

✴ Events represent Facts of information

➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow

The Nature of Events

slide-12
SLIDE 12

✴ Events represent Facts of information

➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow

✴ Events/Facts can be disregarded/Ignored

The Nature of Events

slide-13
SLIDE 13

✴ Events represent Facts of information

➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow

✴ Events/Facts can be disregarded/Ignored ✴ Events/Facts Can not be retracted (once accepted)

The Nature of Events

slide-14
SLIDE 14

✴ Events represent Facts of information

➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow

✴ Events/Facts can be disregarded/Ignored ✴ Events/Facts Can not be retracted (once accepted) ✴ Events/Facts Can not be deleted (once accepted)

➡ Might be needed for legal or moral reasons

The Nature of Events

slide-15
SLIDE 15

✴ Events represent Facts of information

➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow

✴ Events/Facts can be disregarded/Ignored ✴ Events/Facts Can not be retracted (once accepted) ✴ Events/Facts Can not be deleted (once accepted)

➡ Might be needed for legal or moral reasons

✴ Events/Facts (new) can invalidate existing Facts

The Nature of Events

slide-16
SLIDE 16

Mine the

Facts

slide-17
SLIDE 17
slide-18
SLIDE 18

Event Storming

slide-19
SLIDE 19

Event Driven Design

slide-20
SLIDE 20

✴ IntentS

➡ Communication ➡ Conversations ➡ Expectations ➡ Contracts ➡ Control Transfer

Event Driven Design

slide-21
SLIDE 21

✴ IntentS

➡ Communication ➡ Conversations ➡ Expectations ➡ Contracts ➡ Control Transfer

Event Driven Design

✴ Facts

➡ State ➡ History ➡ Causality ➡ Notifications ➡ State Transfer

slide-22
SLIDE 22

✴ IntentS

➡ Communication ➡ Conversations ➡ Expectations ➡ Contracts ➡ Control Transfer

Event Driven Design

✴ Facts

➡ State ➡ History ➡ Causality ➡ Notifications ➡ State Transfer

Commands

slide-23
SLIDE 23

✴ IntentS

➡ Communication ➡ Conversations ➡ Expectations ➡ Contracts ➡ Control Transfer

Event Driven Design

✴ Facts

➡ State ➡ History ➡ Causality ➡ Notifications ➡ State Transfer

Commands Events

slide-24
SLIDE 24

Event Driven Design

slide-25
SLIDE 25

✴Commands

➡ Object form of method/Action request ➡ Imperative: CreateOrder, ShipProduct

Event Driven Design

slide-26
SLIDE 26

✴Commands

➡ Object form of method/Action request ➡ Imperative: CreateOrder, ShipProduct

✴Reactions

➡ Represents side-effects

Event Driven Design

slide-27
SLIDE 27

✴Commands

➡ Object form of method/Action request ➡ Imperative: CreateOrder, ShipProduct

✴Reactions

➡ Represents side-effects

✴Events

➡ Represents something that has happened ➡ Past-tense: OrderCreated, ProductShipped

Event Driven Design

slide-28
SLIDE 28

Commands Events

vs

slide-29
SLIDE 29

Commands Events

vs

  • 1. All about intent
  • 1. Intentless
slide-30
SLIDE 30

Commands Events

vs

  • 1. All about intent
  • 2. Directed
  • 1. Intentless
  • 2. Anonymous
slide-31
SLIDE 31

Commands Events

vs

  • 1. All about intent
  • 2. Directed
  • 3. Single addressable

destination

  • 1. Intentless
  • 2. Anonymous
  • 3. Just happens - for
  • thers (0-N) to observe
slide-32
SLIDE 32

Commands Events

vs

  • 1. All about intent
  • 2. Directed
  • 3. Single addressable

destination

  • 4. Models personal

communication

  • 1. Intentless
  • 2. Anonymous
  • 3. Just happens - for
  • thers (0-N) to observe
  • 4. Models broadcast

(speakers corner)

slide-33
SLIDE 33

Commands Events

vs

  • 1. All about intent
  • 2. Directed
  • 3. Single addressable

destination

  • 4. Models personal

communication

  • 5. Distributed focus
  • 1. Intentless
  • 2. Anonymous
  • 3. Just happens - for
  • thers (0-N) to observe
  • 4. Models broadcast

(speakers corner)

  • 5. Local focus
slide-34
SLIDE 34

Commands Events

vs

  • 1. All about intent
  • 2. Directed
  • 3. Single addressable

destination

  • 4. Models personal

communication

  • 5. Distributed focus
  • 6. Command & Control
  • 1. Intentless
  • 2. Anonymous
  • 3. Just happens - for
  • thers (0-N) to observe
  • 4. Models broadcast

(speakers corner)

  • 5. Local focus
  • 6. Autonomy
slide-35
SLIDE 35

Let the Events Define the

Bounded Context

slide-36
SLIDE 36

Event Driven

Services

slide-37
SLIDE 37
  • 1. REceive & react (or not) 


to facts* that are coming its way

Event Driven

Services

slide-38
SLIDE 38
  • 1. REceive & react (or not) 


to facts* that are coming its way

  • 2. Publish new facts* asynchronously 


to the rest of the world

Event Driven

Services

slide-39
SLIDE 39
  • 1. REceive & react (or not) 


to facts* that are coming its way

  • 2. Publish new facts* asynchronously 


to the rest of the world

  • 3. Invert the control flow


to minimize coupling and increase autonomy

Event Driven

Services

slide-40
SLIDE 40
  • 1. REceive & react (or not) 


to facts* that are coming its way

  • 2. Publish new facts* asynchronously 


to the rest of the world

  • 3. Invert the control flow


to minimize coupling and increase autonomy

Event Driven

Services

*Facts == Immutable Events

slide-41
SLIDE 41

Mutable State Is Fine

But Needs To Be

Contained

And Non Observable

slide-42
SLIDE 42

Publish Facts

To Outside World

slide-43
SLIDE 43

✴ Maintains Integrity & consistency ✴ Is our Unit of Consistency ✴ Is our Unit of Failure ✴ Is our Unit of Determinism ✴ Is fully Autonomous

The Aggregate

slide-44
SLIDE 44

Event Driven

Services

slide-45
SLIDE 45

Event Driven

Services

slide-46
SLIDE 46

Event Driven

Services

Command

slide-47
SLIDE 47

Event Driven

Services

Command

slide-48
SLIDE 48

Event Driven

Services

Command

slide-49
SLIDE 49

Event Driven

Services

Command Event

Event Stream

slide-50
SLIDE 50

Event Driven

Services

Command Event Event Event

Event Stream

slide-51
SLIDE 51

Event Driven

Services

Command Event Event Event

Event Stream

slide-52
SLIDE 52

Event Driven

Services

Command Event Event Event

Event Stream

slide-53
SLIDE 53

Event Driven

Services

Command Event Event Event

Event Stream

slide-54
SLIDE 54

Event Driven

Services

Command Event Event Event

Event Stream

slide-55
SLIDE 55

Event Driven

Services

Eventual

Consistency

Command Event Event Event

Event Stream

slide-56
SLIDE 56

Event

Stream

Use The

slide-57
SLIDE 57

Event

Stream

Use The

as the communication fabric

slide-58
SLIDE 58

Event

Stream

Use The

slide-59
SLIDE 59

Event

Stream

Use The as the integration fabric

slide-60
SLIDE 60

Event

Stream

Use The

slide-61
SLIDE 61

Event

Stream

Use The as the replication fabric

slide-62
SLIDE 62

Event

Stream

Use The

slide-63
SLIDE 63

Event

Stream

Use The

as the consensus fabric

slide-64
SLIDE 64

Event

Stream

Use The

slide-65
SLIDE 65

Event

Stream

Use The

as the Persistence fabric

slide-66
SLIDE 66

The Problem With

CRUD Services

slide-67
SLIDE 67

✴ CRUD is fine for totally isolated data

The Problem With

CRUD Services

slide-68
SLIDE 68

✴ CRUD is fine for totally isolated data ✴ But, cross CRUD services consistency

The Problem With

CRUD Services

slide-69
SLIDE 69

✴ CRUD is fine for totally isolated data ✴ But, cross CRUD services consistency ➡ Is hard ⇨ No Joins

The Problem With

CRUD Services

slide-70
SLIDE 70

✴ CRUD is fine for totally isolated data ✴ But, cross CRUD services consistency ➡ Is hard ⇨ No Joins ➡ Has ad-hoc & weak guarantees

The Problem With

CRUD Services

slide-71
SLIDE 71

✴ CRUD is fine for totally isolated data ✴ But, cross CRUD services consistency ➡ Is hard ⇨ No Joins ➡ Has ad-hoc & weak guarantees

The Problem With

CRUD Services

slide-72
SLIDE 72

“Two-phase commit is the anti-availability protocol.”

  • Pat Helland

Standing on Distributed Shoulders of Giants - Pat Helland

slide-73
SLIDE 73

STRONG

Consistency

Is the wrong default

In distributed systems

slide-74
SLIDE 74

STRONG

Consistency

Is the wrong default

In distributed systems

What can we do?

slide-75
SLIDE 75

Eventual

Consistency

We have to rely on

slide-76
SLIDE 76

Eventual

Consistency

We have to rely on

But relax—it’s how the world works

slide-77
SLIDE 77

Embrace

Reality

We Need to

slide-78
SLIDE 78

Embrace

Reality

We Need to Not Fight it

slide-79
SLIDE 79

Information Has Latency

slide-80
SLIDE 80

Information Is Always

From the Past

slide-81
SLIDE 81

Welcome To The Wild Ocean Of

Non Determinism

Distributed Systems

slide-82
SLIDE 82

“In a system which cannot count on distributed transactions, the management

  • f uncertainty must be implemented in the

business logic.”

  • Pat Helland

Life Beyond Distributed Transactions, Pat Helland (2007)

We Need To Model

Uncertainty

slide-83
SLIDE 83

Events Can Lead To Greater

Certainty

slide-84
SLIDE 84

“An autonomus component can only promise its own behavior.” “Autonomy makes information local, leading to greater certainty and stability.”

  • Mark Burgess

In Search of Certainty, Thinking in Promises - Mark Burgess

slide-85
SLIDE 85

Events Can Help Us Craft

Autonomous Islands

Of Determinism

slide-86
SLIDE 86

Data on the inside vs Data on the outside - Pat Helland

slide-87
SLIDE 87

Data on the inside vs Data on the outside - Pat Helland

Inside Data

Our current present ⇨ state

slide-88
SLIDE 88

Data on the inside vs Data on the outside - Pat Helland

Inside Data

Our current present ⇨ state

Outside Data

Blast from the past ⇨ Events/facts

slide-89
SLIDE 89

Data on the inside vs Data on the outside - Pat Helland

Inside Data

Our current present ⇨ state

Outside Data

Blast from the past ⇨ Events/facts

Between Services

Hope for the future ⇨ commands

slide-90
SLIDE 90

A system of microservices is a never ending stream towards convergence

slide-91
SLIDE 91

There Is No Now

A system of microservices is a never ending stream towards convergence

slide-92
SLIDE 92

Resilience

is by

Design

Photo courtesy of FEMA/Joselyne Augustino

slide-93
SLIDE 93

Events Can Help Us

Manage

Failure

Instead Of Trying To Avoid It

slide-94
SLIDE 94

Requirements for a

Sane Failure Model

  • 1. Contained—Avoid cascading failures
  • 2. Reified—as Events
  • 3. Signalled—Asynchronously
  • 4. Observed—by 1-N
  • 5. Managed—Outside failed Context

Failures need to be

slide-95
SLIDE 95

Event Based

Persistence

slide-96
SLIDE 96

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

slide-97
SLIDE 97

Service B Service A

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

slide-98
SLIDE 98

Service B Service A

CRUD

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD

slide-99
SLIDE 99

Service B Service A

TABLE A

CRUD

TABLE B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD

slide-100
SLIDE 100

Service B Service A

TABLE A

CRUD

TABLE B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD

slide-101
SLIDE 101

Service B Service A

TABLE A

CRUD

TABLE B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD

Atomic Update & event

slide-102
SLIDE 102

Service B Service A

TABLE A

CRUD

TABLE B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD

Atomic Update & event

slide-103
SLIDE 103

Service C Service B Service A

TABLE A

CRUD

TABLE B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD

Atomic Update & event

slide-104
SLIDE 104

Service C Service B Service A

TABLE A

CRUD

TABLE B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD

Atomic Update & event

slide-105
SLIDE 105

Service C Service B Service A

TABLE A

CRUD

TABLE B JOINS Table A Table B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD

Atomic Update & event

slide-106
SLIDE 106

Service C Service B Service A

TABLE A

CRUD

TABLE B JOINS Table A Table B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD Read Only

Atomic Update & event

slide-107
SLIDE 107

Service C Service B Service A

Eventual Consistency

TABLE A

CRUD

TABLE B JOINS Table A Table B

You can use CRUD Together with Event Streams To get an internally consistent Materialized View

CRUD Read Only

Atomic Update & event

slide-108
SLIDE 108

“Update-in-place strikes systems designers as a cardinal sin: it violates traditional accounting practices that have been observed for hundreds of years.”

  • jim Gray

The Transaction Concept, Jim Gray (1981)

slide-109
SLIDE 109

“The truth is the log. The database is a cache

  • f a subset of the log.”
  • Pat Helland

Immutability Changes Everything, Pat Helland (2015)

slide-110
SLIDE 110

Event Logging

The Bedrock

slide-111
SLIDE 111

Event Sourcing

A Cure For the Cardinal Sin

slide-112
SLIDE 112

Event

Sourced

Services

slide-113
SLIDE 113

Happy Path

Event

Sourced

Services

slide-114
SLIDE 114

Happy Path

Event

Sourced

Services

1) Receive and verify Command (“ApprovePayment”)

slide-115
SLIDE 115

Happy Path

Event

Sourced

Services

2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”)

slide-116
SLIDE 116

Happy Path

Event

Sourced

Services

2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log

slide-117
SLIDE 117

Happy Path

Event

Sourced

Services

2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state

slide-118
SLIDE 118

Happy Path

Event

Sourced

Services

5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state

slide-119
SLIDE 119

SAD Path - recover from failure Happy Path

Event

Sourced

Services

5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state

slide-120
SLIDE 120

SAD Path - recover from failure Happy Path

Event

Sourced

Services

5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state 1) Rehydrate Events from Event Log

slide-121
SLIDE 121

SAD Path - recover from failure Happy Path

Event

Sourced

Services

5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state 1) Rehydrate Events from Event Log 2) Update internal component state

slide-122
SLIDE 122

SAD Path - recover from failure Happy Path

Event

Sourced

Services

5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state 1) Rehydrate Events from Event Log 2) Update internal component state

M e m

  • r

y I m a g e

slide-123
SLIDE 123

Event Sourcing

slide-124
SLIDE 124

Event Sourcing

✴ One single Source of Truth with All history

slide-125
SLIDE 125

Event Sourcing

✴ One single Source of Truth with All history ✴ Allows for Memory Image (Durable In-Memory State)

slide-126
SLIDE 126

Event Sourcing

✴ One single Source of Truth with All history ✴ Allows for Memory Image (Durable In-Memory State) ✴ Avoids the Object-relational mismatch

slide-127
SLIDE 127

Event Sourcing

✴ One single Source of Truth with All history ✴ Allows for Memory Image (Durable In-Memory State) ✴ Avoids the Object-relational mismatch ✴ Allows others to Subscribe to state changes

slide-128
SLIDE 128

Event Sourcing

✴ One single Source of Truth with All history ✴ Allows for Memory Image (Durable In-Memory State) ✴ Avoids the Object-relational mismatch ✴ Allows others to Subscribe to state changes ✴ Has good Mechanical sympathy
 (Single Writer Principle etc.)

slide-129
SLIDE 129

Untangle Your

Read and Write Models

With CQRS

slide-130
SLIDE 130

CQRS

slide-131
SLIDE 131

CQRS

slide-132
SLIDE 132

CQRS

Write Side Model Commands

slide-133
SLIDE 133

CQRS

Write Side Model Write to Event Log Commands

slide-134
SLIDE 134

CQRS

Write Side Model Events Write to Event Log Commands

slide-135
SLIDE 135

CQRS

Write Side Model Events Events Write to Event Log Commands

slide-136
SLIDE 136

CQRS

Read Side Model Write Side Model Events Events Read Data Store Write to Event Log Commands

slide-137
SLIDE 137

CQRS

Read Side Model Write Side Model Events Queries Events Read Data Store Write to Event Log Commands

slide-138
SLIDE 138

CQRS

Read Side Model Write Side Model Events Queries Events Read Data Store Write to Event Log Eventual Consistency Commands

slide-139
SLIDE 139

Events

Allow Us To Manage

Time

slide-140
SLIDE 140

“Modelling events forces you to have a 
 temporal focus on what’s going on in the system. Time becomes a crucial factor of the system.”

  • Greg Young

A Decade of DDD, CQRS, Event Sourcing, Greg Young (Presentation from 2016)

slide-141
SLIDE 141

✴ Event is a snapshot in time ✴ Event ID is an index for time ✴ Event Log is our full history The database of Our past

The path to Our present

Event Sourcing

Allows Us To

Model Time

slide-142
SLIDE 142

Event Sourcing

Allows For

Time Travel

slide-143
SLIDE 143

Event Sourcing

Allows For

Time Travel

slide-144
SLIDE 144

Event Sourcing

Allows For

Time Travel

✴Replay the log for historic debugging ✴Replay the log for auditing & traceability ✴Replay the log on failure ✴Replay the log for replication

slide-145
SLIDE 145

We Can Even Fork the Past ...Or Join Two Distinct Pasts

slide-146
SLIDE 146

Key Takeaways

slide-147
SLIDE 147

Key Takeaways

Events-First design helps you to:

✴ reduce risk when modernizing applications ✴ Move Faster towards a Resilient and Scalable architecture ✴ Design autonomous services ✴ Balance Certainty and Uncertainty

slide-148
SLIDE 148

Key Takeaways

Events-First design helps you to:

✴ reduce risk when modernizing applications ✴ Move Faster towards a Resilient and Scalable architecture ✴ Design autonomous services ✴ Balance Certainty and Uncertainty

Event Logging allows you to:

✴ AVOID CRUD and ORM ✴ take control of your system’s history ✴ time-travel ✴ Balance Strong and eventual consistency

slide-149
SLIDE 149

http://akka.io

slide-150
SLIDE 150

Learn More

Download my latest book for free at:

bit.ly/reactive-microsystems