Microservices in a Streaming World There are many good reasons for - - PowerPoint PPT Presentation

▶
microservices in a streaming world there are many good
SMART_READER_LITE
LIVE PREVIEW

Microservices in a Streaming World There are many good reasons for - - PowerPoint PPT Presentation

Apr 07, 2023 572 likes •1.96k views

Microservices in a Streaming World There are many good reasons for building service-based systems Loose Coupling Bounded Contexts Autonomy Ease of scaling Composability But when we do, were building a distributed system

slide-1
SLIDE 1

Microservices in a Streaming World

slide-2
SLIDE 2

There are many good reasons for building service-based systems

  • Loose Coupling
  • Bounded Contexts
  • Autonomy
  • Ease of scaling
  • Composability
slide-3
SLIDE 3

But when we do, we’re building a distributed system

slide-4
SLIDE 4

This can be a bit tricky

slide-5
SLIDE 5

Monolithic & Centralised Approaches

Shared, mutable state

slide-6
SLIDE 6

Decentralisation

slide-7
SLIDE 7
slide-8
SLIDE 8

Stream Processing is a bit different

batch analytics => real time => at scale => accurately

slide-9
SLIDE 9

and comes with an interesting toolset

slide-10
SLIDE 10

Stream Processing Toolset Business Applications

slide-11
SLIDE 11

Some fundamental patterns of distributed systems

slide-12
SLIDE 12

Request / Response

slide-13
SLIDE 13

Mediator / Workflow

Request/Response

slide-14
SLIDE 14

Event Driven

Async / Fire and Forget

slide-15
SLIDE 15

Event Based

  • Simple
  • Synchronous
  • Event Driven
  • Good decoupling
  • Requires Broker
  • Fire & Forget
  • Polling
  • Full decoupling

Request/Response vs.

slide-16
SLIDE 16

SOA / Microservices

Message Broker

Event Based Request/Response

slide-17
SLIDE 17

Combinations

Event- Based Request/ Response

slide-18
SLIDE 18

Combinations

Withdraw £100

Account Service General Ledger

Customer Statements

Fraud Detection Check Funds Async Message Broker

I need money

ReST

slide-19
SLIDE 19

Services generally eschew shared, mutable state

slide-20
SLIDE 20

How do we put these things together?

slide-21
SLIDE 21

Request/Response

slide-22
SLIDE 22

Request/Response

Request Response

ReST

slide-23
SLIDE 23

Request/Response + Registry

Registry

Request Response

ReST

slide-24
SLIDE 24

Asynchronous and Event-Based Communication

slide-25
SLIDE 25

Queues

slide-26
SLIDE 26

Point to Point

Service A Service B

slide-27
SLIDE 27

Load Balancing

Instance 2 Instance 1 Single message allocation has scalability issues

slide-28
SLIDE 28

Batched Allocation

Instance 1 Instance 2

Throughput!

slide-29
SLIDE 29

Lose Ordering Guarantees

Fail!

Instance 1 Instance 2

slide-30
SLIDE 30

Topics

slide-31
SLIDE 31

Topics are Broadcast

Consumer Consumer

Broker

broadcast

slide-32
SLIDE 32

Topics Retain Ordering

Trades Buys Sells Broker

Instance 1 Instance 2

slide-33
SLIDE 33

Even when services fail

Trades Buys Sells Fail! Broker We retain ordering, but we have to detect & reprovision

Instance 1 Instance 2

slide-34
SLIDE 34

A Few Implications

slide-35
SLIDE 35

Queues Lose Ordering Guarantees at Scale

Fail!

Worker 1 Worker 2

slide-36
SLIDE 36

Trades Buys Sells

Topics don’t provide availability

Broker

slide-37
SLIDE 37

Trades Buys Sells

Messages are Transient

Broker

slide-38
SLIDE 38

Is there another way?

slide-39
SLIDE 39

A Distributed Log

Kafka is one example

slide-40
SLIDE 40

Think back to the queue example

Batch Batch

slide-41
SLIDE 41

Shard on the way in

slide-42
SLIDE 42

Each shard is a queue

Strong Ordering (in shard). Good concurrency.

slide-43
SLIDE 43

Each consuming service is assigned a “personal set” of queues

each little queue is sent to only one service in a group

slide-44
SLIDE 44

Services instances naturally rebalance on failure

Service instance dies, data is redirected,

  • rdering guarantees remain
slide-45
SLIDE 45

Very Scalable, Very High Throughput

Sharded In, Sharded Out

slide-46
SLIDE 46

Reduces to a globally

  • rdered queue
slide-47
SLIDE 47

Fault Tolerance

slide-48
SLIDE 48

The Log

Single seek & scan Append

  • nly

messages don’t need to be transient!

slide-49
SLIDE 49

Cleaning the Log

Delete old segments

slide-50
SLIDE 50

Cleaning the Log

Delete old versions that share the same key

K1 K1 K1 K2 K2 K2 K1 V1 V1 V2 V3 V2 V4 V3

slide-51
SLIDE 51
  • Scalable multiprocessing
  • Strong partition-based ordering
  • Efficient data retention
  • Always on
slide-52
SLIDE 52

So how is this useful for microservices?

slide-53
SLIDE 53

Build ‘Always On’ Services

Rely on Fault Tolerant Broker

slide-54
SLIDE 54

Load Balance Services

Load Balance Services (with strong ordering)

slide-55
SLIDE 55

Fault Tolerant Services

Services automatically fail over (retaining ordering)

slide-56
SLIDE 56

Services can return back to

  • ld messages in the log

Rewind & Replay

slide-57
SLIDE 57

Compacted Topics are Interesting

K1 K1 K1 K2 K2 K2 K1 V1 V1 V2 V3 V2 V4 V3

slide-58
SLIDE 58

Lets take a little example

slide-59
SLIDE 59

Getting Exchange Rates

Exchange Rate Service USD/GBP = 0.71 EUR/GBP = 0.77 USD/INR = 67.7 USD/AUD = 1.38 EUR/JPY = 114.41 … I need exchange rates!

slide-60
SLIDE 60

Option1: Request Response

rate for USD/GBP? 0.71 Exchange Rate Service I need exchange rates!

slide-61
SLIDE 61

Option 2: Publish Subscribe

Exchange Rate Service Accumulate current state ETL I need exchange rates!

slide-62
SLIDE 62

Option 3: Accumulate in Compacted Stream

Exchange Rate Service Get all exchange rates Publish to clients

USD/GBP = 0.71 EUR/GBP = 0.77 USD/INR = 67.7 USD/AUD = 1.38 EUR/JPY = 114.41 …

Broker retains latest versions Publish all rate events

slide-63
SLIDE 63

Is it a stream or is it a table?

transitory stateful

slide-64
SLIDE 64

Datasets can live in the broker!

trades books risk results

ex- rates

slide-65
SLIDE 65

Service Backbone

Scalable, Fault Tolerant, Concurrent, Strongly Ordered, Stateful

slide-66
SLIDE 66

… lets add in stream processing

slide-67
SLIDE 67

Max(price) From orders where ccy=‘GBP’

  • ver 1 day window

emitting every second

What is stream processing?

Continuous Queries.

slide-68
SLIDE 68

What is stream processing engine?

Data

Index

Query Engine

Query Engine

vs

Database

Finite, well defined source

Stream Processor

Infinite, poorly defined source

slide-69
SLIDE 69

Windowing

For unordered or unpredictable streams Sliding Fixed (tumbling)

slide-70
SLIDE 70

Features: similar to database query engine

Join Filter Aggr- egate View Window

slide-71
SLIDE 71

KStreams & KTables

stream

Compacted stream

Join

Streaming Data Stored Data KStream KTable

slide-72
SLIDE 72

A little example…

slide-73
SLIDE 73

Buying Lunch Abroad

Payments Service

Exchange Rates Service

Buy

Notification Service

Amount in ££

$$ $$ Text Message: ££ $$

slide-74
SLIDE 74

Request-Response Option

Payments Service

Exchange Rates Service

Buy

Amount in ££ Join etc

Text Message: ££

Iterative join

  • ver the network
slide-75
SLIDE 75

ETL Option

Payments Service

Exchange Rates Service

Buy

Amount in ££ ETL ETL Join etc

Text Message: ££

slide-76
SLIDE 76

Stream Processor Option

Payments Service

Exchange Rates Service

Buy

Stream Processor join etc

Text Message: ££

slide-77
SLIDE 77

Buying Lunch Abroad

Payments Exchange Rates

Looks like a table (compacted stream) Looks like an infinite stream

KStream

KTable

slide-78
SLIDE 78

Buying Lunch Abroad

Payments Exchange Rates

  • Filter(ccy<>’GBP’)
  • Join on ccy
  • Calculate GBP
  • Send text message

buffering

slide-79
SLIDE 79

Local DB (fast joins)

Topic

Compacted Topic

KStream pre-populate

slide-80
SLIDE 80

KTables can also be written to

  • they’re backed by the broker

Manage intermediary state

KStream

KTable

Topic

Compacted Topic

slide-81
SLIDE 81

Scales Out (MPP)

slide-82
SLIDE 82

These tools are pretty handy

for managing decentralised services

slide-83
SLIDE 83

Talk our own data model

Data Stream View

Query

slide-84
SLIDE 84

Handle Unpredictability

9am 5pm Late trades

slide-85
SLIDE 85

Joining Services

Payments

Exchange Rates

Join

slide-86
SLIDE 86

Duality between Stream and Table

Join

KStream

KTable

slide-87
SLIDE 87

More Complex Use Cases

Trades Valuations

Books Customers

General Ledger

slide-88
SLIDE 88

trades

books

risk results

ex- rates

Practical mechanism for managing data intensive, loosely coupled services

  • Stateful streams live

inside the Log

  • Data extracted quickly!
  • Fast, local joins, over

large datasets

  • HA pre-caching
  • Manage intermediary

state

  • Just a simple library

(over Kafka)

slide-89
SLIDE 89

There is much more to stream processing

it is grounded in the world of big-data analytics

slide-90
SLIDE 90

Simple Approaches

Just a library (over Kafka)

slide-91
SLIDE 91

Keeping Services Consistent

slide-92
SLIDE 92

Big Global Bag of State in the Sky

Problem: No BGBSS

slide-93
SLIDE 93

How to you provide the accuracy of this

slide-94
SLIDE 94

In this?

slide-95
SLIDE 95

Centralised vs Federated

Centralised consistency model Distributed consistency model

slide-96
SLIDE 96

One problem is failure

slide-97
SLIDE 97

Duplicate messages are inevitable

have I seen this before?

slide-98
SLIDE 98

Make Services Idempotent

try 1 try 2 try 3 try 4

slide-99
SLIDE 99

Stream processors have to solve this problem

slide-100
SLIDE 100

Exactly Once

not available in Kafka… yet

slide-101
SLIDE 101

So what do we have?

slide-102
SLIDE 102

Use Both Approaches

Event- Based Request/ Response

slide-103
SLIDE 103

Queued Delivery System

Ordered queue

slide-104
SLIDE 104

Scales Horizontally

slide-105
SLIDE 105

Scales Horizontally

slide-106
SLIDE 106

Scales Horizontally

slide-107
SLIDE 107

Scales Horizontally

slide-108
SLIDE 108

Built In Fault Tolerance

slide-109
SLIDE 109

Runs Always On

slide-110
SLIDE 110

For Services Too

slide-111
SLIDE 111

Scales Horizontally

Load Balance

slide-112
SLIDE 112

continue through failure

slide-113
SLIDE 113

Scales Horizontally

with history stored in the Log

slide-114
SLIDE 114

Scales Horizontally

Extending to any number of services

slide-115
SLIDE 115

Scales Horizontally

With any data throughput

slide-116
SLIDE 116

Scales Horizontally

With any data throughput

slide-117
SLIDE 117

Scales Horizontally

With any data throughput

slide-118
SLIDE 118

Scales Horizontally

powerful tools for slicing and dicing streams

slide-119
SLIDE 119

Scales Horizontally

the declarative processing of data

join filter aggregate

slide-120
SLIDE 120

at any throughput

slide-121
SLIDE 121

Scales Horizontally

leveraging fast local persistence

slide-122
SLIDE 122

Scales Horizontally

backed up to the log

slide-123
SLIDE 123

Scales Horizontally

easily join streaming services

slide-124
SLIDE 124

Blend KStreams and KTables

slide-125
SLIDE 125

trades books risk results

ex- rates

with data living in the stream

slide-126
SLIDE 126

but retaining loose coupling

trades books risk results

ex- rates

slide-127
SLIDE 127

Scales Horizontally

with strong ordering and repeatability guarantees (eventually)

slide-128
SLIDE 128

so…

slide-129
SLIDE 129

Microservices push us away from shared, mutable state

slide-130
SLIDE 130

Big Global Bag of State in the Sky

Away from BGBSS’s

slide-131
SLIDE 131

This means data is increasingly remote

slide-132
SLIDE 132

Sure, you can collect it all

copy copy copy copy copy copy copy ETL ETL ETL ETL ETL ETL

slide-133
SLIDE 133

can be a lot of work

slide-134
SLIDE 134

Or you can look it all up

get get get get get get get

get, get, get, get

slide-135
SLIDE 135

but that doesn’t scale well

(with system complexity or with data throughput)

slide-136
SLIDE 136

Better to embrace decentralistion

slide-137
SLIDE 137

We need a decentralised toolset to do this

trades books risk results

ex- rates

slide-138
SLIDE 138

Keep it simple, Keep it moving