DEPLOYING AND SCALING MICROSERVICES Sam Newman Goto Chicago 2016 - - PowerPoint PPT Presentation

▶
deploying and scaling microservices
SMART_READER_LITE
LIVE PREVIEW

DEPLOYING AND SCALING MICROSERVICES Sam Newman Goto Chicago 2016 - - PowerPoint PPT Presentation

Dec 22, 2022 456 likes •2.37k views

DEPLOYING AND SCALING MICROSERVICES Sam Newman Goto Chicago 2016 @samnewman Building Microservices DESIGNING FINE - GRAINED SYSTEMS Sam Newman @samnewman Core Principles @samnewman Core Principles Artifacts @samnewman Core Principles

slide-1
SLIDE 1

DEPLOYING AND SCALING MICROSERVICES

Sam Newman Goto Chicago 2016

slide-2
SLIDE 2

@samnewman

slide-3
SLIDE 3

@samnewman

Sam Newman

Building Microservices

DESIGNING FINE-GRAINED SYSTEMS

slide-4
SLIDE 4

@samnewman

Core Principles

slide-5
SLIDE 5

@samnewman

Artifacts Core Principles

slide-6
SLIDE 6

@samnewman

Artifacts Core Principles Platforms

slide-7
SLIDE 7

@samnewman

Artifacts Core Principles Platforms

slide-8
SLIDE 8

@samnewman

Accounts Returns Invoicing Shipping Inventory Customer Service

slide-9
SLIDE 9

@samnewman

Independent Deployability

slide-10
SLIDE 10

@samnewman

Accounts Returns v345 Invoicing Inventory Customer Service Shipping v123

slide-11
SLIDE 11

@samnewman

Accounts Invoicing Shipping v456 Inventory Customer Service Returns v890

slide-12
SLIDE 12

@samnewman

https://www.flickr.com/photos/torkildr/3462607995/

slide-13
SLIDE 13

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

slide-14
SLIDE 14

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

slide-15
SLIDE 15

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

slide-16
SLIDE 16

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

slide-17
SLIDE 17

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

slide-18
SLIDE 18

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

slide-19
SLIDE 19

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

slide-20
SLIDE 20

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

slide-21
SLIDE 21

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

  • One Artifact For All Environments
slide-22
SLIDE 22

@samnewman

Tests Build Performance Prod UAT https://www.flickr.com/

Source Control

  • One Artifact For All Environments

Same Deployment Process Everywhere

slide-23
SLIDE 23

@samnewman

$ deploy Returns v456 Production

slide-24
SLIDE 24

@samnewman

$ deploy Returns v456 Production

Service Name

slide-25
SLIDE 25

@samnewman

$ deploy Returns v456 Production

Service Name Version

slide-26
SLIDE 26

@samnewman

$ deploy Returns v456 Production

local Service Name Version

slide-27
SLIDE 27

@samnewman

$ deploy Returns v456 Production

local latest Service Name Version

slide-28
SLIDE 28

@samnewman

$ deploy Returns v456 Production

local latest Service Name Version Environment

slide-29
SLIDE 29

@samnewman

Tests Build Large Tests UAT Prod

DB Machine UAT Environment Machine

Perf

slide-30
SLIDE 30

@samnewman

Tests Build UAT Perf Prod

Master DB Machine Production Environment Machine Machine Machine Slave DB

slide-31
SLIDE 31

@samnewman

Same Artifact

slide-32
SLIDE 32

@samnewman

Same Artifact Different Topology

slide-33
SLIDE 33

@samnewman

Core Principles?

slide-34
SLIDE 34

@samnewman

Independent Deployability Core Principles?

slide-35
SLIDE 35

@samnewman

Independent Deployability One Artifact For All Environments Core Principles?

slide-36
SLIDE 36

@samnewman

Independent Deployability One Artifact For All Environments Same Deployment Process Everywhere Core Principles?

slide-37
SLIDE 37

@samnewman

Artifacts Core Principles Platforms

slide-38
SLIDE 38

@samnewman

What do we want from an artifact?

slide-39
SLIDE 39

@samnewman

What do we want from an artifact?

slide-40
SLIDE 40

@samnewman

What do we want from an artifact? Easy to create

slide-41
SLIDE 41

@samnewman

What do we want from an artifact? Easy to deploy Easy to create

slide-42
SLIDE 42

@samnewman

What do we want from an artifact? Easy to deploy Abstract out the tech stack Easy to create

slide-43
SLIDE 43

@samnewman

What do we want from an artifact? Easy to deploy Abstract out the tech stack Good for dev, good for ops Easy to create

slide-44
SLIDE 44

@samnewman

Tarballs

slide-45
SLIDE 45

@samnewman

Tarballs

Giant bundles of stuff

slide-46
SLIDE 46

@samnewman

Tarballs Easy to create Easy to deploy Abstract out the tech stack Good for dev, good for ops

slide-47
SLIDE 47

@samnewman

Tarballs Easy to create

  • Easy to deploy

Abstract out the tech stack Good for dev, good for ops

slide-48
SLIDE 48

@samnewman

Tarballs Easy to create

  • Easy to deploy
  • Abstract out the tech stack

Good for dev, good for ops

slide-49
SLIDE 49

@samnewman

Tarballs Easy to create

  • Easy to deploy
  • Abstract out the tech stack
  • Good for dev, good for ops
slide-50
SLIDE 50

@samnewman

Tarballs Easy to create

  • Easy to deploy
  • Abstract out the tech stack
  • Good for dev, good for ops
slide-51
SLIDE 51

@samnewman

Stack-specific

slide-52
SLIDE 52

@samnewman

Stack-specific

nuget jar pip gems

slide-53
SLIDE 53

@samnewman

Stack-specific Easy to create Abstract out the tech stack Easy to deploy Good for dev, good for ops

slide-54
SLIDE 54

@samnewman

Stack-specific Easy to create

  • Abstract out the tech stack

Easy to deploy Good for dev, good for ops

slide-55
SLIDE 55

@samnewman

Stack-specific Easy to create

  • Abstract out the tech stack

Easy to deploy

  • Good for dev, good for ops
slide-56
SLIDE 56

@samnewman

Stack-specific Easy to create

  • Abstract out the tech stack
  • Easy to deploy
  • Good for dev, good for ops
slide-57
SLIDE 57

@samnewman

Stack-specific Easy to create

  • Abstract out the tech stack
  • Easy to deploy
  • Good for dev, good for ops
slide-58
SLIDE 58

@samnewman

Everything in Go is Awesome

slide-59
SLIDE 59

@samnewman

Everything in Go is Awesome FACT

slide-60
SLIDE 60

@samnewman

OS-Specific

slide-61
SLIDE 61

@samnewman

OS-Specific

$ sudo apt-get install myservice

slide-62
SLIDE 62

@samnewman

OS-Specific

$ sudo apt-get install myservice $ deploy Returns v456 Production

slide-63
SLIDE 63

@samnewman

OS-Specific Easy to deploy Easy to create Abstract out the tech stack Good for dev, good for ops

slide-64
SLIDE 64

@samnewman

OS-Specific Easy to deploy Easy to create

  • Abstract out the tech stack

Good for dev, good for ops

slide-65
SLIDE 65

@samnewman

OS-Specific Easy to deploy

  • Easy to create
  • Abstract out the tech stack

Good for dev, good for ops

slide-66
SLIDE 66

@samnewman

OS-Specific Easy to deploy

  • Easy to create
  • Abstract out the tech stack
  • Good for dev, good for ops
slide-67
SLIDE 67

@samnewman

OS-Specific Easy to deploy

  • Easy to create
  • Abstract out the tech stack
  • Good for dev, good for ops
slide-68
SLIDE 68

@samnewman

Build

slide-69
SLIDE 69

@samnewman

Build Deb Repo

slide-70
SLIDE 70

@samnewman

Host Build Deb Repo

slide-71
SLIDE 71

@samnewman

Host Build Deb Repo

slide-72
SLIDE 72

@samnewman

Host Build Deb Repo

slide-73
SLIDE 73

@samnewman

slide-74
SLIDE 74

@samnewman

A v1.5

slide-75
SLIDE 75

@samnewman

A v2.1 A v1.5

slide-76
SLIDE 76

@samnewman

A v2.1 A v1.5

slide-77
SLIDE 77

@samnewman

Host

slide-78
SLIDE 78

@samnewman

Host

slide-79
SLIDE 79

@samnewman

Host Host Host Host

slide-80
SLIDE 80

@samnewman

Host Host Host Host

Independent Execution Environments FTW!

slide-81
SLIDE 81

@samnewman

Custom Images Easy to deploy Easy to create Abstract out the tech stack Good for dev, good for ops

slide-82
SLIDE 82

@samnewman

Custom Images Easy to deploy Easy to create

  • Abstract out the tech stack

Good for dev, good for ops

slide-83
SLIDE 83

@samnewman

Custom Images Easy to deploy

  • Easy to create
  • Abstract out the tech stack

Good for dev, good for ops

slide-84
SLIDE 84

@samnewman

Custom Images Easy to deploy

  • Easy to create
  • Abstract out the tech stack

Good for dev, good for ops

slide-85
SLIDE 85

@samnewman

Custom Images Easy to deploy

  • Easy to create
  • Abstract out the tech stack

Good for dev, good for ops

slide-86
SLIDE 86

@samnewman

slide-87
SLIDE 87

@samnewman

slide-88
SLIDE 88

@samnewman

Cost of isolated hosts is reduced…

slide-89
SLIDE 89

@samnewman

Cost of isolated hosts is reduced… …in terms of effort…

slide-90
SLIDE 90

@samnewman

Cost of isolated hosts is reduced… …in terms of effort… …and computing resources

slide-91
SLIDE 91

@samnewman

Docker! Easy to deploy Abstract out the tech stack Good for dev, good for ops Easy to create

slide-92
SLIDE 92

@samnewman

Docker! Easy to deploy Abstract out the tech stack Good for dev, good for ops Easy to create

slide-93
SLIDE 93

@samnewman

Docker! Easy to deploy Abstract out the tech stack Good for dev, good for ops Easy to create

slide-94
SLIDE 94

@samnewman

Docker! Easy to deploy Abstract out the tech stack Good for dev, good for ops Easy to create

slide-95
SLIDE 95

@samnewman

Docker! Easy to deploy Abstract out the tech stack Good for dev, good for ops Easy to create

slide-96
SLIDE 96

@samnewman

Artifacts Core Principles Platforms

slide-97
SLIDE 97

@samnewman

What do we want from our deployment platform?

slide-98
SLIDE 98

@samnewman

What do we want from our deployment platform?

Separate artifact from topology

slide-99
SLIDE 99

@samnewman

What do we want from our deployment platform?

Separate artifact from topology Makes handling lots of services easy!

slide-100
SLIDE 100

@samnewman

What do we want from our deployment platform?

Separate artifact from topology Makes handling lots of services easy! Supports docker images

slide-101
SLIDE 101

@samnewman

Deployment Platforms?

slide-102
SLIDE 102

@samnewman

Docker Swarm

slide-103
SLIDE 103

@samnewman

Docker Swarm

slide-104
SLIDE 104

@samnewman

Docker Swarm

slide-105
SLIDE 105

@samnewman

Docker Swarm

slide-106
SLIDE 106

@samnewman

DOCKER SWARM

slide-107
SLIDE 107

@samnewman

DOCKER SWARM Swarm Manager

slide-108
SLIDE 108

@samnewman

DOCKER SWARM Swarm Node Swarm Node Swarm Node Swarm Manager

slide-109
SLIDE 109

@samnewman

DOCKER SWARM Swarm Node Swarm Node Swarm Node Swarm Manager

slide-110
SLIDE 110

@samnewman

DOCKER SWARM Swarm Node Swarm Node Swarm Node Swarm Manager $ docker …

slide-111
SLIDE 111

@samnewman

DOCKER SWARM Swarm Node Swarm Node Swarm Node Swarm Manager $ docker …

slide-112
SLIDE 112

@samnewman

DOCKER SWARM Swarm Node Swarm Node Swarm Node Swarm Manager $ docker …

slide-113
SLIDE 113

@samnewman

DOCKER SWARM Swarm Node Swarm Node Swarm Node Swarm Manager $ docker …

slide-114
SLIDE 114

@samnewman

SCHEDULING STRATEGIES - BINPACK Swarm Node Swarm Node Swarm Node Swarm Manager

slide-115
SLIDE 115

@samnewman

SCHEDULING STRATEGIES - BINPACK Swarm Node Swarm Node Swarm Node Swarm Manager

slide-116
SLIDE 116

@samnewman

SCHEDULING STRATEGIES - SPREAD Swarm Node Swarm Node Swarm Node Swarm Manager

slide-117
SLIDE 117

@samnewman

SCHEDULING STRATEGIES - SPREAD Swarm Node Swarm Node Swarm Node Swarm Manager

slide-118
SLIDE 118

@samnewman

DOCKER COMPOSE

web: build: . ports:

  • "5000:5000"

volumes:

  • .:/code

links:

  • redis

redis: image: redis

slide-119
SLIDE 119

@samnewman

slide-120
SLIDE 120

@samnewman

Doesn’t rebalance

slide-121
SLIDE 121

@samnewman

Doesn’t rebalance Doesn’t restart failed containers

slide-122
SLIDE 122

@samnewman

Doesn’t rebalance Doesn’t restart failed containers Does

slide-123
SLIDE 123

@samnewman

Doesn’t rebalance Doesn’t restart failed containers Does Does

slide-124
SLIDE 124

@samnewman

Doesn’t rebalance Doesn’t restart failed containers Plays nice with the rest of docker Does Does

slide-125
SLIDE 125

@samnewman

Doesn’t rebalance Doesn’t restart failed containers Plays nice with the rest of docker Case studies thin on the ground Does Does

slide-126
SLIDE 126

@samnewman

slide-127
SLIDE 127

@samnewman

Mesos Master MESOS

slide-128
SLIDE 128

@samnewman

Mesos Master MESOS

slide-129
SLIDE 129

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS

slide-130
SLIDE 130

@samnewman

Frameworks!

slide-131
SLIDE 131

@samnewman

slide-132
SLIDE 132

@samnewman

Frameworks

slide-133
SLIDE 133

@samnewman

Frameworks Scheduler

slide-134
SLIDE 134

@samnewman

Frameworks Scheduler Executor

slide-135
SLIDE 135

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS

slide-136
SLIDE 136

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS Hadoop Scheduler

slide-137
SLIDE 137

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS Hadoop Executor Hadoop Scheduler

slide-138
SLIDE 138

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS Hadoop Executor Hadoop Scheduler

slide-139
SLIDE 139

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS Hadoop Executor Hadoop Scheduler Hadoop

slide-140
SLIDE 140

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS Hadoop Executor Hadoop Scheduler Hadoop Marathon Marathon

slide-141
SLIDE 141

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS Hadoop Executor Hadoop Scheduler Hadoop Marathon Marathon Marathon Scheduler

slide-142
SLIDE 142

@samnewman

Mesos Master Mesos Agent Mesos Agent Mesos Agent MESOS Hadoop Executor Hadoop Scheduler Hadoop Marathon Marathon Marathon Scheduler

slide-143
SLIDE 143

@samnewman

OTHER FRAMEWORKS… https://github.com/alde/eremetic

slide-144
SLIDE 144

@samnewman

AWS Lambda

slide-145
SLIDE 145

@samnewman

slide-146
SLIDE 146

@samnewman

Great if you need to run other workloads

slide-147
SLIDE 147

@samnewman

Great if you need to run other workloads Really powerful - fully featured, widely used

slide-148
SLIDE 148

@samnewman

Great if you need to run other workloads Really powerful - fully featured, widely used Fairly complex - lots of moving parts

slide-149
SLIDE 149

@samnewman

slide-150
SLIDE 150

@samnewman

slide-151
SLIDE 151

@samnewman

KUBERNETES ARCHITECTURE…SORT OF

slide-152
SLIDE 152

@samnewman

API Server KUBERNETES ARCHITECTURE…SORT OF

slide-153
SLIDE 153

@samnewman

API Server KUBERNETES ARCHITECTURE…SORT OF

slide-154
SLIDE 154

@samnewman

API Server Kubelet Kubelet Kubelet KUBERNETES ARCHITECTURE…SORT OF

slide-155
SLIDE 155

@samnewman

API Server Kubelet Kubelet Kubelet $ kubectl … KUBERNETES ARCHITECTURE…SORT OF

slide-156
SLIDE 156

@samnewman

API Server Kubelet Kubelet Kubelet $ kubectl … KUBERNETES ARCHITECTURE…SORT OF

slide-157
SLIDE 157

@samnewman

PODS?

slide-158
SLIDE 158

@samnewman

A collection of tightly coupled containers, running on one node

PODS?

slide-159
SLIDE 159

@samnewman

A collection of tightly coupled containers, running on one node

PODS?

Can have metadata, volumes too

slide-160
SLIDE 160

@samnewman

A collection of tightly coupled containers, running on one node

PODS?

Can have metadata, volumes too Pods are mortal - not long running!

slide-161
SLIDE 161

@samnewman

A collection of tightly coupled containers, running on one node

PODS?

Can have metadata, volumes too Pods are mortal - not long running!

A pod = a unit of scheduling

slide-162
SLIDE 162

@samnewman

SERVICES!

slide-163
SLIDE 163

@samnewman

SERVICES!

{ "kind": "Service", "apiVersion": "v1", "metadata": { "name": "my-service" }, "spec": { "selector": { "app": "MyApp" }, "ports": [ { "protocol": "TCP", "port": 80, "targetPort": 9376 } ] } }

slide-164
SLIDE 164

@samnewman

SERVICES!

A mapping of metadata and ports to a set of pods

{ "kind": "Service", "apiVersion": "v1", "metadata": { "name": "my-service" }, "spec": { "selector": { "app": "MyApp" }, "ports": [ { "protocol": "TCP", "port": 80, "targetPort": 9376 } ] } }

slide-165
SLIDE 165

@samnewman

Kubelet

slide-166
SLIDE 166

@samnewman

Kubelet

slide-167
SLIDE 167

@samnewman

Kubelet

slide-168
SLIDE 168

@samnewman

Kubelet Service Proxy

slide-169
SLIDE 169

@samnewman

Kubelet Service Proxy :80

slide-170
SLIDE 170

@samnewman

You don’t scale a service…

Kubelet Service Proxy :80

slide-171
SLIDE 171

@samnewman

You don’t scale a service… …you scale the pods!

Kubelet Service Proxy :80

slide-172
SLIDE 172

@samnewman

slide-173
SLIDE 173

@samnewman

Simpler to setup than Mesos - but more single purpose too

slide-174
SLIDE 174

@samnewman

Simpler to setup than Mesos - but more single purpose too Closer to a PAAS

slide-175
SLIDE 175

@samnewman

Simpler to setup than Mesos - but more single purpose too Closer to a PAAS Pods can be confusing!

slide-176
SLIDE 176

@samnewman

Simpler to setup than Mesos - but more single purpose too Closer to a PAAS Fairly new, but a growing amount of impressive support Pods can be confusing!

slide-177
SLIDE 177

@samnewman

Docker Swarm

slide-178
SLIDE 178

@samnewman

Docker Swarm

slide-179
SLIDE 179

@samnewman

Docker Swarm

slide-180
SLIDE 180

@samnewman

Docker Swarm

slide-181
SLIDE 181

@samnewman

slide-182
SLIDE 182

@samnewman

Core Principles

slide-183
SLIDE 183

@samnewman

Core Principles Independent Deployability

slide-184
SLIDE 184

@samnewman

Core Principles One Artifact For All Environments Independent Deployability

slide-185
SLIDE 185

@samnewman

Core Principles One Artifact For All Environments Independent Deployability Same Deployment Process

slide-186
SLIDE 186

@samnewman

Core Principles One Artifact For All Environments Independent Deployability Same Deployment Process Docker Images As Artifacts

slide-187
SLIDE 187

@samnewman

Core Principles One Artifact For All Environments Independent Deployability Same Deployment Process Docker Images As Artifacts Criteria For Selecting A Platform

slide-188
SLIDE 188

@samnewman

slide-189
SLIDE 189

@samnewman

Sam Newman

Building Microservices

DESIGNING FINE-GRAINED SYSTEMS

AUTHD

http://samnewman.io/podcast

slide-190
SLIDE 190

@samnewman snewman@thoughtworks.com

THANKS!