INCREMENTALISM An Industrial Strategy For Adopting Modern - - PowerPoint PPT Presentation

incrementalism
SMART_READER_LITE
LIVE PREVIEW

INCREMENTALISM An Industrial Strategy For Adopting Modern - - PowerPoint PPT Presentation

INCREMENTALISM An Industrial Strategy For Adopting Modern Automation BACKGROUND MONOCULTURE RDBMS ARE RAD TROUBLEMAKER OPERATOR AND ENGINEER BACKGROUND MONOCULTURE RDBMS ARE RAD TROUBLEMAKER OPERATOR AND ENGINEER Industrial Techno


slide-1
SLIDE 1

ç

INCREMENTALISM

An Industrial Strategy For Adopting Modern Automation

slide-2
SLIDE 2

MONOCULTURE TROUBLEMAKER RDBMS ARE RAD OPERATOR AND ENGINEER

BACKGROUND

slide-3
SLIDE 3

MONOCULTURE TROUBLEMAKER RDBMS ARE RAD OPERATOR AND ENGINEER

BACKGROUND

slide-4
SLIDE 4

Industrial Techno Revolution Development and Operational Practices

slide-5
SLIDE 5
  • 1. Development Practices
  • 2. Secrets Management
  • 3. Packaging
  • 4. Developer-centric, Self-Healing Applications
  • 5. Data Center Aware Services
  • 6. Infrastructure Manipulation
slide-6
SLIDE 6
slide-7
SLIDE 7

YOU ARE A BIT-CHUCKING TECHNO INDUSTRIALIST.

slide-8
SLIDE 8
slide-9
SLIDE 9

DATA CENTERS ARE YOUR FACTORIES. NETWORKS ARE YOUR ROADS. YOU APP PRODUCES WIDGETS. MICROSERVICES ARE YOUR ROBOTS ON THE FACTORY LINE.

slide-10
SLIDE 10

YOU ARE A BIT-CHUCKING TECHNO INDUSTRIALIST. WIDGET SHIPPING ^

slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13
slide-14
SLIDE 14

REDUNDANCY

slide-15
SLIDE 15

CRITICAL INFRASTRUCTURE

slide-16
SLIDE 16
slide-17
SLIDE 17

PROFILE MESSAGES UPSELL CART RELATED COMMENTS SOCIAL

HTML/JSON/gRPC Response

slide-18
SLIDE 18

Response Microservices

slide-19
SLIDE 19

HTML/JSON/gRPC Response MY APP

slide-20
SLIDE 20
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback
slide-21
SLIDE 21
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback

BUSINESS VALUE

slide-22
SLIDE 22
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback

INPUT

slide-23
SLIDE 23
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback

DEPENDENCY

slide-24
SLIDE 24
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback

PROPERTIES OF
 THE APPLICATION

slide-25
SLIDE 25
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback

VALUE

slide-26
SLIDE 26
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback

DISTRACTING, REQUIRED, NECESSARY COMPLEXITY

slide-27
SLIDE 27
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback

WIDGET

slide-28
SLIDE 28
  • New program!
  • Use a piece of user data
  • Talks to a database
  • Returns something useful
  • Highly-Available
  • Self-Healing
  • Feedback

INDUSTRIAL COMPUTING

RAW MATERIAL REDUNDANT ROBOTS REPLACEABLE PARTS QUALITY-MANAGEMENT WIDGET STORAGE

slide-29
SLIDE 29

MY RESULT MY APP

slide-30
SLIDE 30
slide-31
SLIDE 31
slide-32
SLIDE 32

Codified Development Environment Reproducible Dev Environments Disposable R&D Workspace Shared Developer Workspace Developer-driven Infrastructure

slide-33
SLIDE 33

Terminal

$ $EDITOR Vagrantfile Vagrant.configure("2") do |config| config.vm.box = "ubuntu/xenial64" config.vm.box_url = "https://cloud-images.ubuntu.com/ xenial/current/xenial-server-cloudimg-amd64-vagrant.box" config.vm.network "private_network", ip: "192.168.33.10" config.vm.provision "shell", path: "setup.sh" end $ cat setup.sh apt-get install postgresql-server tmux

slide-34
SLIDE 34

Terminal

my-laptop$ vagrant up --destroy-on-error my-laptop$ vagrant ssh vm$ uname -a | tee /vagrant/uname.out Linux compton 2.6.24-19-server #1 SMP Sat Jul 12 00:40:01 UTC 2008 i686 GNU/Linux vm$ logout Shared connection to 192.168.39.130 closed. my-laptop$ cat uname.out Linux compton 2.6.24-19-server #1 SMP Sat Jul 12 00:40:01 UTC 2008 i686 GNU/Linux

slide-35
SLIDE 35

Terminal

$ $EDITOR Vagrantfile Vagrant.configure("2") do |config| config.ssh.shell = "sh" config.vm.synced_folder ".", "/vagrant", nfs: true, id: "vagrant-root" end

slide-36
SLIDE 36

Terminal

my-laptop$ $EDITOR myapp.go my-laptop$ GOOS=linux GOARCH=amd64 go build -o myapp-linux my-laptop$ GOOS=freebsd GOARCH=amd64 go build -o myapp-freebsd my-laptop$ vagrant ssh vm$ /vagrant/myapp-linux vm$ logout my-laptop$ vagrant up freebsd-vm1 my-laptop$ vagrant ssh freebsd-vm1 freebsd-vm1$ /vagrant/myapp-freebsd freebsd-vm1$ logout

slide-37
SLIDE 37

$HOME/go/src/github.com/hashicorp/myapp/Vagrantfile

slide-38
SLIDE 38
slide-39
SLIDE 39

Secret Sprawl Break Glass Procedures Audit Logs Secrets Lifecycle Management

slide-40
SLIDE 40

"I want to deploy this app to prod. Password-less logins are disabled on the databases! Now what?"

slide-41
SLIDE 41

"I want to deploy this app to prod. Password-less logins are disabled on the databases! Now what?"

Private GitHub repo Commit passwords inline in SCM Switch creds based on $HOSTNAME? Establish a protocol for acquiring credentials at runtime

slide-42
SLIDE 42
slide-43
SLIDE 43
slide-44
SLIDE 44
slide-45
SLIDE 45
slide-46
SLIDE 46

Terminal

$ vault read postgresql/creds/readonly Key Value

  • lease_id

postgresql/creds/readonly/5fec46f2-ab40-d9b8-61a2-887c7946eeb6 lease_duration 1h0m0s lease_renewable true password f8a93086-b11d-10cd-8795-f537a10de712 username token-9e57c18f-ac99-8e29-48f2-3fb09066d2b4

slide-47
SLIDE 47

Terminal

$ VAULT_ADDR=http://vault.service.consul vault read postgresql/creds/readonly Key Value

  • lease_id

postgresql/creds/readonly/5fec46f2-ab40-d9b8-61a2-887c7946eeb6 lease_duration 1h0m0s lease_renewable true password f8a93086-b11d-10cd-8795-f537a10de712 username token-9e57c18f-ac99-8e29-48f2-3fb09066d2b4

slide-48
SLIDE 48

Terminal

$ psql -U postgres psql (9.6.1) Type "help" for help. postgres=# \du Role name | Attributes | Member of

  • ------------------------------------------+--------------------------+------------

postgres | Superuser, Create ... | {} token-9e57c18f-ac99-8e29-48f2-3fb09066d2b4 | Password valid until ... | {}

slide-49
SLIDE 49

Terminal

$ vault renew postgresql/creds/readonly/5fec46f2-ab40-d9b8-61a2-887c7946eeb6 Key Value

  • lease_id

postgresql/creds/readonly/5fec46f2-ab40-d9b8-61a2-887c7946eeb6 lease_duration 1h0m0s lease_renewable true

slide-50
SLIDE 50

Terminal

$ cat myapp-db-config.yml.ctmpl

  • {{- with secret "postgresql/creds/readonly" }}

username: "{{ .Data.username }}" password: "{{ .Data.password }}" database: "myapp" {{- end }} $ consul-template -template="myapp-db-config.yml.ctmpl:myapp-db-config.yml" ./myapp <CTRL+C> Received interrupt, cleaning up...

slide-51
SLIDE 51

Terminal

$ vault write postgresql/roles/readonly \ sql="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" Success! Data written to: postgresql/roles/readonly

slide-52
SLIDE 52

Terminal

$ tee my-policy.vault | vault write postgresql/roles/readonly sql=- CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}"; Success! Data written to: postgresql/roles/readonly

slide-53
SLIDE 53

v2.0.0

slide-54
SLIDE 54

Codified Build Environment Reproducible Packaging Shared Packaging Instructions Developer-driven Build and Packaging Steps

slide-55
SLIDE 55

compile.json 1/3

{ "builders": [{ "name": "myapp", "type": "docker", "image": "centos:6", "commit": true, "privileged": true }], "provisioners": [ { "type": "file", "source": "myapp-linux", "destination": "/usr/local/bin/myapp" }, { "type": "file", "source": "local-config-file.repo", "destination": "/usr/local/etc/myapp.conf" }, { "type": "file", "source": "start_myapp.sh", "destination": "/sbin/start_myapp" },

slide-56
SLIDE 56

compile.json 2/3

{ "type": "shell", "inline": [ "/usr/bin/yum -y update", "/usr/bin/yum -y install util-linux-ng patch", "/bin/chmod 0600 /usr/local/etc/myapp.conf", "/bin/chmod 0744 /sbin/start_myapp /usr/local/bin/myapp-linux", "/usr/bin/curl -o /usr/local/etc/some-ca.crt https://host.example.com/pki/ ca.crt", ] } ], "post-processors": [ [ { "type": "docker-tag", "repository": "myorg/myapp" }, { "type": "docker-save", "path": "myapp.tar" },

slide-57
SLIDE 57

compile.json 3/3

{ "type": "artifice", "files": ["myapp.tar"] }, { "type": "compress", "output": "myapp.tar.gz", "compression_level": 9 }, { "type": "atlas", "artifact": "myorg/myapp", "artifact_type": "archive", "metadata": { "created_at": "{{ timestamp }}" } } ] ] }

slide-58
SLIDE 58

compile.json 1/3

{ "builders": [{ "name": "myapp", "type": "docker", "image": "centos:6", "commit": true, "privileged": true }], "provisioners": [ {

"type": "shell", "scripts": [ "setup.sh", "prod-app-script.sh" ] }

slide-59
SLIDE 59

v2.0.0

slide-60
SLIDE 60

v1.1.2 v2.0.0

slide-61
SLIDE 61

What's a Cluster Scheduler?

slide-62
SLIDE 62

redis.job

job "redis" { datacenters = ["asia-east1", "asia-northeast1"] task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 port "redis" {} } } } }

slide-63
SLIDE 63

redis-service.job

job "redis" { datacenters = ["asia-east1", "asia-northeast1"] task "redis" { service { name = "redis" # redis.service.consul port = "redis" check { type = "tcp" interval = "30s" timeout = "2s" } } ...

resources { network { mbits = 10 port "redis" { static = "6379" } } }

slide-64
SLIDE 64

Declare what you want to run

slide-65
SLIDE 65

Scheduler determines where and manages how to run

slide-66
SLIDE 66

v1.1.2

QTY: 1

slide-67
SLIDE 67

Developer-centric Release Management Reproducible Runtime Environments Provider Agnostic Runtime Native Hybrid-Cloud Consumption Model Self-Healing Infrastructure * Service Discovery and Secure Introduction Support

slide-68
SLIDE 68

myapp.job

job "myapp" { region = "apac" datacenters = ["asia-east1", "asia-northeast1"] type = "service" group "myapp" { count = 1 task "api" { driver = "docker" artifact { source = "https://s3.amazonaws.com/myorg/myapp.tar.gz"

  • ptions {

archive = "tar.gz" } } ...

slide-69
SLIDE 69

myapp.job

config { load = ["myapp.tar"] image = "myorg/myapp" command = "/sbin/start_myapp" args = [ "-mode=api" ] network_mode = "host" pid_mode = "host" } service { name = "${TASKGROUP}" # myapp.service.consul tags = [ "api" ] # api.myapp.service.consul port = "api" check { type = "http" path = "/health.txt" interval = "5s" timeout = "2s" } } }

slide-70
SLIDE 70

myapp.job

task "web" { driver = "docker" config { load = ["myapp.tar"] image = "myorg/myapp" command = "/sbin/start_myapp" args = [ "-mode=web" ] network_mode = "host" pid_mode = "host" } service { name = "${TASKGROUP}" # myapp.service.consul tags = [ "web" ] # web.myapp.service.consul port = "web" check { type = "http" path = "/health.txt" interval = "5s" timeout = "2s" } }

slide-71
SLIDE 71

myapp.job

$ nomad plan myapp.job $ nomad run -check-index 12515398 myapp.job

slide-72
SLIDE 72

v1.1.2 v2.0.0

QTY: 0 QTY: 20

slide-73
SLIDE 73
slide-74
SLIDE 74

myapp-green.job

job "myapp-green" { region = "apac" datacenters = ["asia-east1", "asia-northeast1"] type = "service" group "myapp" { count = 19 task "api" { driver = "docker" artifact { source = "https://s3.amazonaws.com/myorg/myapp-v123.tar.gz"

  • ptions {

archive = "tar.gz" } } ...

slide-75
SLIDE 75

myapp-blue.job

job "myapp-blue" { region = "apac" datacenters = ["asia-east1", "asia-northeast1"] type = "service" group "myapp" { count = 1 task "api" { driver = "docker" artifact { source = "https://s3.amazonaws.com/myorg/myapp-v124.tar.gz"

  • ptions {

archive = "tar.gz" } } ...

slide-76
SLIDE 76

myapp-blue.job

$ nomad plan myapp-blue.job + Job: "myapp-blue" + Task Group: "myapp" (1 create) + Task: "api" (forces create) Scheduler dry-run:

  • All tasks successfully allocated.
slide-77
SLIDE 77

100% Green 95% Green 70% Green 90% Blue 100% Blue

green: myapp-v123 blue: myapp-v124

slide-78
SLIDE 78

v1.1.2 v2.0.0

QTY: 0 QTY: 20

slide-79
SLIDE 79

myapp-green.job

job "myapp-green" { region = "apac" datacenters = ["asia-east1", "asia-northeast1"] type = "service" group "myapp" { count = 0 task "api" { driver = "docker" artifact { source = "https://s3.amazonaws.com/myorg/myapp-v123.tar.gz"

  • ptions {

archive = "tar.gz" } } ...

slide-80
SLIDE 80

myapp-blue.job

job "myapp-blue" { region = "apac" datacenters = ["asia-east1", "asia-northeast1"] type = "service" group "myapp" { count = 20 task "api" { driver = "docker" artifact { source = "https://s3.amazonaws.com/myorg/myapp-v124.tar.gz"

  • ptions {

archive = "tar.gz" } } ...

slide-81
SLIDE 81

myapp-blue.job

job "myapp-blue" { region = "apac" datacenters = ["asia-east1", "asia-northeast1"] type = "service" update { # Stagger updates every 120 seconds stagger = "120s" # Update a single task at a time max_parallel = 1 } ...

slide-82
SLIDE 82

myapp-blue.job

$ nomad status myapp-blue ID = myapp-blue Name = myapp-blue Type = service Priority = 50 Datacenters = asia-east1 Status = running Periodic = false Summary Task Group Queued Starting Running Failed Complete Lost myapp 0 0 1 0 0 0 Allocations ID Eval ID Node ID Task Group Desired Status Created At 24cfd201 81efc2fa 8d0331e9 myapp run running 11/11/16 21:03:19 AEDT

slide-83
SLIDE 83

myapp-blue.job

$ nomad alloc-status --verbose a7365fe4 ID = a7365fe4-cb28-a6e9-f3d4-f99e49c89776 Eval ID = c3c9a1db-dbeb-8afa-0a83-4f1b8b5a03f5 Name = myapp-blue.myapp[0] Node ID = 1f029d38-8d4b-a552-261f-e457b60f9b4b Job ID = myapp-blue Client Status = running Created At = 11/11/16 22:04:53 AEDT Evaluated Nodes = 1 Filtered Nodes = 0 Exhausted Nodes = 0 Allocation Time = 1.085001ms Failures = 0 ==> Task Resources Task: "api" CPU Memory MB Disk MB IOPS Addresses 500 256 300 0 db: 127.0.0.1:38537 Task: "web" CPU Memory MB Disk MB IOPS Addresses

slide-84
SLIDE 84

Terminal

$ vault read postgresql/creds/readonly Key Value

  • lease_id

postgresql/creds/readonly/5fec46f2-ab40-d9b8-61a2-887c7946eeb6 lease_duration 1h0m0s lease_renewable true password f8a93086-b11d-10cd-8795-f537a10de712 username token-9e57c18f-ac99-8e29-48f2-3fb09066d2b4

slide-85
SLIDE 85

Terminal

$ env VAULT_TOKEN=.... vault read postgresql/creds/readonly Key Value

  • lease_id

postgresql/creds/readonly/5fec46f2-ab40-d9b8-61a2-887c7946eeb6 lease_duration 1h0m0s lease_renewable true password f8a93086-b11d-10cd-8795-f537a10de712 username token-9e57c18f-ac99-8e29-48f2-3fb09066d2b4

slide-86
SLIDE 86

myapp.job

job "myapp" { region = "apac" datacenters = ["asia-east1", "asia-northeast1"] type = "service" group "myapp" { count = 1 task "api" { driver = "docker" env { VAULT_TOKEN = "7ea47d76-a653-4d43-9507-dbeed3b3747f" } artifact { source = "https://s3.amazonaws.com/myorg/myapp.tar.gz"

  • ptions {

archive = "tar.gz" } }

slide-87
SLIDE 87

myapp.job

job "myapp" { region = "apac" datacenters = ["asia-east1", "asia-northeast1"] type = "service" group "myapp" { count = 1 task "api" { driver = "docker" vault { policies = ["myapp", "api"] change_mode = "signal" change_signal = "SIGUSR1" } artifact { source = "https://s3.amazonaws.com/myorg/myapp.tar.gz"

  • ptions {

archive = "tar.gz" }

slide-88
SLIDE 88

Containerized Virtualized Standalone

Docker Windows Server Containers Qemu / KVM Hyper-V Xen Java Jar Static Binaries C# Rkt

slide-89
SLIDE 89
slide-90
SLIDE 90

Key Value Store HTTP API Host & Service Level Health Checks Datacenter Aware Service Discovery HTTP + DNS

slide-91
SLIDE 91
slide-92
SLIDE 92

CLIENT CLIENT CLIENT CLIENT CLIENT CLIENT SERVER SERVER SERVER

REPLICATION REPLICATION RPC RPC L A N G O S S I P

slide-93
SLIDE 93

CLIENT CLIENT CLIENT CLIENT CLIENT CLIENT SERVER SERVER SERVER

REPLICATION REPLICATION RPC RPC L A N G O S S I P

SERVER SERVER SERVER

REPLICATION REPLICATION W A N G O S S I P

slide-94
SLIDE 94

DB 1 DB 2 DB N

HEALTH CHECKING SERVICE

"Are you healthy?" "What about you?" "Yessir!" "Nah"

slide-95
SLIDE 95

DB 1 DB 2 DB N

HEALTH CHECKING SERVICE

1,000'S OF REQUESTS

slide-96
SLIDE 96

CONSUL

DB 1 DB 2 DB N

My status has changed

slide-97
SLIDE 97

CONSUL

DB 1 DB 2 DB N

10'S OF REQUESTS

slide-98
SLIDE 98

v1.1.2

QTY: 0 QTY: 20

v2.0.0

slide-99
SLIDE 99
slide-100
SLIDE 100
slide-101
SLIDE 101
slide-102
SLIDE 102

v1.1.2

QTY: 0 QTY: 20

v2.0.0

slide-103
SLIDE 103

Terminal

$ terraform plan -var-file=yow2016.tfvars -out=yow2016.tfplan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. Your plan was also saved to the path below. Call the "apply" subcommand with this plan file and Terraform will exactly execute this execution plan. Path: yow2016.tfplan + consul_key_prefix.myservice_config datacenter: "<computed>" path_prefix: "myservice/mycomponent/" subkeys.%: "3" subkeys.appParam1: "val1" subkeys.appParam2: "var2" subkeys.dbHostname: "my-db.service.consul" Plan: 1 to add, 0 to change, 0 to destroy.

slide-104
SLIDE 104

Terminal

$ cat myservice-consul-kv-config.tf variable "path_prefix" { default = "myservice/mycomponent" } resource "consul_key_prefix" "myservice_config" { path_prefix = "${var.path_prefix}/" subkeys = { "appParam1" = "val1" "appParam2" = "var2" "dbHostname" = "my-db.service.consul" } }

slide-105
SLIDE 105

Terminal

$ git diff myservice-consul-kv-config.tf diff --git a/myservice-consul-kv-config.tf b/myservice-consul-kv-config.tf index 76533d8..990d595 100644

  • -- a/myservice-consul-kv-config.tf

+++ b/myservice-consul-kv-config.tf @@ -5,7 +5,5 @@ resource "consul_key_prefix" "myservice_config" { "appParam1" = "val1" "appParam2" = "var2" "dbHostname" = "my-db.service.consul" + "dbHostnameFollower" = "slave.my-db.query.consul" + "dbHostnameLeader" = "master.my-db.query.consul" } }

slide-106
SLIDE 106

Terminal

$ terraform plan -var-file=yow2016.tfvars -out=yow2016.tfplan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. consul_key_prefix.myservice_config: Refreshing state... (ID: myservice/mycomponent/) Your plan was also saved to the path below. Call the "apply" subcommand with this plan file and Terraform will exactly execute this execution plan. Path: yow2016.tfplan ~ consul_key_prefix.myservice_config subkeys.%: "3" => "5" subkeys.dbHostnameFollower: "" => "slave.my-db.query.consul" subkeys.dbHostnameLeader: "" => "master.my-db.query.consul" Plan: 0 to add, 1 to change, 0 to destroy.

slide-107
SLIDE 107

Terminal

$ terraform apply yow2016.tfplan consul_key_prefix.myservice_config: Modifying... subkeys.%: "3" => "5" subkeys.dbHostnameFollower: "" => "slave.my-db.query.consul" subkeys.dbHostnameLeader: "" => "master.my-db.query.consul" consul_key_prefix.myservice_config: Modifications complete Apply complete! Resources: 0 added, 1 changed, 0 destroyed. The state of your infrastructure has been saved to the path

  • below. This state is required to modify and destroy your

infrastructure, so keep it safe. To inspect the complete state use the `terraform show` command. State path: terraform.tfstate

slide-108
SLIDE 108

Terminal

$ terraform fmt cat myservice-consul-kv-config.tf cat myservice-consul-kv-config.tf $ cat cat myservice-consul-kv-config.tf resource "consul_key_prefix" "myservice_config" { path_prefix = "${var.path_prefix}/" subkeys = { "appParam1" = "val1" "appParam2" = "var2" "dbHostname" = "my-db.service.consul" "dbHostnameFollower" = "slave.my-db.query.consul" "dbHostnameLeader" = "master.my-db.query.consul" }

slide-109
SLIDE 109

Terminal

$ cat myservice-consul-kv-config.tf variable "path_prefix" { default = "myservice/mycomponent" } variable "service_db_name" {} resource "consul_key_prefix" "myservice_config" { path_prefix = "${var.path_prefix}/" subkeys = { "appParam1" = "val1" "appParam2" = "var2" "dbHostname" = "${var.service_db_name}.service.consul" "dbHostnameFollower" = "slave.${var.service_db_name}.query.consul" "dbHostnameLeader" = "master.${var.service_db_name}.query.consul" } }

slide-110
SLIDE 110

Terminal

$ cat yow2016.tfvars "address" = "127.0.0.1:8500" "datacenter" = "yow2016" "token" = "" "service_db_name" = "my-db"

slide-111
SLIDE 111

Terminal

$ terraform plan -var-file=yow2016.tfvars -out=yow2016.tfplan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. consul_key_prefix.myservice_config: Refreshing state... (ID: myservice/mycomponent/) No changes. Infrastructure is up-to-date. This means that Terraform could not detect any differences between your configuration and the real physical resources that exist. As a result, Terraform doesn't need to do anything.

slide-112
SLIDE 112

Terminal

$ cat myservice-consul-kv-config.tf variable "db_leader_tag" { default = "leader" } variable "db_follower_tag" { default = "follower" } # snip resource "consul_key_prefix" "myservice_config" { path_prefix = "${var.path_prefix}/" subkeys = { "appParam1" = "val1" "appParam2" = "var2" "dbHostname" = "${var.service_db_name}.service.consul" "dbHostnameFollower" = "${var.db_follower_tag}.${var.service_db_name}.query.consul" "dbHostnameLeader" = "${var.db_leader_tag}.${var.service_db_name}.query.consul" } }

slide-113
SLIDE 113

Terminal

% git diff diff --git a/myservice-consul-kv-config.tf b/myservice-consul-kv-config.tf index 76590ef..e22eff0 100644

  • -- a/myservice-consul-kv-config.tf

+++ b/myservice-consul-kv-config.tf @@ -1,9 +1,9 @@ variable "db_leader_tag" {

  • default = "leader"

+ default = "rw" } variable "db_follower_tag" {

  • default = "follower"

+ default = "ro" } variable "path_prefix" {

slide-114
SLIDE 114

Terminal

$ terraform plan -var-file=yow2016.tfvars -out=yow2016.tfplan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. consul_key_prefix.myservice_config: Refreshing state... (ID: myservice/mycomponent/) Your plan was also saved to the path below. Call the "apply" subcommand with this plan file and Terraform will exactly execute this execution plan. Path: yow2016.tfplan ~ consul_key_prefix.myservice_config subkeys.dbHostnameFollower: "follower.my-db.query.consul" => "ro.my-db.query.consul" subkeys.dbHostnameLeader: "leader.my-db.query.consul" => "rw.my-db.query.consul" Plan: 0 to add, 1 to change, 0 to destroy. $ git reset --hard HEAD is now at 2ab88f9 Revise terminology from master/slave to leader/follower

slide-115
SLIDE 115
  • 1. Codify Everything
  • 2. Pre-Plan outcomes at build-time
  • 3. Create reproducible artifacts
  • 4. Idempotent APIs and Tooling
  • 5. Developer-Centric Operations
  • 6. Make small, well understood changes changes
  • 7. Start where it makes sense for your organization
slide-116
SLIDE 116

CONTACT INFO

@SeanChittenden sean@hashicorp.com THANK YOU! Questions?

slide-117
SLIDE 117

IMBUED TRUST

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

slide-118
SLIDE 118

SMALL SUCCESS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

VAULT

slide-119
SLIDE 119

WIDE EYES

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

VAULT

slide-120
SLIDE 120

HA BACKEND

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

VAULT

slide-121
SLIDE 121

ROBUST VAULT!

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

VAULT

slide-122
SLIDE 122

HA CONSUL + HA VAULT

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

VAULT

slide-123
SLIDE 123

SECRETS AT LAST

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

SECRETS VAULT

slide-124
SLIDE 124

AUTOMATION

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

NOMAD SECRETS VAULT

slide-125
SLIDE 125

FULL STACK VALUE

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

BUSINESS OBJECTIVE MYAPP NOMAD SECRETS VAULT

slide-126
SLIDE 126
slide-127
SLIDE 127

RISK MITIGATED

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

BUSINESS OBJECTIVE MYAPP NOMAD SECRETS VAULT

SECRETS CONSUL CLUSTER APPLICATION CONSUL CLUSTER

slide-128
SLIDE 128
slide-129
SLIDE 129

SELF-HEALING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

TRIAGE DIAGNOSE TREAT PREVENT

slide-130
SLIDE 130

SELF-ASSEMBLING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

slide-131
SLIDE 131

SELF-ASSEMBLING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

FOUNDATION

slide-132
SLIDE 132

SELF-ASSEMBLING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

FOUNDATION PLATFORM

slide-133
SLIDE 133

SELF-ASSEMBLING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

FOUNDATION PLATFORM APP

slide-134
SLIDE 134

SELF-ASSEMBLING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

FOUNDATION PLATFORM APP "Easy"

slide-135
SLIDE 135

SELF-ASSEMBLING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

FOUNDATION PLATFORM APP "Easy" Tough

slide-136
SLIDE 136

SELF-ASSEMBLING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

FOUNDATION PLATFORM APP "Easy" Hard Tough

slide-137
SLIDE 137

SELF-ASSEMBLING SYSTEMS

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

FOUNDATION PLATFORM APP

slide-138
SLIDE 138

ERROR BUDGET

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

slide-139
SLIDE 139

ERROR BUDGET

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

Big Budget
 Less Important Smaller Budget
 More Important

slide-140
SLIDE 140

PLAN FOR KNOWN FAILURE

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

slide-141
SLIDE 141

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

PLAN FOR KNOWN FAILURE

slide-142
SLIDE 142

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

KNOWN KNOWNS PLAN FOR KNOWN FAILURE

slide-143
SLIDE 143

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

KNOWN UNKNOWN PLAN FOR KNOWN FAILURE

slide-144
SLIDE 144

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

PLAN FOR KNOWN FAILURE

slide-145
SLIDE 145

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

UNKNOWN UNKNOWN PLAN FOR KNOWN FAILURE

slide-146
SLIDE 146

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

PLAN FOR KNOWN FAILURE

slide-147
SLIDE 147

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

PLAN FOR KNOWN FAILURE

slide-148
SLIDE 148

RISK MANAGEMENT

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

slide-149
SLIDE 149

RISK MANAGEMENT

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

Insiders OpenSSL Application
 Vulnerabilities

slide-150
SLIDE 150

RISK MANAGEMENT

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

slide-151
SLIDE 151

AUTOMATION

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

slide-152
SLIDE 152

AUTOMATION

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

GOOD BAD UGLY

slide-153
SLIDE 153

EMBRACE AUTOMATION

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2

Creative Industrious Lazy Mental Drift

slide-154
SLIDE 154
  • Self-Healing
  • Self-Assembly
  • Error Budgeting
  • Failure Planning
  • Risk Management
  • Automation

THINGS WE EMBRACE

INCREMENTALISM LIFE CYCLE CODIFY EXAMPLE 1 TENETS EXAMPLE 2