Radical Agility with Autonomous Teams and Microservices - - PowerPoint PPT Presentation

▶
radical agility
SMART_READER_LITE
LIVE PREVIEW

Radical Agility with Autonomous Teams and Microservices - - PowerPoint PPT Presentation

Feb 18, 2023 150 likes •1.01k views

Radical Agility with Autonomous Teams and Microservices jan.loeffler@zalando.de / @jlsoft2 GOTO Copenhagen 2015-10-06 We shape our buildings; thereafter they shape us Conways Law organizations which design systems ... are constrained

slide-1
SLIDE 1

Radical Agility

with Autonomous Teams and Microservices

jan.loeffler@zalando.de / @jlsoft2 GOTO Copenhagen 2015-10-06

slide-2
SLIDE 2

We shape our buildings; thereafter they shape us

slide-3
SLIDE 3

Conway’s Law “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations”

Melvin Conway

slide-4
SLIDE 4

AN ARCHITECTURE FOR INNOVATION

slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8
slide-9
SLIDE 9

A BRIEF HISTORY OF ZALANDO TECHNOLOGY

slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13 October

900+ Apps 800+ Tech employees

slide-14
SLIDE 14

Platform

Platform team

request servers deploy

slide-15
SLIDE 15

Platform

70+ delivery teams Platform team

deploy request servers request storage

slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18

DELIVER AMAZING PRODUCTS EFFICIENTLY AT SCALE, AND FEELING GREAT ABOUT IT.

slide-19
SLIDE 19

PURPOSE - AUTONOMY - MASTERY

slide-20
SLIDE 20
slide-21
SLIDE 21
slide-22
SLIDE 22
slide-23
SLIDE 23

DRIVE The Surprising Truth About What Motivates Us Daniel Pink

slide-24
SLIDE 24

FROM CONTROL & COMMAND TO PURPOSE AND TRUST

slide-25
SLIDE 25

DELIVERY LEAD PEOPLE LEAD

slide-26
SLIDE 26 GLOBAL REGRESSION INNOVATION LAB TECH SERVICE PRODUCT OWNER PRODUCT SPECIALIST DELIVERY LEAD PEOPLE LEAD

BUSINESS ASSURANCE PRODUCT OVERARCHING

ADMIN & SUPPORT CONTROLLING EXECUTIVE SUPPORT COMPLIANCE RISK , SECURITY & STRATEGY ONBOARDING & TECHADEMY AGILE COACHING PROJECT MANGEMENT ENGINEERING PRODUCTIVITY DELIVERY LEAD PEOPLE LEAD

DELIVERY

slide-27
SLIDE 27
slide-28
SLIDE 28

OKR

slide-29
SLIDE 29
slide-30
SLIDE 30

API FIRST

slide-31
SLIDE 31

REST

slide-32
SLIDE 32

SAAS

slide-33
SLIDE 33

MICRO SERVICES

slide-34
SLIDE 34

CLOUD

slide-35
SLIDE 35

OPEN SOURCE

slide-36
SLIDE 36
slide-37
SLIDE 37

Compliance Innovation

slide-38
SLIDE 38
slide-39
SLIDE 39
slide-40
SLIDE 40

WHERE TO GO

DataCenter I DataCenter II AWS

APP 1 APP 2 APP 3 APP 4 APP 5 APP 6 APP 1 APP 2 APP 3 APP 4 APP 5 APP 6 APP 1 APP 2 APP 3 APP 4

slide-41
SLIDE 41
slide-42
SLIDE 42

STUPS.io

STUPS To Unleash Penguin Swarms

slide-43
SLIDE 43
slide-44
SLIDE 44

One AWS account per Team Deployment with Docker Managed SSH Access REST / OAuth 2.0 mandatory

slide-45
SLIDE 45

AWS STUPS

DOCKER DEPLOY SSH ACCESS AUDIT REPORTS FULL AWS ACCESS

slide-46
SLIDE 46

Internet *.abc.example.org *.xyz.example.org Team ABC Team XYZ

EC2 EC2 ELB ELB EC2

slide-47
SLIDE 47

ELB myapp-1 myapp.example.org EC2 + Docker EC2 + Docker EC2 + Docker

slide-48
SLIDE 48

ELB myapp-1 EC2 + Docker EC2 + Docker EC2 + Docker ELB myapp-2 EC2 + Docker EC2 + Docker myapp.example.org

slide-49
SLIDE 49

ELB myapp-2 EC2 + Docker EC2 + Docker myapp.example.org

slide-50
SLIDE 50

SCM Ticket system Issue “ABC-123” Commit “afb123” msg: ABC-123..

slide-51
SLIDE 51

SCM Pier One Docker Registry build Ticket system Image “docker/myart:1.0” commit: afb123 Issue “ABC-123” Commit “afb123” msg: ABC-123..

slide-52
SLIDE 52

Pier One Docker Registry build

approved

Application Version “1.0” artifact: docker/myart:1.0 Ticket system Application Registry SCM Image “docker/myart:1.0” commit: afb123 Issue “ABC-123” Commit “afb123” msg: ABC-123..

✓ Specification ✓ Artefact tested

slide-53
SLIDE 53

Pier One Docker Registry build

approved

EC2 Instance

Docker Container

Application Version “1.0” artifact: docker/myart:1.0 AMI Ticket system Application Registry SCM Image “docker/myart:1.0” commit: afb123 Issue “ABC-123” Commit “afb123” msg: ABC-123..

✓ Specification ✓ Artefact tested

slide-54
SLIDE 54

AWS

Senza CLI

Docker Registry docker pull docker push

AMI

slide-55
SLIDE 55

AWS

Developer Console

get access token

AMI

Password Rotator

OAuth Provider

store passwords get password

S3

rotate passwords

Application Registry

slide-56
SLIDE 56
slide-57
SLIDE 57

Spilo

Highly available open-source PostgreSQL appliance

https://github.com/zalando/spilo

slide-58
SLIDE 58
slide-59
SLIDE 59

Try STUPS.io https://stups.io https://docs.stups.io All components are Open Source https://github.com/zalando https://github.com/zalando-stups

slide-60
SLIDE 60

Jan Löffler

  • Head of Platform Engineering
  • Twitter: @jlsoft2
  • jan.loeffler@zalando.de
  • http://www.slideshare.net/jlsoft/
slide-61
SLIDE 61

We shape our buildings; thereafter they shape us

slide-62
SLIDE 62

STUPS website & docs https://stups.io https://docs.stups.io All components are Open Source https://github.com/zalando https://github.com/zalando-stups

slide-63
SLIDE 63

BACKUP

slide-64
SLIDE 64
slide-65
SLIDE 65

D E P L O Y M E N T

slide-66
SLIDE 66

FROM zalando/openjdk:8u40-b09-4 EXPOSE 8080 COPY target/hello-world.jar / COPY target/scm-source.json / CMD java $(java-dynamic-memory-opts) ↲

  • jar /hello-world.jar

DOCKERFILE

slide-67
SLIDE 67

$ docker build -t ↲ pierone.example.org/myteam/hello-world:0.2 . $ pierone login Getting OAuth2 token "pierone".. OK Storing Docker client configuration in ~/.dockercfg.. OK $ docker push pierone.example.org/myteam/hello-world:0.2

DOCKER BUILD & PUSH

slide-68
SLIDE 68

$ pierone tags myteam hello-world Team │Artifact │Tag │Created│By | myteam hello-world 0.1-andre-test 13d ago ahartmann myteam hello-world 0.1 3d ago ahartmann myteam hello-world 0.2 3m ago hjacobs $ pierone scm myteam hello-world 0.2 Tag│Author │URL │Revision │Status│Created│By | 0.2 hjacobs git:git@github.. 442b7502 10m ago hjacobs

VERIFY IMAGE UPLOAD

slide-69
SLIDE 69

SENZA: DEFINITION YAML

SenzaInfo: StackName: hello-world Parameters:

  • ImageVersion:

Description: "Docker image version of Hello World." SenzaComponents:

  • Configuration:

Type: Senza::StupsAutoConfiguration # auto-detect network setup

  • AppServer: # will create a launch configuration and ASG with scaling triggers

Type: Senza::TaupageAutoScalingGroup InstanceType: t2.micro SecurityGroups: [app-hello-world] ElasticLoadBalancer: AppLoadBalancer TaupageConfig: runtime: Docker source: "stups/hello-world:{{Arguments.ImageVersion}}" ports: 8080: 8080

slide-70
SLIDE 70

SENZA: STACK DEPLOYMENT

$ senza create hello-world.yaml 1 0.2 Generating Cloud Formation template.. OK Creating Cloud Formation stack hello-world-1.. OK $ senza events hello-world.yaml 1

Stack Name│Ver.│Resource Type │Resource ID │Status │Status Reason │Event Time hello-world 1 CloudFormation::Stack hello-world-1 CREATE_IN_PROGRESS User Initiated 10m ago ... hello-world 1 CloudFormation::Stack hello-world-1 CREATE_COMPLETE 6m ago

slide-71
SLIDE 71

docker run -d --log-driver=syslog ↲

  • -restart=on-failure:10 ↲
  • e DB_SUBNAME=.. ↲
  • v /meta:/meta:ro ↲
  • e CREDENTIALS_DIR=/meta/credentials ↲
  • p 8080:8080 -p 7979:7979 ↲
  • u 999 ↲

pierone.example.org/stups/pierone:0.5

TAUPAGE: DOCKER COMMAND LINE

slide-72
SLIDE 72

SENZA: MANAGE STACKS

slide-73
SLIDE 73

L O G G I N G

slide-74
SLIDE 74

docker run .. --log-driver=syslog .. /etc/rsyslog.d/24-application.conf :syslogtag, startswith, "docker" ↲ /var/log/application.log /etc/logrotate.d/.. Don’t forget log rotation..

TAUPAGE: DOCKER SYSLOG

slide-75
SLIDE 75

APPLICATION LOGS: TAUPAGE SUPPORTS LOGENTRIES AND SCALYR

slide-76
SLIDE 76

S S H A C C E S S

slide-77
SLIDE 77

SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER

slide-78
SLIDE 78

M O N I T O R I N G

slide-79
SLIDE 79

TODO: Screenshot

ZMON

slide-80
SLIDE 80

ZMON APPLIANCE *.foo.example.org *.bar.example.org Team “Foo” Team “Bar”

EC2 Instance EC2 Instance EC2 Instance EC2 Instance

ZMON Appliance ZMON Appliance

KairosDB EC2 Instance EC2 Instance

ZMON Controller

ELB ELB

slide-81
SLIDE 81

HYSTRIX TURBINE

slide-82
SLIDE 82

D O C K E R

slide-83
SLIDE 83
  • Ubuntu & OpenJDK base image
  • Log to STDOUT
  • Config via environ. vars (+ KMS decryption)
  • Non-root execution
  • Persistence via EBS mounts
  • Immutable stacks, no orchestration
  • DNS endpoints, etcd e.g. for Hystrix streams

RECAP: DOCKER IN STUPS

slide-84
SLIDE 84

Securing REST based Microservices

Lock all doors

  • Open only ports that you need

permanently

  • Grant SSH access only temporary
  • Use a firewall for each app container

Encrypt all channels

  • Keep all tranferred data confidential
  • Use TLS

Authenticate & Authorize

  • Make sure you talk to the right guy
  • Check if access is permitted
  • Use OAuth 2.0

Monitor & Limit your API calls

  • Control who accesses your API how
  • ften, when and from where
  • Use Rate Limiting against fraud
  • Use external Security Services against

DDoS Isolate your apps

  • Only one app per container / server
  • Use Docker or CoreOS Rkt
slide-85
SLIDE 85

OAuth 2.0 explained

User “Jan” Browser Resource Server (App Server) Authorization Server (OpenAM / OpenDJ)

I want to see all my past orders! Hey Order API, could you please list all past orders of Jan Loeffler? Sorry, this resource is protected! Please show me your Access Token! Hey Mr. DJ, can I have an Access Token for the Order API? No issue Sir, I just have to ask the User for some details. Hi, could you please show me your credentials? I need to check your identity. Sure, I am jan.loeffler@zalando.de and my password is secret. Your seem to be ok! Here is your access token d2fa5d27-2acc-4b06-95i8-6d3018d94b4f Hey Order API, here is my access token: d2fa5d27-2acc-4b06-95i8-6d3018d94b4f Hey DJ, I was just given this token: d2fa5d27-2acc-4b06-95i8-6d3018d94b4f To whom does it belong? Sure, the token belongs to jan. loeffler@zalando.de and is still valid. So, here are all orders of Jan Loeffler. Here are all all your orders!