Highway to heaven Building microservices in the cloud Frankfurter - - PowerPoint PPT Presentation

▶
highway to heaven
SMART_READER_LITE
LIVE PREVIEW

Highway to heaven Building microservices in the cloud Frankfurter - - PowerPoint PPT Presentation

Feb 03, 2023 480 likes •860 views

Highway to heaven Building microservices in the cloud Frankfurter Entwicklertag 2016 Agenda Meet AutoScout24 Shifting gears How we build our services How we organize ourselves @cdeger & @wolfwolf AutoScout24 - 30.000ft C C C

slide-1
SLIDE 1

Highway to heaven

Building microservices in the cloud

Frankfurter Entwicklertag 2016

slide-2
SLIDE 2

Agenda

Meet AutoScout24 Shifting gears How we build our services How we organize ourselves @cdeger & @wolfwolf

slide-3
SLIDE 3

AutoScout24 - 30.000ft

C C

slide-4
SLIDE 4

Baseline AutoScout24 IT

Highly optimized, but of last decade IT platform supported growth for >6 years Microsoft oriented stack Enterprise IT setup - MTBF over MTTR Proven agile and lean principles

C C C

slide-5
SLIDE 5

New CEO

C C

slide-6
SLIDE 6

Do you attract talent?

C C

slide-7
SLIDE 7

C C

slide-8
SLIDE 8

Project Tatsu

W W

slide-9
SLIDE 9

Monolith to Microservices Data center to AWS .NET / Windows to JVM / Linux Devs + Ops to DevOps culture Involve product people

Five challenges

slide-10
SLIDE 10

Technical transformation

W W

slide-11
SLIDE 11

Change the wheels while driving

W W

slide-12
SLIDE 12

Strategic Goals

Goals of the business side

Architectural Principles

High-Level Principles

Design and Delivery Principles

Tactical measures

Reduce Time to Market

Speed, Fast Feedback

Cost Efficiency

Collect metrics to allow decisions cost vs. value.

Support Data-Driven Decisions

Listen to users and validate hypothesis. Provide as many relevant metrics & data as possible.

You build it, you run it

The team is responsible for shaping, building, running and maintaining its products. Fast feedback from live and customers helps us to continuously improve.

Organized around Business Capabilities

Build teams around products not projects. Follow the domain and respect bounded contexts. Inverse Conway Maneuver.

Shared Nothing

By default avoid sharing and tight coupling, except for the big things in common. Don’t create the next monolith.

Macro and Micro Architecture

Clear separation. Autonomous micro services within the rules and constraints of the macro architecture.

AWS First

Favor AWS platform service over managed service, over self- hosted OSS, over self-rolled solutions.

Data-Driven/ Metric-Driven

Collect metrics from processes and applications. Analyze, alert and act on them.

Eliminate Accidental Complexity

Strive to keep it simple. Focus on essential complexity. You build

  • ne, you delete one.

Autonomous Teams

Make fast local decisions. Be responsible. Know your

  • boundaries. Share findings.

Infrastructure As Code

Automate everything: Reproducible, traceable and tested. Immutable servers over snowflake servers.

Collaboration Culture

Engineers from all backgrounds work together in collaborative teams as engineers and share responsibilities. No silos.

Be Bold

Go into production early. Value monitoring over tests. Recover and learn. Optimize for MTTR not MTBF.

Security, Compliance and Data Privacy

Security must be included from the beginning and everybody’s

  • concern. Keep data-privacy in mind.

Another goal

Work in progress...

Containment and Boundaries

Align blast radius and vendor lock-in with the boundaries of the

  • rganization or business capabilities.

C C

slide-13
SLIDE 13

Principles

Organized around business capabilities You build it, you run it Be bold Macro and micro architecture Shared nothing

slide-14
SLIDE 14

Shared infrastructure

Shared nothing Availability over shared nothing Convenience offerings No side effects Fast local decisions over committee Respect family ties

W W

slide-15
SLIDE 15

How (not) to share

Use over re-use Re-use only after hardening Copy n’paste, OSS, library Pull instead of push EventPublisher24

W W

slide-16
SLIDE 16

How many environments?

Which versions on staging? Prod differs anyway: Load, data, patterns

V2 V3 V6 V5 V4 V7 V5 V8 Engineer CI Dev Staging V1 V4 Prod C C

slide-17
SLIDE 17

Dev and prod is enough

Consumer driven contracts Shadow traffic Semantic monitoring Smoke tests or canary releases

V2 V3 V6 V5 V4 V7 V5 V8 Engineer CI Dev Prod C C

slide-18
SLIDE 18

Event Sourcing and data pumps

One way data highway Event Sourcing - store history of all changes

W W

slide-19
SLIDE 19

SQS + S3 Kinesis + S3 Kinesis + DynamoDB SQS + DynamoDB Proxy + DynamoDB DynamoDB

Evolution

W W

slide-20
SLIDE 20

Frontend integration

Shared nothing Autonomous teams One domain High optimisation

C C

slide-21
SLIDE 21

Frontend integration

C C

slide-22
SLIDE 22

PageSpeed Module

css (page+fragment) js (page+fragment)

ngx_pagespeed

css (page) js (page) css (fragment) js (fragment)

C C

slide-23
SLIDE 23

How to build autonomous teams

Do not fall back into old behaviours Beware of Mandelbrot teams Pager duty so that you run it Fix broken windows Part-time ops not working Not all T-shapes are the same

Wolf W W

slide-24
SLIDE 24

Infrastructure guild

Agree on things to do Share learnings Delegate implementation to teams Empty backlog should be normal Infrastructure product teams needed?

W W

slide-25
SLIDE 25

cdeger@autoscout24.com @cdeger wschlegel@thoughtworks.com @wolfwolf

slide-26
SLIDE 26

Attributions

Blue sky, white-gray clouds by nature protector Natubico, www.vivism.info [CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File%3ABlue_sky%2C_white-gray_clouds.JPG A Danish Perspective by NASA [Public domain] http://commons.wikimedia.org/wiki/File%3AA_Danish_Perspective.jpg http://commons.wikimedia.org/wiki/File%3ANASAComputerRoom7090.NARA.jpg GREG EINRAD Amazon16 by Neil Palmer/CIAT [CC BY-SA 2.0] https://www.flickr.com/photos/ciat/5641594952 BERGSTEIGER Barber in Cameroon by James Emery from Douglasville, United States (Daddy Joe_1355) [CC BY 2.0] http://commons.wikimedia.org/wiki/File%3ABarber_in_Cameroon.jpg Wide objectives by Kivela (Own work) [Public domain] href="http://commons.wikimedia.org/wiki/File%3AWide_objectives.jpg Transformer Fire Barrier by GerryS1 (Own work) [CC BY-SA 3.0 or GFDL] http://commons.wikimedia.org/wiki/File%3ATransformer_Fire_Barrier.jpg

slide-27
SLIDE 27

Attributions (cont)

Alonso Renault Pitstop Chinese GP 2008 by Bert van Dijk (Pitstop F1 ING Renault) [CC BY-SA 2.0] http://commons.wikimedia.org/wiki/File%3AAlonso_Renault_Pitstop_Chinese_GP_2008.jpg Principle of Panchasheel by Prakash Adhikary (Own work) [CC BY 3.0] http://commons.wikimedia.org/wiki/File%3APrinciple_of_Panchasheel.JPG Traffic Jam by Doo Ho Kim [CC BY-SA 2.0] https://www.flickr.com/photos/titicat/3049591547 Pellets by The original uploader was Richard Mayer at German Wikipedia [GFDL or CC-BY-SA-3.0] http://commons.wikimedia.org/wiki/File%3APellets.jpg Pipes and Valves by Uwe Hermann [CC BY-SA 2.0] https://www.flickr.com/photos/73628542@N00/6272975359 Size variation in Coccinella undecimpunctata (2127991716) by Gilles San Martin from Namur, Belgium [CC BY-SA 2.0] http://commons.wikimedia.org/wiki/File%3ASize_variation_in_Coccinella_undecimpunctata_(2127991716).jpg Mille crêpe by Laitr Keiows (Own work) [CC BY-SA 3.0 or GFDL] http://commons.wikimedia.org/wiki/File%3AMille_cr%C3%AApe.jpg Country Energy power line replacement 01 by Bidgee (Own work) [CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File%3ACountry_Energy_power_line_replacement_01.jpg Puzzling by Bernd Gessler (Own work) [CC BY-SA 3.0] https://commons.wikimedia.org/wiki/File%3APuzzling.JPG

slide-28
SLIDE 28

Attributions (cont)

Sharing Sucks (4536747557) by eyeliam from Portland, United States [CC BY 2.0] http://commons.wikimedia.org/wiki/File%3ASharing_Sucks_(4536747557).jpg 7Line 9184 (8263568241) by Metropolitan Transportation Authority of the State of New York (7Line_9184 Uploaded by tm) [CC BY 2.0] http://commons.wikimedia.org/wiki/File%3A7Line_9184_(8263568241).jpg England rugby team 1905 by Russell & Sons (The Graphic) [Public domain or Public domain] http://commons.wikimedia.org/wiki/File%3AEngland_rugby_team_1905.jpg Wandergeselle by Sigismund von Dobschütz [CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File%3AWandergeselle_02.JPG Faber-Rechenschieber 5304 by User:Karl Gruber (Own work) [CC BY-SA 4.0] http://commons.wikimedia.org/wiki/File%3AFaber-Rechenschieber_5304.JPG Wheel clamps Texas by Richard Anderson from Denton, United States (Boots.) [CC BY-SA 2.0] http://commons.wikimedia.org/wiki/File%3AWheel_clamps_Texas.jpg GuadalupeNOLA15Oct07Thanks by Infrogmation of New Orleans (Photo by Infrogmation) [GFDL or CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File%3AGuadalupeNOLA15Oct07Thanks.jpg AtariBasic by Calin99 (Own work) [GPL] http://commons.wikimedia.org/wiki/File%3AAtariBasic.png

Spare wheel by Brian Snelson [CC BY 2.0] https://commons.wikimedia.org/wiki/File:Spare_wheel_-_Flickr_-_exfordy.jpg

slide-29
SLIDE 29

Backup

slide-30
SLIDE 30

Unified Logs

C C

slide-31
SLIDE 31

Two stacks

Cash stack meets shiny new stack One company Lights on in cash stack Feature freeze Where to build new features? Ease of integration helps business people

C C

slide-32
SLIDE 32

From documents to events

Refactoring toward deeper insight From CRUD to sync Offline first Writes are expensive USD 20 over USD 500

C C

slide-33
SLIDE 33

How many layers to estimate a price

Evolving architecture R backend and Play application Single Play application Play backend and Play web server Long feedback cycles Frequency of change

W W W

slide-34
SLIDE 34

Watchlist evolution

DynamoDB as JSON document store Scheduled user journey test Capacity exceeded Requests throttled Latency spikes for real users

C C C

slide-35
SLIDE 35

Focus sliders

Product over platform

time

W W

IT business

slide-36
SLIDE 36

Focus sliders

Product over platform Delivery over learning

time

W W

IT business