Ubers Journey into Microservices Emily Reinhold, Software Engineer, - - PowerPoint PPT Presentation

▶

Jul 23, 2023 567 likes •1.1k views

Ubers Journey into Microservices Emily Reinhold, Software Engineer, Uber JUNE 15, 2016 Background Our Monolith Fun Facts API Started January 2011 API Trends Lines of Code over Time Unique Authors over Time 300K 600 225K

SLIDE 1

Uber’s Journey into Microservices

Emily Reinhold, Software Engineer, Uber

JUNE 15, 2016

SLIDE 2

Our Monolith

Background

SLIDE 3

“API”
Started January 2011

Fun Facts

SLIDE 4

Lines of Code over Time

0K 75K 150K 225K 300K 2011 2012 2013 2014 2015 2016

Unique Authors over Time

150 300 450 600 2011 2012 2013 2014 2015 2016

Start of migration. Yay!

API Trends

SLIDE 5

“API”
Started January 2011
Access to core data models

Fun Facts

SLIDE 6

Core Data Models

Users Trips Payments Products

Cities

Documents Exchange Rates

Vehicles

Promos

The Uber Monolith

SLIDE 7

Postgresql
Python 2.7
HTTP/JSON
uWSGI

Technical Details

SLIDE 8

Request Flow

Matching Service haproxy API box nginx uWSGI process

Postgres Downstream Services Third Parties

API

SLIDE 9

Minimal overhead for cross-

domain features

Benefits

SLIDE 10

“Cross Domain”

Users Trips Payments Products

Cities

Documents Exchange Rates

Vehicles

Promos

The Uber Monolith

SLIDE 11

Minimal overhead for cross-

domain features

Reuse code across domains
Broad, extensible interface

Benefits

SLIDE 12

Honks
Halloween 2014
Postgresql Master Failure

Historic Outages

Image courtesy of Anita Hart. Licensed under CC-BY-SA 2.0. https://flic.kr/p/8PG15b

SLIDE 13

Postgresql Master Failure

SLIDE 14

Honks
Halloween 2014
Postgresql Master Failure
Kafka Outage

Historic Outages

SLIDE 15

Kafka Outage

SLIDE 16

Deploy burden
Single Postgres Master

Technical Bottlenecks

Image courtesy of Paul Schadler. Licensed under CC-BY 2.0. https://flic.kr/p/aDoGkP

SLIDE 17

Rapid hiring
Slow test suite
Toe stepping, merge conflicts

Developability

SLIDE 18

Growth of business
Growth of features
Growth of organization

Scalability 3 Ways

SLIDE 19

In Microservices

Our Future

SLIDE 20

Horizontally scalable

Benefits

SLIDE 21

Scalability

Trips Users Promos

SLIDE 22

Horizontally scalable
Improved onboarding
Clear ownership

Benefits

SLIDE 23

Ownership

No hiding! You’re on call!

SLIDE 24

Horizontally scalable
Improved onboarding
Clear ownership
Failure isolation

Benefits

SLIDE 25

Granular Failures

Old Way

API

client show Payments Promos User tags

Edge Service

New Way

Edge Service Payments Promos Users

SLIDE 26

Lessons Learned

SLIDE 27

Drastically changing tech

stack

Limit Scope

SLIDE 28

Tech Stack

That’s me!

eng.uber.com/building-tincup/

SLIDE 29

Data Migration

1. Develop new

schema

2. Implement

converter

3. Dual write
4. Backfill
6. Move read

queries

5. Validation

7. Remove old

writes

SLIDE 30

Drastically changing tech

stack

Infrastructure not ready
Understaffed
Lacked tooling

Limit Scope

SLIDE 31

Things Happen

Image courtesy of Kim Scarborough. Licensed under CC-BY-SA 2.0. https://flic.kr/p/23HUAK

SLIDE 32

Drastically changing tech

stack

Infrastructure not ready
Understaffed
Lacked tooling
Understanding timelines

Limit Scope

SLIDE 33

Monolith’s consumers

directly impacted

Aligning with Consumers

SLIDE 34

Consumers

API

Pricing Matching Signup Exchange rates Cities Users Trips Trips Exchange Rates Cities Users

SLIDE 35

Monolith’s consumers directly impacted
Importance of migrating not well

communicated

Constant interruption

Aligning with Consumers

SLIDE 36

For service developers and

migrating consumers

Tool to split up request

Build Tools Initially

SLIDE 37

Request Splitter

API

Cities Currencies Users Trips

Old Way

requesting trip info, with relationships

n user,

currency, city

New Way API

Cities Currencies Users Trips

Tool

requesting trip info, with relationships

n user,

currency, city request user request city request currency request trip

SLIDE 38

For service developers and migrating

consumers

Tool to split up request
Tool to validate new response
Tool to gain visibility into consumers

Build Tools Initially

SLIDE 39

Most difficult aspect of migration
Silently call new system

Consumer Migration Tips

SLIDE 40

Silent Swap

API

Matching Signup Exchange rates Cities Users Trips Users

SLIDE 41

Most difficult aspect of migration
Silently call new system
Most successful migration kept

interface the same

Consumer Migration Tips

SLIDE 42

Tendency to avoid changes
Tackling core (User) directly

required refactor

Love your Monolith

SLIDE 43

Likely to impact other teams
Teams shift focus

Migration Blockers

SLIDE 44

Migration Successes

SLIDE 45

Shared accomplishment
Relationships formed
New understanding

Collaboration

SLIDE 46

New tech stack: more efficient
No more uWSGI!

Hardware efficiency

SLIDE 47

uWSGI Utilization

SLIDE 48

CPU Usage

SLIDE 49

New tech stack: more efficient
No more uWSGI!
Calls asynchronous

Hardware efficiency

SLIDE 50

How our systems work
Design for stability
Evenly spread ownership

Deeper Understanding

SLIDE 51

First & Last Name Email: emilyr@uber.com Engineering Blog: eng.uber.com Twitter: @UberEng

Thank you

SLIDE 52