Need representative, end-to-end applications 3. Cluster management - - PowerPoint PPT Presentation

▶
need representative end to end applications
SMART_READER_LITE
LIVE PREVIEW

Need representative, end-to-end applications 3. Cluster management - - PowerPoint PPT Presentation

Jul 22, 2023 263 likes •623 views

A N O PEN -S OURCE OURCE B ENC HMARK S UI OR M IC NCHMARK UITE TE FOR ICRO ROSERVICES SERVICES AND ND T HEIR IR H ARDWARE RDWARE -S OFTWARE OFTWARE I MPLICA OR C LOU ICATIONS TIONS FOR OUD AND E DGE S YSTE STEMS MS Yu Gan, Yanqi Zhang,

slide-1
SLIDE 1

AN OPEN-SOURCE

OURCE BENC NCHMARK HMARK SUI UITE TE FOR OR MIC ICRO ROSERVICES SERVICES AND ND

THEIR

IR HARDWARE RDWARE-SOFTWARE OFTWARE IMPLICA ICATIONS TIONS FOR OR CLOU OUD AND

EDGE SYSTE

STEMS MS

Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayantara Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, Kelvin Hu, Meghna Pancholi, Yuan He, Brett Clancy, Chris Colen, Fukang Wen, Catherine Leung, Siyuan Wang, Leon Zaruvinsky, Mateo Espinosa, Rick Lin, Zhongling Liu, Jake Padilla and Christina Delimitrou

Cornell University

ASPLOS 2019 Session Cloud I

slide-2
SLIDE 2

EXEC

ECUTIV UTIVE SUM UMMARY MARY

  • Cloud applications migrating from monoliths to microservices
  • Monoliths: all functionality in a single service
  • Microservices: many single-concerned, loosely-coupled services
  • Modularity, specialization, faster development
  • Datacenters designed for monoliths  microservices have different requirements
  • An end-to-end benchmark suite for large-scale microservices
  • Architectural and system implications
  • Hardware design
  • OS/networking overheads
  • Cluster management
  • Application & programming frameworks
  • Tail at scale

1

slide-3
SLIDE 3

FROM

ROM MON ONOLITHS OLITHS TO TO MIC ICRO ROSERVICES SERVICES

  • Monolithic applications
  • Single binary with entire business logic
  • Limitations
  • Too complex for continuous development
  • Obstacle to adopting new frameworks
  • Poor scalability & elasticity

2

Monolith Application

login

  • rders

payments shipping

slide-4
SLIDE 4

FROM

ROM MON ONOLITHS OLITHS TO TO MIC ICRO ROSERVICES SERVICES

  • Microservices
  • Fine-grained, loosely-coupled, and single-

concerned

  • Communicate with RPCs or RESTful APIs
  • Pros
  • Agile development
  • Better modularity & elasticity
  • Testing and debugging in isolation
  • Cons
  • Different hardware & software constraints
  • Dependencies  complicate cluster

management

3

login

  • rders

payments shipping

slide-5
SLIDE 5

FROM

ROM MON ONOLITHS OLITHS TO TO MIC ICRO ROSERVICES SERVICES

4

slide-6
SLIDE 6

MOT

OTIVATIO IVATION

  • Explore implications of microservices across the system stack

5

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management

4.

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-7
SLIDE 7

MOT

OTIVATIO IVATION

  • Explore implications of microservices across the system stack

6

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management

4.

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale

Need representative, end-to-end applications built with microservices

slide-8
SLIDE 8

MOT

OTIVATIO IVATION

  • Previous work in cloud benchmarking
  • CloudSuite [ASPLOS’12]
  • Sirius [ASPLOS’15]
  • TailBench [IISWC’17]
  • μSuite [IISWC’18]
  • DeathStarBench suite
  • Focus on large-scale microservices that stress typical datacenter design

7

Focus either on monolithic applications or applications with few tiers

slide-9
SLIDE 9

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Design principles
  • Representativeness

» Use of popular open-source applications and frameworks » Service architecture following public documentation of real systems using microservices

8

slide-10
SLIDE 10

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Design principles
  • Representativeness
  • End-to-end operation

» Full functionality using microservices

9

slide-11
SLIDE 11

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Design principles
  • Representativeness
  • End-to-end operation
  • Heterogeneity

» Wide range of programming languages and microservices frameworks

10

slide-12
SLIDE 12

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Design principles
  • Representativeness
  • End-to-end operation
  • Heterogeneity
  • Modularity

» Single-concerned and loosely-coupled services

11

slide-13
SLIDE 13

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Design principles
  • Representativeness
  • End-to-end operation
  • Heterogeneity
  • Modularity
  • Reconfigurability

» Easy to update or change components with minimal effort

12

slide-14
SLIDE 14

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • 5 end-to-end applications, tens of unique microservices each
  • Social Network
  • Media Service
  • E-Commerce Service
  • Banking System
  • Drone Coordination System

13

slide-15
SLIDE 15

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Social network

14

slide-16
SLIDE 16

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Media service

15

slide-17
SLIDE 17

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • E-commerce service

16

slide-18
SLIDE 18

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Banking system

17

slide-19
SLIDE 19

DEA

EATH THSTAR TARBENCH ENCH SUI UITE TE

  • Drone coordination system

18

Client Load Balancer NGINX Controller Construct Route TargetDB ImageDB VideoDB LocationDB SpeedDB LuminosityDB OrientationDB Image Video Location Speed Luminosity Orientation Stocking ImageDB MotionCtrl Image Recognition Obstacle Avoidance Controller Log(node.js) Edge Router Edge Swarm

Frontend Edge Cloud

slide-20
SLIDE 20

MongoDB MongoDB MongoDB MongoDB MongoDB MongoDB Memcached Redis Memcached Memcached Memcached Memcached Memcached RabbitMQ Read Post User Timeline Write Home Timeline Unique ID URL Shorten Image Video Text User Tag Favorite Search Recommender User Compose Post Post Storage Read Home Timeline Social Graph Load Balancer NGINX Video Store Frontend Image Store Frontend Client Video storage Image storage Social graph storage Home timeline storage User timeline storage Post storage User storage

Frontend Logic Caching & Storage

Index1 Indexn Index0

CAS

ASE STU TUDY:

: SOC

OCIA IAL NE NETWORK TWORK

  • User sign up/login

19

slide-21
SLIDE 21

CAS

ASE STU TUDY:

: SOC

OCIA IAL NE NETWORK TWORK

  • Write posts

20

Client Load Balancer NGINX Unique ID URL Shorten Image Video Text User Tag Favorite Search Recommender User Compose Post Post Storage Read Home Timeline Social Graph Memcached MongoDB Redis Memcached MongoDB Read Post Memcached MongoDB Memcached MongoDB User Timeline RabbitMQ Write Home Timeline

Frontend Logic Caching & Storage

Memcached MongoDB Memcached MongoDB Video storage Image storage Social graph storage Home timeline storage User timeline storage Post storage User storage Video Store Frontend Image Store Frontend Index1 Indexn Index0

slide-22
SLIDE 22

CAS

ASE STU TUDY:

: SOC

OCIA IAL NE NETWORK TWORK

  • Read home timeline

21

MongoDB MongoDB MongoDB MongoDB MongoDB MongoDB Memcached Redis Memcached Memcached Memcached Memcached Memcached RabbitMQ Read Post User Timeline Write Home Timeline Unique ID URL Shorten Image Video Text User Tag Favorite Search Recommender User Compose Post Post Storage Read Home Timeline Social Graph Load Balancer NGINX Video Store Frontend Image Store Frontend Client Video storage Image storage Social graph storage Home timeline storage User timeline storage Post storage User storage

Frontend Logic Caching & Storage

Index1 Indexn Index0

slide-23
SLIDE 23

CAS

ASE STU TUDY:

: SOC

OCIA IAL NE NETWORK TWORK

  • Search

22

Index1 Indexn Index0 MongoDB MongoDB MongoDB MongoDB MongoDB MongoDB Memcached Redis Memcached Memcached Memcached Memcached Memcached RabbitMQ Unique ID URL Shorten Image Video Text User Tag Favorite Search Recommender User Compose Post Post Storage Read Home Timeline Social Graph Read Post User Timeline Write Home Timeline Load Balancer NGINX Client

Frontend Logic Caching & Storage

Video storage Image storage Social graph storage Home timeline storage User timeline storage Post storage User storage Video Store Frontend Image Store Frontend

slide-24
SLIDE 24

CAS

ASE STU TUDY:

: SOC

OCIA IAL NE NETWORK TWORK

  • Recommendation

23

MongoDB MongoDB MongoDB MongoDB MongoDB MongoDB Memcached Redis Memcached Memcached Memcached Memcached Memcached RabbitMQ Unique ID URL Shorten Image Video Text User Tag Favorite Search Recommender User Compose Post Post Storage Read Home Timeline Social Graph Read Post User Timeline Write Home Timeline Load Balancer NGINX Client

Frontend Logic Caching & Storage

Video storage Image storage Social graph storage Home timeline storage User timeline storage Post storage User storage Video Store Frontend Image Store Frontend Index1 Indexn Index0

slide-25
SLIDE 25

ARC

RCHITECT ITECTURAL URAL AN AND SYS YSTE TEM IMPLI LICA CATIONS TIONS

  • Explore implications of microservices across the system stack

24

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management

4.

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-26
SLIDE 26

HAR

ARDWA WARE RE DES ESIG IGN

  • Brawny vs. wimpy cores
  • Microservices are more sensitive to performance

unpredictability than monoliths

25

Microservices Monoliths

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-27
SLIDE 27

HAR

ARDWA WARE RE DES ESIG IGN

  • Brawny vs. wimpy cores
  • Microservices are more sensitive to performance

unpredictability than monoliths

26

Microservices Monoliths

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-28
SLIDE 28

HAR

ARDWA WARE RE DES ESIG IGN

  • Brawny vs. wimpy cores
  • Microservices are more sensitive to performance

unpredictability than monolithic apps

  • Xeon vs Cavium servers
  • Cycle breakdown of each microservice
  • Smaller fraction of frontend stalls than monoliths
  • I-cache pressure
  • Lower I-cache pressure than monoliths

27

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-29
SLIDE 29

OS OS/NETWORK

ETWORK OV OVERHE ERHEAD ADS

  • RPC overheads
  • A large fraction of time spent in network stack
  • FPGA network acceleration
  • Offload TCP stack on FPGA
  • 10 − 68x improvement on network processing latency
  • 43% - 2.2x improvement on end-to-end latency

28

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-30
SLIDE 30

Memcached NGINX

CLUS

USTER TER MAN ANAGE AGEMENT MENT

  • Latency back-pressure
  • Bottleneck services pressure upstreaming services
  • Cause: Imperfect pipelining

» HTTP/TCP HoL blocking » Limited number of worker threads/connections

29

NGINX bottleneck NGINX Memcached

🔦

NGINX Memcached

🔦

Memcached bottleneck

Example: HTTP 1.1 HoL blocking

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-31
SLIDE 31

CLUS

USTER TER MAN ANAGE AGEMENT MENT

  • Cascading QoS violations
  • Hotspots propagating along the dependency graph
  • No obvious correlation to CPU utilization
  • Difficulty in discovering the bottleneck and long time to

recover from QoS violations

30

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-32
SLIDE 32

APPLICAT

ICATION ION AN AND FRA RAMEWORKS MEWORKS

  • Serverless frameworks
  • Compared long-running microservices on EC2 with short–

running microservices on AWS Lambda

  • Agile resource adjustments with diurnal load pattern
  • Higher performance variability due to

» No control of lambda placement » Communication through S3 » Loading of dependencies

31

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at scale
slide-33
SLIDE 33

TAI

AIL AT AT SCA CALE

  • Impact of slow servers
  • Larger cluster  larger impact of slow servers
  • More severe tail latency increase compared to monoliths

32

  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 1. Hardware design
  • 2. OS/Network overheads
  • 3. Cluster management
  • 4. Application and frameworks
  • 5. Tail at Scale
slide-34
SLIDE 34

CONCLUS

ONCLUSION IONS

  • Cloud applications from monoliths to microservices
  • Study implications of microservices across the system stack
  • Open-source benchmark suite for cloud and IoT microservices
  • Explored the implications of microservices
  • More sensitive to performance unpredictability
  • Potential of hardware acceleration for networking
  • Need for cluster managers that account for dependencies
  • Tail at scale effects more prominent in microservices

33

slide-35
SLIDE 35

QUE

UESTI TION ONS?

  • Cloud applications from monoliths to microservices
  • Study implications of microservices across the system stack
  • Open-source benchmark suite for cloud and IoT microservices
  • Explored the implications of microservices
  • More sensitive to performance unpredictability
  • Potential of hardware acceleration for networking
  • Need for cluster managers that account for dependencies
  • Tail at scale effects more prominent in microservices

34