KS & Q Q UE ANKS UESTIONS TIONS Microservices introduce new - - PowerPoint PPT Presentation

▶
ks q
SMART_READER_LITE
LIVE PREVIEW

KS & Q Q UE ANKS UESTIONS TIONS Microservices introduce new - - PowerPoint PPT Presentation

Aug 24, 2022 681 likes •1.95k views

Q S IM : E NABLING A CCURATE AND S CALABLE S IMULATION FOR I NTERACTIVE M ICROSERVICES Yanqi Zhang, Yu Gan, Christina Delimitrou Cornell University Session: Datacenters and Cloud Computing, March 26 th M AR TO M IC ARCHI CHING NG TO ICRO

slide-1
SLIDE 1

𝝂QSIM: ENABLING ACCURATE AND SCALABLE SIMULATION

FOR INTERACTIVE MICROSERVICES

Yanqi Zhang, Yu Gan, Christina Delimitrou Cornell University

Session: Datacenters and Cloud Computing, March 26th

slide-2
SLIDE 2

MAR

ARCHI CHING NG TO TO MIC ICRO ROSER SERVI VICES CES

1

Monolith Application

slide-3
SLIDE 3

MAR

ARCHI CHING NG TO TO MIC ICRO ROSER SERVI VICES CES

1

Monolith Application Microservices

slide-4
SLIDE 4

MAR

ARCHI CHING NG TO TO MIC ICRO ROSER SERVI VICES CES

1

  • Easier development & update

Monolith Application Microservices

slide-5
SLIDE 5

MAR

ARCHI CHING NG TO TO MIC ICRO ROSER SERVI VICES CES

1

  • Easier development & update
  • Debugging & error isolation

Monolith Application Microservices

slide-6
SLIDE 6

MAR

ARCHI CHING NG TO TO MIC ICRO ROSER SERVI VICES CES

1

  • Easier development & update
  • Debugging & error isolation
  • Elasticity

Monolith Application Microservices

slide-7
SLIDE 7

MAR

ARCHI CHING NG TO TO MIC ICRO ROSER SERVI VICES CES

1

  • Easier development & update
  • Debugging & error isolation
  • Elasticity
  • PL/framework heterogeneity

Monolith Application Microservices

slide-8
SLIDE 8

MAR

ARCHI CHING NG TO TO MIC ICRO ROSER SERVI VICES CES

2

Visualization tool: https://github.com/netflix/vizceral.git

Real time traffic of Social Network Microservices

slide-9
SLIDE 9

MAR

ARCHI CHING NG TO TO MIC ICRO ROSER SERVI VICES CES

2

  • Complicate cluster management due to microservice dependencies

Visualization tool: https://github.com/netflix/vizceral.git

Real time traffic of Social Network Microservices

slide-10
SLIDE 10

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

3

slide-11
SLIDE 11

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

  • Performance unpredictability usually occurs at large scale

3

slide-12
SLIDE 12

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

  • Performance unpredictability usually occurs at large scale
  • Leverage simulation

3

slide-13
SLIDE 13

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

  • Performance unpredictability usually occurs at large scale
  • Leverage simulation
  • Architecture simulator: Gem5, Zsim (ISCA’13)
  • Cluster level queueing simulator: BigHouse (ISPASS’12)

3

slide-14
SLIDE 14

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

  • Performance unpredictability usually occurs at large scale
  • Leverage simulation
  • Architecture simulator: Gem5, Zsim (ISCA’13)
  • Cluster level queueing simulator: BigHouse (ISPASS’12)
  • Microservices introduce new simulation requirements

3

slide-15
SLIDE 15

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

  • Performance unpredictability usually occurs at large scale
  • Leverage simulation
  • Architecture simulator: Gem5, Zsim (ISCA’13)
  • Cluster level queueing simulator: BigHouse (ISPASS’12)
  • Microservices introduce new simulation requirements
  • Scalability to study cluster management

3

slide-16
SLIDE 16

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

  • Performance unpredictability usually occurs at large scale
  • Leverage simulation
  • Architecture simulator: Gem5, Zsim (ISCA’13)
  • Cluster level queueing simulator: BigHouse (ISPASS’12)
  • Microservices introduce new simulation requirements
  • Scalability to study cluster management
  • Accurate multi-stage models for individual service/microservice

3

slide-17
SLIDE 17

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

  • Performance unpredictability usually occurs at large scale
  • Leverage simulation
  • Architecture simulator: Gem5, Zsim (ISCA’13)
  • Cluster level queueing simulator: BigHouse (ISPASS’12)
  • Microservices introduce new simulation requirements
  • Scalability to study cluster management
  • Accurate multi-stage models for individual service/microservice
  • Modeling arbitrary dependency graphs and dataflow paths

3

slide-18
SLIDE 18

INF

NFRASTRUCTURE RASTRUCTURE CH CHALL ALLENGES ENGES FOR OR MIC ICROS ROSERVICES ERVICES

  • Performance unpredictability usually occurs at large scale
  • Leverage simulation
  • Architecture simulator: Gem5, Zsim (ISCA’13)
  • Cluster level queueing simulator: BigHouse (ISPASS’12)
  • Microservices introduce new simulation requirements
  • Scalability to study cluster management
  • Accurate multi-stage models for individual service/microservice
  • Modeling arbitrary dependency graphs and dataflow paths
  • Modeling blocking and synchronization behavior

3

slide-19
SLIDE 19

𝝂QSIM

IM DES ESIG IGN

4

slide-20
SLIDE 20

𝝂QSIM

IM DES ESIG IGN

  • Accurate yet scalable service models
  • Event-driven queueing simulator

4

slide-21
SLIDE 21

𝝂QSIM

IM DES ESIG IGN

  • Accurate yet scalable service models
  • Event-driven queueing simulator
  • Simple microservice: single stage

4

Stage 0

slide-22
SLIDE 22

𝝂QSIM

IM DES ESIG IGN

  • Accurate yet scalable service models
  • Event-driven queueing simulator
  • Simple microservice: single stage
  • Complex service: decomposed to multiple stages

4

Stage 0 Stage 0 Stage 1 Stage 2

slide-23
SLIDE 23

𝝂QSIM

IM DES ESIG IGN

  • Accurate yet scalable service models
  • Event-driven queueing simulator
  • Simple microservice: single stage
  • Complex service: decomposed to multiple stages
  • Modeling dependency graph & dataflow path
  • Specify dependency graph & dataflow paths

4

Stage 0 Stage 0 Stage 1 Stage 2

Service A Service B Service A

client server client

slide-24
SLIDE 24

𝝂QSIM

IM DES ESIG IGN

  • Accurate yet scalable service models
  • Event-driven queueing simulator
  • Simple microservice: single stage
  • Complex service: decomposed to multiple stages
  • Modeling dependency graph & dataflow path
  • Specify dependency graph & dataflow paths
  • Modeling blocking/synchronization
  • Encode blocking/synchronization behavior in

dataflow path

4

Stage 0 Stage 0 Stage 1 Stage 2

Service A Service B Service A

client server client

slide-25
SLIDE 25

𝝂QSIM

IM DES ESIG IGN

  • Accurate yet scalable service models
  • Event-driven queueing simulator
  • Simple microservice: single stage
  • Complex service: decomposed to multiple stages
  • Modeling dependency graph & dataflow path
  • Specify dependency graph & dataflow paths
  • Modeling blocking/synchronization
  • Encode blocking/synchronization behavior in

dataflow path

  • Model sources of blocking: network connections,

threads blocked by I/O accesses

4

Stage 0 Stage 0 Stage 1 Stage 2

Service A Service B Service A

client server client

slide-26
SLIDE 26

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
slide-27
SLIDE 27

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
slide-28
SLIDE 28

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice

TCP RX TCP TX

slide-29
SLIDE 29

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)

TCP RX TCP TX

slide-30
SLIDE 30

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)

TCP RX TCP TX Epoll Socket read

slide-31
SLIDE 31

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-32
SLIDE 32

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-33
SLIDE 33

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-34
SLIDE 34

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-35
SLIDE 35

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-36
SLIDE 36

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-37
SLIDE 37

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

Connection A Connection B 3 1 2

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-38
SLIDE 38

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

Connection A Connection B 3 1 2

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-39
SLIDE 39

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

Connection A Connection B 3 1 2

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-40
SLIDE 40

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

Connection A Connection B 3 1 2

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-41
SLIDE 41

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

Connection A Connection B 3 1 2

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-42
SLIDE 42

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

Connection A Connection B 3 1 2

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-43
SLIDE 43

MOD

ODELING ELING INDIVI IVIDU DUAL AL MIC ICRO ROSERVIC ERVICES ES

  • Memcached

5

Connection A Connection B 3 1 2

  • 𝝂qSim microservice model
  • Multiple stages per microservice
  • Provided models for common OS queueing utilities (e.g., epoll & socket read)
  • Each stage optionally coupled with a queue
  • Multiple execution paths (e.g., Memcached read/write) per microservice

TCP RX TCP TX Epoll Socket read Mem$ read Mem$ write Socke t send

slide-44
SLIDE 44

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

6

slide-45
SLIDE 45

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

6

  • Microservice dependency graph

NGINX Memcached MongoDB

slide-46
SLIDE 46

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

6

  • Microservice dependency graph
  • Dataflow
  • Sequence of microservices to execute
  • Differ across request types
  • Dataflow node encodes blocking/synchronization operation

NGINX Memcached MongoDB

slide-47
SLIDE 47

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

6

  • Microservice dependency graph
  • Dataflow
  • Sequence of microservices to execute
  • Differ across request types
  • Dataflow node encodes blocking/synchronization operation
  • Deployment
  • Available servers and hardware resources
  • Service to server mapping
  • Services on the same server share network stack & disk I/O

NGINX Memcached MongoDB

slide-48
SLIDE 48

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

7

slide-49
SLIDE 49

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol

7

slide-50
SLIDE 50

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph

7

NGINX

Memcached

slide-51
SLIDE 51

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph
  • Dataflow

7

NGINX

Memcached

NGINX Memcached NGINX

Block recv connection Unblock recv connection

slide-52
SLIDE 52

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph
  • Dataflow

7

Connection A Connection B

NGINX

Memcached

NGINX Memcached NGINX

NGINX

Memcached

1 3 2

Block recv connection Unblock recv connection

slide-53
SLIDE 53

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph
  • Dataflow

7

Connection A Connection B

NGINX

Memcached

NGINX Memcached NGINX

NGINX

Memcached

1 3 2

Block recv connection Unblock recv connection

slide-54
SLIDE 54

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph
  • Dataflow

7

Connection A Connection B

NGINX

Memcached

NGINX Memcached NGINX

NGINX

Memcached

1 3 2

Block recv connection Unblock recv connection

slide-55
SLIDE 55

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph
  • Dataflow

7

Connection A Connection B

NGINX

Memcached

NGINX Memcached NGINX

NGINX

Memcached

1 3 2

Block recv connection Unblock recv connection

slide-56
SLIDE 56

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph
  • Dataflow

7

Connection A Connection B

NGINX

Memcached

NGINX Memcached NGINX

NGINX

Memcached

1 3 2

Block recv connection Unblock recv connection

slide-57
SLIDE 57

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph
  • Dataflow

7

Connection A Connection B

NGINX

Memcached

NGINX Memcached NGINX

NGINX

Memcached

1 3 2

Block recv connection Unblock recv connection

slide-58
SLIDE 58

MOD

ODELING ELING DEP EPENDEN ENDENCY CY GRA RAPH & DATA ATAFLOW FLOW

  • Example: 2-tier application & http 1/1.1 protocol
  • Dependency graph
  • Dataflow

7

Connection A Connection B

NGINX

Memcached

NGINX Memcached NGINX

NGINX

Memcached

1 3 2

Block recv connection Unblock recv connection

slide-59
SLIDE 59

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post Follow User User Info MongoDB Memcached

slide-60
SLIDE 60

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post

R

Follow User User Info MongoDB Memcached

slide-61
SLIDE 61

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post Follow User User Info MongoDB Memcached

r r r r r r

slide-62
SLIDE 62

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post Follow User User Info MongoDB Memcached

r r r r r r

slide-63
SLIDE 63

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post Follow User User Info MongoDB Memcached

r r r

slide-64
SLIDE 64

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post Follow User User Info MongoDB Memcached

r r r

slide-65
SLIDE 65

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post

R

Follow User User Info MongoDB Memcached

r r r

slide-66
SLIDE 66

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post

R

Follow User User Info MongoDB Memcached

r r r

slide-67
SLIDE 67

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post

R

Follow User User Info MongoDB Memcached

r r r

slide-68
SLIDE 68

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post

R

Follow User User Info MongoDB Memcached

r r r

slide-69
SLIDE 69

DEP

EPENDEN ENDENCY CY GRA RAPH FOR OR COMPL OMPLEX EX MIC ICRO ROSERVICES SERVICES

8

  • Social network application

MongoDB

NGINX

Compose Post Video Image

Text UserTag

Posts Storage Write Timeline Write Graph Memcached MongoDB Memcached MongoDB Memcached Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf Unique ID URL Shorten Read Timeline Read Post

R

Follow User User Info MongoDB Memcached

r r r

slide-70
SLIDE 70

SIM

IMULATOR ULATOR IN INPUTS UTS

9

slide-71
SLIDE 71

SIM

IMULATOR ULATOR IN INPUTS UTS

9

  • Microservice queueing model
  • Identifying sources of queueing
  • Typical queues: network, epoll/kqueue, socket read, disk I/O…
  • Microservice models reusable (open source community)
slide-72
SLIDE 72

SIM

IMULATOR ULATOR IN INPUTS UTS

9

  • Microservice queueing model
  • Identifying sources of queueing
  • Typical queues: network, epoll/kqueue, socket read, disk I/O…
  • Microservice models reusable (open source community)
  • Processing time distribution
  • Instrumenting applications and profiling on real servers
  • Instrumentations reusable
slide-73
SLIDE 73

SIM

IMULATOR ULATOR IN INPUTS UTS

9

  • Microservice queueing model
  • Identifying sources of queueing
  • Typical queues: network, epoll/kqueue, socket read, disk I/O…
  • Microservice models reusable (open source community)
  • Processing time distribution
  • Instrumenting applications and profiling on real servers
  • Instrumentations reusable
  • Microservice dependencies & dataflow paths
  • Obtained from app developers
slide-74
SLIDE 74

SIM

IMULATOR ULATOR IN INPUTS UTS

9

  • Microservice queueing model
  • Identifying sources of queueing
  • Typical queues: network, epoll/kqueue, socket read, disk I/O…
  • Microservice models reusable (open source community)
  • Processing time distribution
  • Instrumenting applications and profiling on real servers
  • Instrumentations reusable
  • Microservice dependencies & dataflow paths
  • Obtained from app developers
  • Server & system resources
slide-75
SLIDE 75

VAL

ALIDATIO IDATION

10

  • Validation experiments
  • Multi-tier microservices: 2/3-tier application
  • Load balancing & fanout effects
  • Microservices based on RPC
  • Comparison with BigHouse
  • Server platform for trace collection
  • 10-server cluster
  • Intel(R) Xeon(R) CPU E5-2660 v3
  • 2 sockets, 10 cores/socket, 2 threads/core
  • Min/max DVFS frequency: 1.2GHz/2.6GHz
  • Network bandwidth: 1Gbps
slide-76
SLIDE 76

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application

NGINX Memcached

slide-77
SLIDE 77

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application

NGINX Memcached

R

slide-78
SLIDE 78

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application

NGINX Memcached

R

slide-79
SLIDE 79

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application

NGINX Memcached

R

slide-80
SLIDE 80

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application

NGINX Memcached

R

slide-81
SLIDE 81

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R

slide-82
SLIDE 82

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R

slide-83
SLIDE 83

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R

slide-84
SLIDE 84

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R

slide-85
SLIDE 85

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R R

slide-86
SLIDE 86

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R R

slide-87
SLIDE 87

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R R

slide-88
SLIDE 88

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R R

slide-89
SLIDE 89

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R R

slide-90
SLIDE 90

2/3-TIER

IER APPLICA ICATION TION

11

  • 2-tier application
  • 3-tier application

NGINX Memcached MongoDB NGINX Memcached

R R R

slide-91
SLIDE 91

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-92
SLIDE 92

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-93
SLIDE 93

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-94
SLIDE 94

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-95
SLIDE 95

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-96
SLIDE 96

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-97
SLIDE 97

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-98
SLIDE 98

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-99
SLIDE 99

LOA

OAD BAL ALAN ANCING CING

12

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-100
SLIDE 100

LOA

OAD BAL ALAN ANCING CING

12

  • Linear throughput increase from cluster size of 4 to 8
  • Sub-linear increase from cluster size of 8 to 16

Load Balancer

NGINX Web Server NGINX Web Server NGINX Web Server

R R

slide-101
SLIDE 101

REQ

EQUEST UEST FAN ANOU OUT

13

NGINX Root Aggregator

NGINX Web Server NGINX Web Server NGINX Web Server

R

slide-102
SLIDE 102

REQ

EQUEST UEST FAN ANOU OUT

13

NGINX Root Aggregator

NGINX Web Server NGINX Web Server NGINX Web Server

R

slide-103
SLIDE 103

REQ

EQUEST UEST FAN ANOU OUT

13

NGINX Root Aggregator

NGINX Web Server NGINX Web Server NGINX Web Server

r r r

slide-104
SLIDE 104

REQ

EQUEST UEST FAN ANOU OUT

13

NGINX Root Aggregator

NGINX Web Server NGINX Web Server NGINX Web Server

r r r

slide-105
SLIDE 105

REQ

EQUEST UEST FAN ANOU OUT

13

NGINX Root Aggregator

NGINX Web Server NGINX Web Server NGINX Web Server

R

slide-106
SLIDE 106

REQ

EQUEST UEST FAN ANOU OUT

13

NGINX Root Aggregator

NGINX Web Server NGINX Web Server NGINX Web Server

R

slide-107
SLIDE 107

REQ

EQUEST UEST FAN ANOU OUT

13

  • Throughput decreases with request fanout

NGINX Root Aggregator

NGINX Web Server NGINX Web Server NGINX Web Server

R

slide-108
SLIDE 108

RPC & LAR

ARGE GE-SCALE CALE MIC ICRO ROSERVIC ERVICES ES

14

  • Simplified social network

Frontend

Post Service Media Service User Service Mem$ Mongo Mem$ Mongo Mem$ Mongo

R RPC

slide-109
SLIDE 109

RPC & LAR

ARGE GE-SCALE CALE MIC ICRO ROSERVIC ERVICES ES

14

  • Simplified social network

Frontend

Post Service Media Service User Service Mem$ Mongo Mem$ Mongo Mem$ Mongo

R RPC

slide-110
SLIDE 110

RPC & LAR

ARGE GE-SCALE CALE MIC ICRO ROSERVIC ERVICES ES

14

  • Simplified social network

Frontend

Post Service Media Service User Service Mem$ Mongo Mem$ Mongo Mem$ Mongo

r r RPC

slide-111
SLIDE 111

RPC & LAR

ARGE GE-SCALE CALE MIC ICRO ROSERVIC ERVICES ES

14

  • Simplified social network

Frontend

Post Service Media Service User Service Mem$ Mongo Mem$ Mongo Mem$ Mongo

r r RPC

slide-112
SLIDE 112

RPC & LAR

ARGE GE-SCALE CALE MIC ICRO ROSERVIC ERVICES ES

14

  • Simplified social network

Frontend

Post Service Media Service User Service Mem$ Mongo Mem$ Mongo Mem$ Mongo

R RPC

slide-113
SLIDE 113

RPC & LAR

ARGE GE-SCALE CALE MIC ICRO ROSERVIC ERVICES ES

14

  • Simplified social network

Frontend

Post Service Media Service User Service Mem$ Mongo Mem$ Mongo Mem$ Mongo

R RPC

slide-114
SLIDE 114

RPC & LAR

ARGE GE-SCALE CALE MIC ICRO ROSERVIC ERVICES ES

14

  • Simplified social network

Frontend

Post Service Media Service User Service Mem$ Mongo Mem$ Mongo Mem$ Mongo

R RPC

slide-115
SLIDE 115

COMPARI

OMPARISON SON WIT ITH BIG IGHOU OUSE SE

15

  • Memcached
  • NGINX

Epoll Socket read Mem$ proc Socket send TCP RX TCP TX Epoll Socket read Nginx proc TCP RX TCP TX

slide-116
SLIDE 116

COMPARI

OMPARISON SON WIT ITH BIG IGHOU OUSE SE

15

  • Memcached
  • NGINX

Epoll Socket read Mem$ proc Socket send TCP RX TCP TX Epoll Socket read Nginx proc TCP RX TCP TX

  • BigHouse saturates at 50kQPS
  • 𝝂qSim & real server saturates at 450kQPS
slide-117
SLIDE 117

COMPARI

OMPARISON SON WIT ITH BIG IGHOU OUSE SE

15

  • Memcached
  • NGINX

Epoll Socket read Mem$ proc Socket send TCP RX TCP TX Epoll Socket read Nginx proc TCP RX TCP TX

  • BigHouse saturates at 50kQPS
  • 𝝂qSim & real server saturates at 450kQPS
  • BigHouse saturates at 2.5kQPS
  • 𝝂qSim & real server saturates at 15kQPS
slide-118
SLIDE 118

SIM

IMULATION ULATION SPEE EED

16

  • Simulation speed relevant factors
  • Simulated input load
  • Processing time distribution
  • Microservice model complexity & network complexity
slide-119
SLIDE 119

SIM

IMULATION ULATION SPEE EED

16

  • Simulation speed relevant factors
  • Simulated input load
  • Processing time distribution
  • Microservice model complexity & network complexity
  • Performance compared to real system
  • Tail@Scale experiment: 26.5x slow down for cluster of 500 servers
  • 4-thread Memcached at 50kQPS: 2.5x speed up
  • 1-process Nginx at 10kQPS: 4x speed up
slide-120
SLIDE 120

SIM

IMULATION ULATION SPEE EED

16

  • Simulation speed relevant factors
  • Simulated input load
  • Processing time distribution
  • Microservice model complexity & network complexity
  • Performance compared to real system
  • Tail@Scale experiment: 26.5x slow down for cluster of 500 servers
  • 4-thread Memcached at 50kQPS: 2.5x speed up
  • 1-process Nginx at 10kQPS: 4x speed up
  • Future work
  • Parallelizing simulation
  • Each thread simulates a partition of the network
slide-121
SLIDE 121

CONCLUS

ONCLUSION ION

17

  • Microservices introduce new system challenges
  • Need scalable simulation techniques to study large scale effects
  • 𝝂qSim: validated microservice simulator
  • Modeling the internal queueing structure of individual microservices
  • Modeling dataflow behavior across the microservices
  • Validated against simple & complex microservices and accurately capturing

throughput/latency

  • Planning to open source @ microservices.ece.cornell.edu
slide-122
SLIDE 122

THAN

ANKS KS & Q

QUE

UESTIONS TIONS

18

  • Microservices introduce new system challenges
  • Need scalable simulation techniques to study large scale effects
  • 𝝂qSim: validated microservice simulator
  • Modeling the internal queueing structure of individual microservices
  • Modeling dataflow behavior across the microservices
  • Validated against simple & complex microservices and accurately capturing

throughput/latency

  • Planning to open source @ microservices.ece.cornell.edu
slide-123
SLIDE 123

SPEE

EED COMPARI OMPARISON SON WIT ITH BIG IGHOUS OUSE (B

(BAC

ACKUP KUP)

19

𝝂qSim simulated real time 5s 20s 60s 𝝂qSim simulation time 12.8 s 51.2s 154.1 s BigHouse warmup samples 5 x50k 20 x50k 60 x50k 5000 BigHouse simulation time 8.2s 28.7s 154s 1.2s 𝝂qSim simulated real time 5s 20s 60s 𝝂qSim simulation time 0.7s 2.7s 8.1s BigHouse warmup samples 5 x2k 20 x2k 60 x2k 5000 BigHouse simulation time 9s 23s 64s 7.1s

4-thread Memcached at 50k 1-process NGINX at 2k

slide-124
SLIDE 124

USE

SE CAS ASES ES – TAI AIL@S

@SCA

CALE LE (B

(BAC

ACKUP UP)

20

  • For simple single-tier single stage
  • Different Cluster Sizes
  • Different Fraction of Slow Servers
slide-125
SLIDE 125

USE

SE CAS ASES ES – POW OWER ER MAN ANAGEMEN AGEMENT (B

(BAC

ACKU KUP)

21

  • End-to-End QoS Target
  • 5ms
  • Input Load
  • Diurnal pattern
  • Results