[PPT] - Q UASAR : R ESOURCE -E FFICIENT A ND Q O S-A WARE C LUSTER M PowerPoint Presentation

SLIDE 1

Christina Delimitrou and Christos Kozyrakis

Stanford University http://mast.stanford.edu

ASPLOS – March 3rd 2014

QUASAR: RESOURCE-EFFICIENT AND QOS-AWARE CLUSTER MANAGEMENT

SLIDE 2

2

 Problem: low datacenter utilization

 Overprovisioned reservations by users

 Problem: high jitter on application performance

 Interference, HW heterogeneity

 Quasar: resource-efficient cluster management

 User provides resource reservations performance goals  Online analysis of resource needs using info from past apps  Automatic selection of number & type of resources  High utilization and low performance jitter

Executive Summary

SLIDE 3

3

Datacenter Underutilization

 A few thousand server cluster at Twitter managed by Mesos  Running mostly latency-critical, user-facing apps

80% of servers @ < 20% utilization Servers are 65% of TCO

SLIDE 4

4

Datacenter Underutilization

 Goal: raise utilization without introducing performance jitter

1 L. A. Barroso, U. Holzle. The Datacenter as a Computer, 2009.

Twitter Google1

SLIDE 5

5

Reserved vs. Used Resources

 Twitter: up to 5x CPU & up to 2x memory overprovisioning

3-5x 1.5-2x

SLIDE 6

6

Reserved vs. Used Resources

 20% of job under-sized, ~70% of jobs over-sized

SLIDE 7

7

Rightsizing Applications is Hard

SLIDE 8

8

Performance Depends on

Cores Performance Scale-up

SLIDE 9

9

Performance Depends on

Cores Performance Heterogeneity

SLIDE 10

10

Performance Depends on

Cores Performance Heterogeneity Servers Performance Scale-out

SLIDE 11

11

Performance Depends on

Cores Performance Heterogeneity Servers Performance Scale-out Input size Performance Input load

SLIDE 12

12

Performance Depends on

Cores Performance Heterogeneity Cores Performance Interference Input size Performance Input load Servers Performance Scale-out

When sw changes, when platforms change, etc.

SLIDE 13

13

Performance Depends on

Cores Performance Heterogeneity Cores Performance Interference Input size Performance Input load Servers Performance Scale-out

When sw changes, when platforms change, etc.

SLIDE 14

14

Rethinking Cluster Management

 User provides resource reservations performance goals  Joint allocation and assignment of resources

 Right amount depends on quality of available resources  Monitor and adjust dynamically as needed

 But wait…

 The manager must know the resource/performance tradeoffs

SLIDE 15

15

Big cluster data

 Combine:

 Small signal from short run of new app  Large signal from previously-run apps

 Generate:  Detailed insights for resource management  Performance vs scale-up/out,

heterogeneity, …

 Looks like a classification problem

Resource/performance tradeoffs

Small app signal

Understanding Resource/Performance Tradeoffs

SLIDE 16

16

Something familiar…

 Collaborative filtering – similar to Netflix Challenge system

 Predict preferences of new users given preferences of other users  Singular Value Decomposition (SVD) + PQ reconstruction (SGD)  High accuracy, low complexity, relaxed density constraints

Sparse utility matrix Initial decomposition

SVD PQ SGD

Reconstructed utility matrix Final decomposition

SVD

movies users

SLIDE 17

17

Application Analysis with Classification

 4 parallel classifications

 Lower overheads & similar accuracy to exhaustive classification

Rows Columns Recommendation Netflix Users Movies Movie ratings Heterogeneity Interference Scale-up Scale-out

SLIDE 18

18

Heterogeneity Classification

 Profiling on two randomly selected server types  Predict performance on each server type

Rows Columns Recommendation Netflix Users Movies Movie ratings Heterogeneity Apps Platforms Server type Interference Scale-up Scale-out

SLIDE 19

19

Interference Classification

 Predict sensitivity to interference

 Interference intensity that leads to >5% performance loss

 Profiling by injecting increasing interference

Rows Columns Recommendation Netflix Users Movies Movie ratings Heterogeneity Apps Platforms Server type Interference Apps Sources of interference Interference sensitivity Scale-up Scale-out

SLIDE 20

20

Scale-Up Classification

 Predict speedup from scale-up  Profiling with two allocations (cores & memory)

Rows Columns Recommendation Netflix Users Movies Movie ratings Heterogeneity Apps Platforms Server type Interference Apps Sources of interference Interference sensitivity Scale-up Apps Resource vectors Resources/node Scale-out

SLIDE 21

21

Scale-Out Classification

 Predict speedup from scale-out  Profiling with two allocations (1 & N>1 nodes)

Rows Columns Recommendation Netflix Users Movies Movie ratings Heterogeneity Apps Platforms Server type Interference Apps Sources of interference Interference sensitivity Scale-up Apps Resource vectors Resources/node Scale-out Apps Nodes Number of nodes

SLIDE 22

22

Classification Validation

Heterogeneity Interference Scale-up Scale-out

avg

max

avg max avg max avg max

Single-node

4% 8% 5% 10% 4% 9%

Batch distributed

4% 5% 2% 6% 5% 11% 5% 17%

Latency-critical

5% 6% 7% 10% 6% 11% 6% 12%

SLIDE 23

23

Quasar Overview

QoS

SLIDE 24

24

Quasar Overview

QoS

SLIDE 25

25

Quasar Overview

QoS

H <a..b...> I <..cd...> SU <e..f..> SO <kl....>

SLIDE 26

26

Quasar Overview

H <a..b...> I <..cd...> SU <e..f..> SO <kl....>

UΣVT

           

mn m m n n

u u u u u u u u u ... ... ...

2 1 2 22 21 1 12 11

   

H <abcdefghi> I <qwertyuio> SU <esdfghjkl> SO <kljhgfdsa>

QoS

SLIDE 27

27

Quasar Overview

H <a..b...> I <..cd...> SU <e..f..> SO <kl....>

UΣVT

           

mn m m n n

u u u u u u u u u ... ... ...

2 1 2 22 21 1 12 11

   

H <abcdefghi> I <qwertyuio> SU <esdfghjkl> SO <kljhgfdsa>

QoS

SLIDE 28

28

Quasar Overview

H <a..b...> I <..cd...> SU <e..f..> SO <kl....>

UΣVT

           

mn m m n n

u u u u u u u u u ... ... ...

2 1 2 22 21 1 12 11

   

H <abcdefghi> I <qwertyuio> SU <esdfghjkl> SO <kljhgfdsa>

Profiling [10-60sec] Sparse input signal Classification [20msec] Dense

utput

signal Resource selection [50msec-2sec]

QoS

SLIDE 29

29

Greedy Resource Selection

 Goals  Allocate least needed resources to meet QoS target  Pack together non-interfering applications  Overview

 Start with most appropriate server types  Look for servers with interference below critical intensity  Depends on which applications are running on these servers  First scale-up, next scale-out

SLIDE 30

30

Quasar Implementation

 6,000 loc of C++ and Python  Runs on Linux and OS X  Supports frameworks in C/C++, Java and Python

 ~100-600 loc for framework-specific code

 Side-effect free profiling using Linux containers with chroot

SLIDE 31

31

Evaluation: Cloud Scenario

 Cluster

 200 EC2 servers, 14 different server types

 Workloads: 1,200 apps with 1sec inter-arrival rate  Analytics: Hadoop, Spark, Storm  Latency-critical: Memcached, HotCrp, Cassandra  Single-threaded: SPEC CPU2006  Multi-threaded: PARSEC, SPLASH-2, BioParallel, Specjbb  Multiprogrammed: 4-app mixes of SPEC CPU2006

 Objectives: high cluster utilization and good app QoS

SLIDE 32

32

Demo

Instance Size

memcached Cassandra Storm Hadoop Single-node Spark

Core Allocation Map Quasar

Instance Core

Core Allocation Map Reservation & LL

Performance histogram Quasar Performance histogram Reservation & LL

Cluster Utilization

Progress bar Progress bar

100% 0% 100% 0%

Quasar Reservation + LL

SLIDE 33

33

SLIDE 34

34

Quasar achieves:

 88% of applications get >95% performance  ~10% overprovisioning as opposed to up to 5x  Up to 70% cluster utilization at steady-state  23% shorter scenario completion time

Cloud Scenario Summary

SLIDE 35

35

Conclusions

 Quasar: high utilization, high app performance  From reservation to performance-centric cluster management  Uses info from previous apps for accurate & online app analysis  Joint resource allocation and resource assignment  See paper for:  Utilization analysis of Twitter cluster  Detailed validation & sensitivity analysis of classification  Further evaluation scenarios and features

 E.g., setting framework parameters for Hadoop

SLIDE 36

36

 Quasar: high utilization, high app performance  From reservation to performance-centric cluster management  Uses info from previous apps for accurate & online app analysis  Joint resource allocation and resource assignment  See paper for:  Utilization analysis of Twitter cluster  Detailed validation & sensitivity analysis of classification  Further evaluation scenarios and features

 E.g., setting framework parameters for Hadoop

Questions??

SLIDE 37

37

Thank you Questions??

SLIDE 38

38

Cloud Provider: Performance

SLIDE 39

39

Cloud Provider: Performance

 Most applications violate their QoS constraints

SLIDE 40

40

Cloud Provider: Performance

 83% of performance target when only assignment is heterogeneity &

interference aware

SLIDE 41

41

Cloud Provider: Performance

 98% of performance target on average

SLIDE 42

42

Cluster Utilization

 Baseline (Reservation+LL):

 Imbalance in server utilization  Per-app QoS violations + higher execution time

 Quasar increases server utilization by 47%

 High performance for user  Better utilization for DC operator  resource efficiency

Quasar Least-Loaded (LL)

SLIDE 43

43

Reducing Overprovisioning

 ~10% overprovisioning, compared to 40%-5x for Reservation+LL

SLIDE 44

44

Scheduling Overheads

 4.1% of execution time on average, up to 15% for short-lived

workloads – mostly from profiling

Distributed decisions Cold-start solutions