The Only Constant is Change: Incorporating Time-Varying Bandwidth - - PowerPoint PPT Presentation

The Only Constant is Change: Incorporating Time-Varying Bandwidth Reservations in Data Centers

Di Xie, Ning Ding, Y. Charlie Hu, Ramana Kompella

1

Cloud Computing is Hot

2

Private Cluster

Key Factors for Cloud Viability

• Cost
• Performance

3

Performance Variability in Cloud

• BW variation in cloud

due to contention [Schad’10 VLDB]

• Causing unpredictable

performance

4

100 200 300 400 500 600 700 800 900 1000

Local Cluster Amazon EC2

Bandwidth (Mbps)

Reserving BW in Data Centers

• SecondNet [Guo’10]

– Per VM-pair, per VM access bandwidth reservation

• Oktopus [Ballani’11]

– Virtual Cluster (VC) – Virtual Oversubscribed Cluster (VOC)

5

How BW Reservation Works

6

. . . Virtual Cluster Model Time Bandwidth N VMs Virtual Switch

• 1. Determine the model
• 2. Allocate and enforce the model

T B

Only fixed-BW reservation Request <N, B>

Network Usage for MapReduce Jobs

Hadoop Sort, 4GB per VM

7

Network Usage for MapReduce Jobs

Hadoop Sort, 4GB per VM Hadoop Word Count, 2GB per VM

8

Network Usage for MapReduce Jobs

Hadoop Sort, 4GB per VM Hadoop Word Count, 2GB per VM Hive Join, 6GB per VM

9

Network Usage for MapReduce Jobs

Hadoop Sort, 4GB per VM Hadoop Word Count, 2GB per VM Hive Join, 6GB per VM Hive Aggregation, 2GB per VM

10

Network Usage for MapReduce Jobs

Hadoop Sort, 4GB per VM Hadoop Word Count, 2GB per VM Hive Join, 6GB per VM Hive Aggregation, 2GB per VM

11

Time-varying network usage

Motivating Example

• 4 machines,

2 VMs/machine, non-oversubscribed network

• Hadoop Sort

– N: 4 VMs – B: 500Mbps/VM

1Gbps 500Mbps 500Mbps

Not enough BW

12

Motivating Example

• 4 machines,

2 VMs/machine, non-oversubscribed network

• Hadoop Sort

– N: 4 VMs – B: 500Mbps/VM

13

1Gbps 500Mbps

Under Fixed-BW Reservation Model

14

1Gbps 500Mbps Job3 Job2 Virtual Cluster Model Job1 Time 0 5 10 15 20 25 30 500 Bandwidth

15

16

17

18

19

Temporally-Interleaved Virtual Cluster (TIVC)

• Key idea: Time-Varying BW Reservations
• Compared to fixed-BW reservation

– Improves utilization of data center

• Better network utilization
• Better VM utilization

– Increases cloud provider’s revenue – Reduces cloud user’s cost – Without sacrificing job performance

20

Challenges in Realizing TIVC

21

. . . Virtual Cluster Model Time Bandwidth N VMs Virtual Switch T B

Request <N, B>

Time Bandwidth T B

Request <N, B(t)>

Q1: What are right model functions? Q2: How to automatically derive the models?

Challenges in Realizing TIVC

22

Q3: How to efficiently allocate TIVC? Q4: How to enforce TIVC?

Challenges in Realizing TIVC

• What are the right model functions?
• How to automatically derive the models?
• How to efficiently allocate TIVC?
• How to enforce TIVC?

23

Challenges in Realizing TIVC

• What are the right model functions?
• How to automatically derive the models?
• How to efficiently allocate TIVC?
• How to enforce TIVC?

24

How to Model Time-Varying BW?

25

Hadoop Hive Join

TIVC Models

26

Virtual Cluster

Bandwidth Time

B T1 T2 T Bb

Bandwidth Time

T31 B Bb T11 T12 T21 T22 T32 T

Bandwidth Time

T31 B Bb T11 T12 T21 T22 T32 T

T11 T32

Hadoop Sort

27

Hadoop Word Count

28

v

Hadoop Hive Join

29

Hadoop Hive Aggregation

30

Challenges in Realizing TIVC

What are the right model functions?

• How to automatically derive the models?
• How to efficiently allocate TIVC?
• How to enforce TIVC?

31

Possible Approach

• “White-box” approach

– Given source code and data of cloud application, analyze quantitative networking requirement – Very difficult in practice

• Observation: Many jobs are repeated many times

– E.g., 40% jobs are recurring in Bing’s production data center [Agarwal’12] – Of course, data itself may change across runs, but size remains about the same

32

Our Approach

• Solution: “Black-box” profiling based approach
• 1. Collect traffic trace from profiling run
• 2. Derive TIVC model from traffic trace
• Profiling: Same configuration as production runs

– Same number of VMs – Same input data size per VM – Same job/VM configuration

33

How much BW should we give to the application?

Impact of BW Capping

34

Impact of BW Capping

35

No-elongation BW threshold

Choosing BW Cap

• Tradeoff between performance and cost

– Cap > threshold: same performance, costs more – Cap < threshold: lower performance, may cost less

• Our Approach: Expose tradeoff to user
• 1. Profile under different BW caps
• 2. Expose run times and cost to user
• 3. User picks the appropriate BW cap

36

Only below threshold ones

From Profiling to Model Generation

• Collect traffic trace from each VM

– Instantaneous throughput of 10ms bin

• Generate models for individual VMs
• Combine to obtain overall job’s TIVC model

– Simplify allocation by working with one model – Does not lose efficiency since per-VM models are roughly similar for MapReduce-like applications

37

Generate Model for Individual VM

• 1. Choose Bb
• 2. Periods where B > Bb, set to Bcap

38

BW Time Bcap Bb

Maximal Efficiency Model

• Enumerate Bb to find the maximal efficiency

model

39

Volume Bandwdith Reserved Volume Traffic n Applicatio Efficiency 

BW Time Bcap Bb

Challenges in Realizing TIVC

What are the right model functions? How to automatically derive the models?

• How to efficiently allocate TIVC?
• How to enforce TIVC?

40

TIVC Allocation Algorithm

• Spatio-temporal allocation algorithm

– Extends VC allocation algorithm to time dimension – Employs dynamic programming

• Properties

– Locality aware – Efficient and scalable

• 99th percentile 28ms on a 64,000-VM data center in

scheduling 5,000 jobs

41

Challenges in Realizing TIVC

What are the right model functions? How to automatically derive the models? How to efficiently allocate TIVC?

• How to enforce TIVC?

42

Enforcing TIVC Reservation

• Possible to enforce completely in hypervisor

– Does not have control over upper level links – Requires online rate monitoring and feedback – Increases hypervisor overhead and complexity

• Observation: Few jobs share a link simultaneously

– Most small jobs will fit into a rack – Only a few large jobs cross the core – In our simulations, < 26 jobs share a link in 64,000-VM data center

43

Enforcing TIVC Reservation

• Enforcing BW reservation in switches

– Avoid complexity in hypervisors – Can be implemented on commodity switches

• Cisco Nexus 7000 supports 16k policers

44

Challenges in Realizing TIVC

What are the right model functions? How to automatically derive the models? How to efficiently allocate TIVC? How to enforce TIVC?

45

Proteus: Implementing TIVC Models

46

• 1. Determine the model
• 2. Allocate and enforce the model

Evaluation

• Large-scale simulation

– Performance – Cost – Allocation algorithm

• Prototype implementation

– Small-scale testbed

47

Simulation Setup

• 3-level tree topology

– 16,000 Hosts x 4 VMs – 4:1 oversubscription

• Workload

– N: exponential distribution around mean 49 – B(t): derive from real Hadoop apps

48

50Gbps 10Gbps

… … …

1Gbps

…

20 Aggr Switch 20 ToR Switch 40 Hosts

… … …

Batched Jobs

• Scenario: 5,000 time-insensitive jobs

49

42% 21% 23% 35% 1/3 of each type Completion time reduction All rest results are for mixed

Varying Oversubscription and Job Size

50

25.8% reduction for non-oversubscribed network

Dynamically Arriving Jobs

• Scenario: Accommodate users’ requests in

shared data center

– 5,000 jobs, Poisson arrival, varying load

51

Rejected: VC: 9.5% TIVC: 3.4%

Analysis: Higher Concurrency

• Under 80% load

52

7% higher job concurrency 28% higher VM utilization

Rejected jobs are large

28% higher revenue

Charge VMs

VM

Tenant Cost and Provider Revenue

• Charging model

– VM time T and reserved BW volume B – Cost = N (kv T + kb B) – kv = 0.004$/hr, kb = 0.00016$/GB

53

12% less cost for tenants Providers make more money Amazon target utilization

Testbed Experiment

• Setup

– 18 machines – Tc and NetFPGA rate limiter

• Real MapReduce jobs
• Procedure

– Offline profiling – Online reservation

54

Testbed Result

55

TIVC finishes job faster than VC, Baseline finishes the fastest Baseline suffers elongation, TIVC achieves similar performance as VC

Conclusion

• Network reservations in cloud are important

– Previous work proposed fixed-BW reservations – However, cloud apps exhibit time-varying BW usage

• We propose TIVC abstraction

– Provides time-varying network reservations – Uses simple pulse functions – Automatically generates model – Efficiently allocates and enforces reservations

• Proteus shows TIVC benefits both cloud provider

and users significantly

56