Empirical Evaluation of Latency-Sensitive Application Performance - - PowerPoint PPT Presentation

Department of Computer Science

Empirical Evaluation of Latency-Sensitive Application Performance in the Cloud

Sean Barker and Prashant Shenoy

University of Massachusetts Amherst

University of Massachusetts Amherst - Department of Computer Science

Cloud Computing

! Cloud platforms built with data centers: large-scale, concentrated servers clusters

• Machines rented out to

companies or individuals

• Hosting for arbitrary applications
• May supplement local resources

! Cheap enough to rent machines by the hour

2

Type CPUs Memory Disk Cost/hr Small 1 1.7 GB 160 GB $0.085 Large 4 7.5 GB 850 GB $0.34 XL 8 15 GB 1690 GB $0.68

Current prices on Amazon Elastic Compute Cloud (EC2)

University of Massachusetts Amherst - Department of Computer Science

Multimedia Cloud Computing Scenarios

! Clouds designed primarily for web & e-commerce apps, but may also be used for multimedia ! Rent game server for an evening

• No firewall or bandwidth issues, only a few dollars

! Rent high-CPU machines for HD video transcoding

• Home PC may take several hours to transcode one video,

cloud can transcode many in a fraction of this time

! Rent servers for webcast of live event

• Large, inexpensive temporary bandwidth allocation

3

! Data center servers are typically well-equipped

• Providers share individual

machines machines among multiple users

! Example: one user runs game server, another runs high-performance database on same machine ! Multimedia has unique performance requirements

• Low latency games, low jitter & high bandwidth streaming

! Are cloud platforms designed for conventional web applications suitable for multimedia?

University of Massachusetts Amherst - Department of Computer Science

Resource Sharing in the Cloud

4

8 GB RAM

Core 1 Core 2 Core 3 Core 4

1000 GB Disk 1000 GB Disk

4 GB RAM

Core 1 Core 2 Core 3 Core 4

1000 GB Disk 1000 GB Disk

4 GB RAM

University of Massachusetts Amherst - Department of Computer Science

Outline

! Motivation ! Virtualized clouds ! Amazon EC2 study ! Laboratory cloud study ! Real world multimedia case studies ! Related work & conclusions

5

University of Massachusetts Amherst - Department of Computer Science

Virtualized Clouds

! Cloud platforms are virtualized data centers ! Virtualization facilitates machine distribution among multiple users with virtual machines (VMs)

6

VM

Hardware

VM VM Game Server Web Server Media Server

Customer A

Users

Customer C Customer B

! Each VM is assigned slice of physical resources ! VM access to hardware managed by hypervisor

• Enforces limits and isolates VMs from each other

! Are these resource sharing mechanisms suitable for the timeliness constraints of multimedia?

VM

VM

VM

App A App C

Users App B

Hardware Hypervisor University of Massachusetts Amherst - Department of Computer Science

Virtual Machine Isolation

8

resource starvation

Hypervisor VM VM VM App A

Users

Hardware App B App C

University of Massachusetts Amherst - Department of Computer Science

Outline

! Motivation ! Virtualized clouds ! Amazon EC2 study ! Laboratory cloud study ! Real world multimedia case studies ! Related work & conclusions

9

University of Massachusetts Amherst - Department of Computer Science

EC2 Study – Overview

! Amazon Elastic Compute Cloud (EC2)

• Popular virtualized cloud platform

! Unknown applications coexisting on machine

• No control over VM placement

! Goal: evaluate performance with unknown background server load ! Methodology: measured CPU, disk, and network consistency over period of days

10

University of Massachusetts Amherst - Department of Computer Science

EC2 CPU Performance

200 400 600 800 1000 1200 1400 CPU time (ms) Time (5 minute intervals) EC2 Local

11

• Volatility on EC2 vs stability on dedicated server

2.5x average

• utliers:

1.5-2x avg

no competing VMs: no outliers

University of Massachusetts Amherst - Department of Computer Science

EC2 Disk Performance

10000 20000 30000 40000 50000 60000 70000 80000 90000 Long write time (ms) Time (5 minute intervals) EC2 Local

12

• Similarly: inconsistent EC2 disk performance

widely fluctuating disk performance

University of Massachusetts Amherst - Department of Computer Science

EC2 Network Latency (LAN)

50 100 150 200 250 First three hops latency (ms) Time (5 minute intervals)

13

• Latency variations in EC2 LAN

University of Massachusetts Amherst - Department of Computer Science

EC2 Study – Summary

! Performance variations observed on EC2

• Not observed on local server running a single VM

! Can only speculate on causes without access to the hypervisor ! Need to experiment on a controlled platform similar to Amazon’s

14

University of Massachusetts Amherst - Department of Computer Science

Laboratory Cloud Study – Overview

! Local cloud running the Xen hypervisor

• Same virtualization technology used by EC2
• Advantage: local cloud gives us control of interference

! Built-in mechanisms for sharing hardware between VMs

• CPU credit scheduler
• Round-robin disk servicing
• Linux-level tool tc for network sharing

! How well do these tools isolate background work? ! Methodology: evaluated performance impact of competing VM

15

University of Massachusetts Amherst - Department of Computer Science

CPU Performance with Background Load

50 100 150 200 CPU time (ms) Time (5 second intervals)

16

• Default 1 to 1 sharing with variable background load

No background work: VM gets 100% CPU Max background work: VM gets 50% CPU

University of Massachusetts Amherst - Department of Computer Science

Disk Performance with Background Load

20 40 60 80 100 1 2 3 4 8 Performance Impact (%) Disk Thread Pairs on Collocated VM Fair Share Small Read Small Write Read Throughput Write Throughput

17

• Degraded by half over ‘fair’, but stable with increasing load

‘unfair’ impact

University of Massachusetts Amherst - Department of Computer Science

Laboratory Cloud Study – Summary

! Significant interference possible from background VMs ! Xen configuration can guarantee share of CPU

• Default settings allow fluctuation in shared CPU

! Disk sharing less fair and harder to control

• Consistent with observed EC2 behavior

! Network sharing effects evaluated in case studies on laboratory cloud (next)

18

University of Massachusetts Amherst - Department of Computer Science

Case Study 1 – Doom 3 Game Server

! Multiplayer Doom 3 game server ! Introduced controlled interference as before ! Measured map load times and server latency ! Network sharing configuration via tc:

• Idle: No bandwidth usage by resource-hog VM
• Off (default): No rate-limiting, network free-for-all
• Shared: 50% (min) to 100% (max) of bandwidth per VM
• Dedicated: 50% (max) of bandwidth per VM

19

University of Massachusetts Amherst - Department of Computer Science

Game Server Map Load

1000 2000 3000 4000 5000 Idle Disk CPU Disk + CPU Average Server Load Time (ms) Collocated VM Activity

20

• Interference produces up to 50% degradation

University of Massachusetts Amherst - Department of Computer Science

Game Server Latency

21

! Server crippled without bandwidth controls (tc off) ! Dedicated vs shared bandwidth:

• Dedicated: lower latency, higher jitter
• Sharing: higher latency, lower jitter

Configuration

• Avg. Latency

(ms)

• Std. Deviation

(jitter) Timeouts No interference

8.1 10.2 0%

tc off (free-for-all)

N/A N/A 100%

tc, sharing b/w

33.9 16.9 2%

tc, dedicated b/w

23.6 29.6 7%

University of Massachusetts Amherst - Department of Computer Science

Case Study 2 – Darwin Streaming Server

! Streaming video to multiple clients ! Introduced controlled interference as before ! Measured sustained streaming bandwidth and stream jitter (latency variation) ! Varied tc settings and number of clients

• Max video stream rate of 1 Mbps per client

22

University of Massachusetts Amherst - Department of Computer Science

Streaming Server Bandwidth

200 400 600 800 1000 idle (fair)

• ff

shared dedicated average bitrate per stream (kbps) tc sharing type 4 streams 8 streams

23

• both tc configurations recovered bandwidth

decreased stream quality

University of Massachusetts Amherst - Department of Computer Science

Streaming Server Jitter

2 4 6 8 10 12 14 16 idle (fair)

• ff

shared dedicated average stream jitter (ms) tc sharing type 4 streams 8 streams

24

• Jitter improved by shared, but worsened by dedicated

University of Massachusetts Amherst - Department of Computer Science

Real World Case Studies – Summary

! Real applications show substantial impacts from background interference ! Network is particularly vulnerable without administrative controls ! Proper configuration is important

• CPU and network isolation tools fairly well-developed
• Disk isolation needs better mechanisms

25

University of Massachusetts Amherst - Department of Computer Science

Related Work

! Fair-share schedulers and quality-of-service

• Nieh and Lam (SOSP ‘97) for multimedia
• Sundaram et al. (ACM MM ‘00) for QoS-aware OS

! Virtualization and hypervisors

• Xen, VMware ESX Server

! Improving performance isolation

• Gupta et al. (Middleware ‘06) for Xen mechanisms

! We focus on evaluation of existing mechanisms with specific attention to multimedia

26

University of Massachusetts Amherst - Department of Computer Science

Conclusions

! Clouds exhibit performance variations

• Applications with timeliness requirements are

particularly sensitive

! Appropriate hypervisor configuration can help

• In some cases, prevents resource starvation
• Some resource sharing mechanisms need improvement

! Future work: evaluation of non-Xen platforms ! Questions?

27