Empirical Evaluation of Latency-Sensitive Application Performance - PowerPoint PPT Presentation

Empirical Evaluation of Latency-Sensitive Application Performance in the Cloud Sean Barker and Prashant Shenoy University of Massachusetts Amherst Department of Computer Science

Cloud Computing ! Cloud platforms built with data centers: large-scale, concentrated servers clusters • Machines rented out to companies or individuals • Hosting for arbitrary applications • May supplement local resources ! Cheap enough to Type CPUs Memory Disk Cost/hr rent machines by Small 1 1.7 GB 160 GB $0.085 the hour Large 4 7.5 GB 850 GB $0.34 XL 8 15 GB 1690 GB $0.68 Current prices on Amazon Elastic Compute Cloud (EC2) University of Massachusetts Amherst - Department of Computer Science 2

Multimedia Cloud Computing Scenarios ! Clouds designed primarily for web & e-commerce apps, but may also be used for multimedia ! Rent game server for an evening • No firewall or bandwidth issues, only a few dollars ! Rent high-CPU machines for HD video transcoding • Home PC may take several hours to transcode one video, cloud can transcode many in a fraction of this time ! Rent servers for webcast of live event • Large, inexpensive temporary bandwidth allocation University of Massachusetts Amherst - Department of Computer Science 3

Resource Sharing in the Cloud ! Data center servers are Core Core Core Core Core Core Core Core typically well-equipped 1 2 3 4 1 2 3 4 • Providers share individual 8 GB RAM 4 GB RAM 4 GB RAM machines machines among multiple users 1000 GB 1000 GB 1000 GB 1000 GB Disk Disk Disk Disk ! Example: one user runs game server, another runs high-performance database on same machine ! Multimedia has unique performance requirements • Low latency games, low jitter & high bandwidth streaming ! Are cloud platforms designed for conventional web applications suitable for multimedia? University of Massachusetts Amherst - Department of Computer Science 4

Outline ! Motivation ! Virtualized clouds ! Amazon EC2 study ! Laboratory cloud study ! Real world multimedia case studies ! Related work & conclusions University of Massachusetts Amherst - Department of Computer Science 5

Virtualized Clouds ! Cloud platforms are virtualized data centers ! Virtualization facilitates machine distribution among multiple users with virtual machines (VMs) Users Customer A Customer C Game Web Media Server Server Server VM VM VM Hardware Customer B University of Massachusetts Amherst - Department of Computer Science 6

Virtual Machine Isolation ! Each VM is assigned slice of physical resources ! VM access to hardware managed by hypervisor • Enforces limits and isolates VMs from each other Users Users resource starvation App App App B App A App B App C A C VM VM VM VM VM VM Hypervisor Hypervisor Hardware Hardware ! Are these resource sharing mechanisms suitable for the timeliness constraints of multimedia? University of Massachusetts Amherst - Department of Computer Science 8

Outline ! Motivation ! Virtualized clouds ! Amazon EC2 study ! Laboratory cloud study ! Real world multimedia case studies ! Related work & conclusions University of Massachusetts Amherst - Department of Computer Science 9

EC2 Study – Overview ! Amazon Elastic Compute Cloud (EC2) • Popular virtualized cloud platform ! Unknown applications coexisting on machine • No control over VM placement ! Goal: evaluate performance with unknown background server load ! Methodology: measured CPU, disk, and network consistency over period of days University of Massachusetts Amherst - Department of Computer Science 10

EC2 CPU Performance 1400 EC2 Local 2.5x 1200 average outliers: 1.5-2x avg 1000 CPU time (ms) 800 600 400 no competing VMs: no outliers 200 0 Time (5 minute intervals) • Volatility on EC2 vs stability on dedicated server University of Massachusetts Amherst - Department of Computer Science 11

EC2 Disk Performance 90000 EC2 Local 80000 70000 Long write time (ms) 60000 50000 40000 widely fluctuating 30000 disk performance 20000 10000 0 Time (5 minute intervals) • Similarly: inconsistent EC2 disk performance University of Massachusetts Amherst - Department of Computer Science 12

EC2 Network Latency (LAN) 250 First three hops latency (ms) 200 150 100 50 0 Time (5 minute intervals) • Latency variations in EC2 LAN University of Massachusetts Amherst - Department of Computer Science 13

EC2 Study – Summary ! Performance variations observed on EC2 • Not observed on local server running a single VM ! Can only speculate on causes without access to the hypervisor ! Need to experiment on a controlled platform similar to Amazon’s University of Massachusetts Amherst - Department of Computer Science 14

Laboratory Cloud Study – Overview ! Local cloud running the Xen hypervisor • Same virtualization technology used by EC2 • Advantage: local cloud gives us control of interference ! Built-in mechanisms for sharing hardware between VMs • CPU credit scheduler • Round-robin disk servicing • Linux-level tool tc for network sharing ! How well do these tools isolate background work? ! Methodology: evaluated performance impact of competing VM University of Massachusetts Amherst - Department of Computer Science 15

CPU Performance with Background Load 200 Max background work: 150 VM gets 50% CPU CPU time (ms) 100 50 No background work: VM gets 100% CPU 0 Time (5 second intervals) • Default 1 to 1 sharing with variable background load University of Massachusetts Amherst - Department of Computer Science 16

Disk Performance with Background Load 100 80 Performance Impact (%) 60 ‘unfair’ impact 40 Fair Share 20 Small Read Small Write Read Throughput Write Throughput 0 1 2 3 4 8 Disk Thread Pairs on Collocated VM • Degraded by half over ‘fair’, but stable with increasing load University of Massachusetts Amherst - Department of Computer Science 17

Laboratory Cloud Study – Summary ! Significant interference possible from background VMs ! Xen configuration can guarantee share of CPU • Default settings allow fluctuation in shared CPU ! Disk sharing less fair and harder to control • Consistent with observed EC2 behavior ! Network sharing effects evaluated in case studies on laboratory cloud (next) University of Massachusetts Amherst - Department of Computer Science 18

Case Study 1 – Doom 3 Game Server ! Multiplayer Doom 3 game server ! Introduced controlled interference as before ! Measured map load times and server latency ! Network sharing configuration via tc : • Idle: No bandwidth usage by resource-hog VM • Off (default): No rate-limiting, network free-for-all • Shared: 50% (min) to 100% (max) of bandwidth per VM • Dedicated: 50% (max) of bandwidth per VM University of Massachusetts Amherst - Department of Computer Science 19

Game Server Map Load 5000 Average Server Load Time (ms) 4000 3000 2000 1000 0 Idle Disk CPU Disk + CPU Collocated VM Activity • Interference produces up to 50% degradation University of Massachusetts Amherst - Department of Computer Science 20

Game Server Latency Avg. Latency Std. Deviation Configuration Timeouts (ms) (jitter) No interference 8.1 10.2 0% tc off (free-for-all) N/A N/A 100% tc , sharing b/w 33.9 16.9 2% 23.6 29.6 7% tc , dedicated b/w ! Server crippled without bandwidth controls ( tc off) ! Dedicated vs shared bandwidth: • Dedicated: lower latency, higher jitter • Sharing: higher latency, lower jitter University of Massachusetts Amherst - Department of Computer Science 21

Case Study 2 – Darwin Streaming Server ! Streaming video to multiple clients ! Introduced controlled interference as before ! Measured sustained streaming bandwidth and stream jitter (latency variation) ! Varied tc settings and number of clients • Max video stream rate of 1 Mbps per client University of Massachusetts Amherst - Department of Computer Science 22

Streaming Server Bandwidth 1000 4 streams average bitrate per stream (kbps) 8 streams decreased 800 stream quality 600 400 200 0 idle (fair) off shared dedicated tc sharing type • both tc configurations recovered bandwidth University of Massachusetts Amherst - Department of Computer Science 23

Streaming Server Jitter 16 4 streams 8 streams 14 average stream jitter (ms) 12 10 8 6 4 2 0 idle (fair) off shared dedicated tc sharing type • Jitter improved by shared, but worsened by dedicated University of Massachusetts Amherst - Department of Computer Science 24

Real World Case Studies – Summary ! Real applications show substantial impacts from background interference ! Network is particularly vulnerable without administrative controls ! Proper configuration is important • CPU and network isolation tools fairly well-developed • Disk isolation needs better mechanisms University of Massachusetts Amherst - Department of Computer Science 25

Related Work ! Fair-share schedulers and quality-of-service • Nieh and Lam (SOSP ‘97) for multimedia • Sundaram et al. (ACM MM ‘00) for QoS-aware OS ! Virtualization and hypervisors • Xen, VMware ESX Server ! Improving performance isolation • Gupta et al. (Middleware ‘06) for Xen mechanisms ! We focus on evaluation of existing mechanisms with specific attention to multimedia University of Massachusetts Amherst - Department of Computer Science 26