SLIDE 1
software evolution & architecture lab
University of Zurich, Switzerland
An Approach and Case Study of Cloud Instance Type Selection for Multi-Tier Web Applications
Christian Davatz, Christian Inzinger, Joel Scheuner, Philipp Leitner
An Approach and Case Study of Cloud Instance Type Selection for - - PowerPoint PPT Presentation
An Approach and Case Study of Cloud Instance Type Selection for - - PowerPoint PPT Presentation
An Approach and Case Study of Cloud Instance Type Selection for Multi-Tier Web Applications Christian Davatz, Christian Inzinger, Joel Scheuner, Philipp Leitner University of Zurich, Switzerland software evolution & architecture lab
SLIDE 2
SLIDE 3
Common Questions
What cloud provider should I choose? Should I go for many small or few large instances? General-purpose or *-optimized? Pay for better IOPS or not? ……………
➡ Need for Benchmarking
SLIDE 4
Existing Benchmarking Work
SLIDE 5
Existing Benchmarking Work
SLIDE 6
Existing Benchmarking Work
SLIDE 7
Basic Approach to Benchmarking Clouds
Benchmark Manager Provider API
results provision
Instance
start benchmark destroy
Philipp Leitner and Jürgen Cito. 2016. Patterns in the Chaos — A Study of Performance Variation and Predictability in Public IaaS Clouds. ACM Trans. Internet Technol. 16, 3, Article 15 (April 2016), 23 pages. DOI: http://dx.doi.org/10.1145/2885497
Used for instance in:
Joel Scheuner, Jürgen Cito, Philipp Leitner, Harald C. Gall (2015). Cloud WorkBench: Benchmarking IaaS Providers Based on Infrastructure-as-Code. In Proceedings of the 24th International Conference on World Wide Web, pp. 239–242, New York, NY, USA.
SLIDE 8
Benchmark Manager Provisioner Provider API
Cloud Provider under Test
System under Test Benchmark Driver
request response results provision provision acquire start-up
For Multi-Tier (Application) Benchmarks
SLIDE 9
A Concrete Instantiation
CWB Server Chef Server Vagrant
Scheduler
Provider API
IaaS Provider
JMeter Master SUT AcmeAir Webapplication MongoDB request DRIVER response Test Plan results
JMeter Slave
provision provision provision provision acquire start-up
CWB Client
Chef Client Chef Client
JMeter Slave
Chef Client Chef Client
JMeter Slave
Chef Client Chef Client
SLIDE 10
AcmeAir
OSS “Microservice”-Based App: https://github.com/acmeair/acmeair
Two-Tier App
SLIDE 11
CBW
- J. Scheuner, P. Leitner, J. Cito and H.C. Gall: Cloud Work Bench - Infrastructure-as-
Code Based Cloud Benchmarking 2014 IEEE 6th International Conference on Cloud Computing Technology and Science, Singapore, 2014, pp. 246-253. doi: 10.1109/ CloudCom.2014.98
Code: https://github.com/ sealuzh/cloud- workbench Demo: https:// www.youtube.com/ watch?v=0yGFGvHvobk
SLIDE 12
- J. Scheuner, P. Leitner, J. Cito and H.C. Gall: Cloud Work Bench - Infrastructure-as-
Code Based Cloud Benchmarking 2014 IEEE 6th International Conference on Cloud Computing Technology and Science, Singapore, 2014, pp. 246-253. doi: 10.1109/ CloudCom.2014.98
CBW
SLIDE 13
Research Questions
RQ1: What sustained performance, measured in throughput of successful
requests per second, can we achieve with each configuration?
RQ2: Can we observe statistically significantly different performance for
each configuration?
RQ3: Which configuration is the most cost-effective way to host AcmeAir
for the defined workload?
SLIDE 14
Used Configs
SLIDE 15
Used Configs
SLIDE 16
Used Metric
“Sustainable Throughput”
SLIDE 17
Results
RQ1 + RQ2
SLIDE 18
Results
RQ3
Metric: Mio. Requests per $
SLIDE 19
Lessons Learned
Importance of Benchmarking
Least cost-effective instance type only about 67% of perf / $ of best configuration
No clear “cheap” cloud provider
Comparable offerings from different providers are similarly cost- effective
No easy rules of thumb
Compute-optimized instances may be better for our workload, but results vary
SLIDE 20
Summary
CWB Server Chef Server Vagrant
Scheduler
Provider API
IaaS Provider
JMeter Master SUT AcmeAir Webapplication MongoDB request DRIVER response Test Plan results
JMeter Slave
provision provision provision provision acquire start-up
CWB Client
Chef Client Chef Client
JMeter Slave
Chef Client Chef Client
JMeter Slave
Chef Client Chef Client
SLIDE 21
Summary
CWB Server Chef Server Vagrant
Scheduler
Provider API
IaaS Provider
JMeter Master SUT AcmeAir Webapplication MongoDB request DRIVER response Test Plan results
JMeter Slaveprovision provision provision provision acquire start-up
CWB Client Chef Client Chef Client JMeter Slave Chef Client Chef Client JMeter Slave Chef Client Chef Client SLIDE 22
Summary
CWB Server Chef Server Vagrant
Scheduler
Provider API
IaaS Provider
JMeter Master SUT AcmeAir Webapplication MongoDB request DRIVER response Test Plan results
JMeter Slaveprovision provision provision provision acquire start-up
CWB Client Chef Client Chef Client JMeter Slave Chef Client Chef Client JMeter Slave Chef Client Chef Client SLIDE 23