Performance Test Lab in the Cloud Leslie Segal Testware Associate, - - PowerPoint PPT Presentation

Build Your Own Performance Test Lab in the Cloud

Leslie Segal Testware Associate, Inc. leslie.segal@testwareinc.com

Problem(s) I was trying to Solve

2

Need more VUs Use Vendor Cloud Schedule Test

Can’t run test – Tool version different Times up – reschedule test Can’t run test – need dll for login Times up – reschedule test

Don’t have Tool Use Vendor Cloud Can’t run true test – can’t spoof IPs Times up – reschedule test Production Issue – need to run test now

Crap! I should just build my own Crap! How fast can I build my own ?

Issues with Vendor Cloud Test Lab

• Must schedule in advance
• Limit on number of Load Generators (LG) you

can use (1 LG per 1,000 vu)

• Can’t install anything on cloud machine
• Permissions typically locked down
• Different version of tool software
• Accessing your application under test
• (open your firewall, open cloud firewall)

3

Advantages of Setting up your Own Cloud-Based Performance Test Lab

• Cost (hardware & tool)
• Need to look at all costs to determine if truly cost

effective

• Time (don't have to wait for long approval

processes for HW & SW)

• Control when (right-now), where (US, CAN, EU)

and how many Load Generators

• Easy way to try vendor tools
• Test tool upgrades

4

Why Test from Different Locations?

• Different Users in different locations will have

different experiences

• Those closest to your servers typically have the best

experience

• Users inside and outside the firewall will have

different experiences

• Using a Content Delivery Network (CDN)?
• Need to test that time sensitive data is consistent

throughout

• Are you mobile?
• Huge variety of mobile networks throughout the world

5

Performance Test Architecture

6

Local Area Network

Load Generators Controller Analysis AUT

Tool Components

HP LoadRunner

• Controllers individually

licensed

• Multiple types of Virtual

User protocols

• Have cloud offering (uses

Amazon Cloud)

• Can supplement existing

licenses with cloud licenses

• r use standalone
• Need license server
• Separate component for

scripting

Borland SilkPerformer

• Unlimited controllers
• 3 Levels of Virtual User

Protocols

• Have cloud offering (uses

Amazon Cloud)

• Can supplement existing

licenses with cloud licenses

• r use standalone
• Need license server
• Integrated scripting

component

7

Load Generators

• Number and size depends on several factors
• Number of virtual users (VU) you want to generate
• Type of VU
• Web service, DB, Web, Mobile
• Type of Application
• Thin client, fat client, amount of network traffic
• Distribution of Load and Transactions
• 50% of normal load from West Coast/20% Canada/30% East

Coast

• 10% of users download content from your site
• 90% of users outside the firewall

8

Load Testing on Physical HW vs. VM

• Cloud machines are typically Virtual Machines

Differences Between Physical & VM

• VM is a slice of a physical machine
• VM needs to perform an extra step to ‘talk’ to actual

hardware

• Excessive requests can skew results
• VM can handle about 80% of load of same VU on

physical machine

• Need to actively monitor CPU, Memory, I/O, Network

Utilization of LG while test is running

9

Cloud Cost Components

10 Machine Size Storage

Machine Types

• Options
• On Demand
• Reserved
• Size
• Standard
• High Memory
• High CPU
• High I/O
• Operating Systems
• Linux
• Windows
• Ubuntu
• DB
• Oracle
• SQL Server

11 Can start small and resize any instance

Amazon Cloud Sample Pricing

12

Rackspace Pricing (Windows)

13

Netel Cloud (CAN $)

14

Use Calculators to Estimate

15

Calculator - Amazon

16 Note Large = 7.5GB RAM

Cost for Data Transfer

17 Amazon Rackspace

Throughput limitations

• Rackspace

18

IP Addresses (IPv4)

• Amazon
• Each AWS Cloud Server comes with a non-persistent private IP address

NATed to a public IP address AWS allows 5 Elastic IP addresses per account

• Cost for not used IP addresses - $0.005 per Elastic IP address not associated

with a running instance per hour on a pro rata basis (~$44/yr)

• Rackspace
• Each Rackspace Cloud Server comes with a dedicated and persistent

public IP address – get same IP if shut down and restart instance

• Note that the server must remain in Active state to retain IP and there is a

charge for an Active Server that is powered off

• Can purchase 5 public IP addresses for $2/month/IP

19

Locations (as of Jan 2013)

Amazon Rackspace US East (N. Virginia) US (Dallas) US West (Oregon) US (Chicago) US West (Northern California) UK (but separate account required) EU (Ireland) Asia Pacific (Singapore) Asia Pacific (Tokyo) Asia Pacific (Sydney) South America (Sao Paulo) 20

Addressing Security Issues

• Rackspace
• Isolated networking
• Windows Firewall
• Dedicated Servers
• Private Cloud (software)
• Amazon
• Dedicated servers
• Windows Firewall
• Private Cloud
• VPN Gateway

21

Other Considerations

• Image Backups
• Can be created at any time
• Minimal cost to store images (~$.10/GB/month)
• File Level Backups
• Backup tools
• Can set up separate drive and clone it
• Where are you writing your scripts?
• Do you need to spin up a cloud machine to create scripts or just for

execution?

• Managed service level and/or monitoring agreements for

additional cost

• Software to remind you if you leave instances running

(or auto shut them down)

22

Road Map to Set up your Own Performance Test Lab in 24 Hours or Less

Sign up for Account Launch Controller Server Install Controller Software (and license) Launch Load Generator Server Install LG Software Modify Firewall, User Access & install 3rd party SW Verify Connectively to Controller and Image LG Spawn N copies

• f LG Image

Start Testing 23 Some magic here

Step 1: Start in the Right Place

24

Step 2: Know What you Need

25

Step 3: Pick Good Name and Enough Space

26 Windows O/S needs 30GB space

Step 4: Wait

• Depending on size and number of instances will take between

5-15 minutes before the instances are available for use

• Best to create 1 instance, set it up and then clone it
• Setup includes
• User access (default only Admin)
• Set Admin Password
• Configure Firewall
• Install software
• In addition to testing software will also need browsers (only IE with

no add-ins), add-ins for browsers (java, flash, etc.)

• General use software such as adobe reader, zip file handler, excel,

etc. 27

Step 5: Use (and Stop using)

• You will be charged for usage as soon as the machine starts

spinning up

• You will be charged for any portion of an hour
• If you are not using a machine, remember to stop it
• 23 cents/hr. is nothing until you accidently leave 20 machines

running over the weekend

• Consider Saving results & doing analysis locally
• Can also have separate drive that you can un-mount from cloud

machine, turn off machine and access data from local machine

• Work on scripts locally or use smaller instance in cloud for

script development

• Consider having license server in cloud (on small instance that

is always on)

28