SLIDE 1
@burythehammer
MATT LONG
@burythehammer
TESTING PROGRAMMABLE INFRASTRUCTURE (WITH RUBY) @burythehammer - - PowerPoint PPT Presentation
TESTING PROGRAMMABLE INFRASTRUCTURE (WITH RUBY) @burythehammer - - PowerPoint PPT Presentation
@burythehammer MATT LONG TESTING PROGRAMMABLE INFRASTRUCTURE (WITH RUBY) @burythehammer PROGRAMMABLE INFRASTRUCTURE IS GREAT, BUT WE'RE MISSING SOMETHING. TESTING. @burythehammer HELLO, I'M MATT I'M A TESTER @burythehammer
SLIDE 2
SLIDE 3
@burythehammer
I'M A TESTER
HELLO, I'M MATT
@burythehammer
SLIDE 4
@burythehammer
I WORK HERE ↑
SLIDE 5
@burythehammer
I AM NOT A SYSADMIN
SLIDE 6
@burythehammer
WHAT IS PROGRAMMABLE INFRASTRUCTURE?
SLIDE 7
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
THE APPLICATION OF METHODS AND TOOLING FROM SOFTWARE DEVELOPMENT TO THE MANAGEMENT OF IT INFRASTRUCTURE PROGRAMMABLE INFRASTRUCTURE IS..
THE INTERNET
SLIDE 8
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
WHAT DO WE MEAN BY THIS?
▸ Automated provisioning & configuration ▸ Configuration as code ▸ Version / source controlled
SLIDE 9
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
TOOLING EXAMPLES
SLIDE 10
@burythehammer
PROGRAMMABLE INFRASTRUCTURE IS AWESOME!
Credit: Vault Boy, Bethesda Softworks
SLIDE 11
@burythehammer
BUT IT CAN GET COMPLEX
SLIDE 12
@burythehammer
TESTING IS USED TO MITIGATE COMPLEXITY & RISK
SLIDE 13
@burythehammer
BUT INFRA TESTING IS RARE
Credit: Gunshow, KC Green
SLIDE 14
@burythehammer
TESTING A CLOUD BROKER
AN INFRASTRUCTURE HEAVY PRODUCT
SLIDE 15
@burythehammer
THE PROBLEM
SLIDE 16
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
WE WANT TO MOVE TO THE CLOUD... BUT WE'RE WARY OF LOCK IN
Large organisation
SLIDE 17
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
USERS
CLOUD BROKER
SLIDE 18
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
BENEFITS
▸ Quick, easy provisioning ▸ one team previously took 3 months ▸ Don't need to work at a low level ▸ Templates for common dev environments ▸ Built in best practice: monitoring, security ▸ Track spending
SLIDE 19
@burythehammer
THIS IS A REALLY COMPLICATED APPLICATION
SLIDE 20
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
SLIDE 21
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
WORKFLOW
▸ Log into Web UI ▸ Fill in information about environment ▸ Broker creates and bootstraps resources ▸ SSH into resources
SLIDE 22
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
WEB TESTING
▸ Log into Web UI ▸ Fill in information about environment
SLIDE 23
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
???
▸ Broker creates and bootstraps resources ▸ SSH into resources
SLIDE 24
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
INFRASTRUCTURE TESTING
▸ Broker creates and bootstraps resources ▸ SSH into resources
SLIDE 25
@burythehammer
HOW DO YOU TEST INFRASTRUCTURE?
SLIDE 26
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
WHAT TO TEST?
Deployment scripts unit tests? Does the VPN server work? Can instances access one another? Are services running? Linting?
SLIDE 27
@burythehammer
THIS SEEMS FAMILIAR..
SLIDE 28
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
Does the VPN box work? Can I SSH into a server? Do our deployment scripts work? Linting? Are services running?
ANOTHER TESTING PYRAMID?
credit: Ubuntu dev quality guide https://developer.ubuntu.com/en/phone/platform/quality/Can instances access one another? expensive, slow cheap, fast
SLIDE 29
@burythehammer
TOOLING
SLIDE 30
@burythehammer
UNIT TESTING
▸ Bash scripts ▸ Ansible scripts ▸ Terraform scripts
SLIDE 31
@burythehammer
LINTING
▸ Quick sanity check ▸ Available in all tools ▸ Run in CI before committing
SLIDE 32
@burythehammer
UNIT TESTING IS HARD
▸ Cultural issues ▸ Technical issues ▸ Return on investment
SLIDE 33
@burythehammer
INTEGRATION TESTING
▸ Packages installed ▸ Services running ▸ Ports listening
SLIDE 34
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
▸ Ruby / RSpec based ▸ Great community ▸ Very readable ▸ Very quick! ▸ Can SSH into instances
SERVERSPEC
SLIDE 35
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
SERVERSPEC EXAMPLE
describe package('jenkins') do it { should be_installed } end describe service('jenkins') do it { should be_enabled } it { should be_running } end describe port(8080) do it { should be_listening } end
SLIDE 36
@burythehammer
ACCEPTANCE TESTING
▸ SSHing into machines ▸ Using applications ▸ e.g. samba, openvpn
SLIDE 37
@burythehammer
TESTING PROGRAMMABLE INFRASTRUCTURE
▸ Executable specifications ▸ Both test & business can read it ▸ Available in Ruby, Java, C#... etc ▸ Great reporting ▸ Good CI integration
CUCUMBER
SLIDE 38
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
Scenario: Searching for Wikipedia Given I am on the website "www.google.com" When I search for "Wikipedia" Then the first link should be "www.wikipedia.org"
CUCUMBER EXAMPLE
SLIDE 39
@burythehammer
TESTING PROGRAMMABLE INFRASTRUCTURE
RUBY
▸ Fantastic testing community ▸ More suitable for SSHing into boxes ▸ "Win RM" gem ▸ Ops already familiar with it ▸ Reduces tech stack
SLIDE 40
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
TOOLS WE DIDN'T USE
▸ Bats ▸ ShUnit2 ▸ Goss ▸ Inspec / TestInfra ▸ AWS Spec ▸ Test Kitchen
SLIDE 41
@burythehammer
OUR SOLUTION
SLIDE 42
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
USERS
CLOUD BROKER
SLIDE 43
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
USERS
WEB TEST FRAMEWORK
SLIDE 44
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
USERS
WEB TESTS
https://github.com/opencredo/test-automation-quickstart
SLIDE 45
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
USERS
INFRASTRUCTURE TEST FRAMEWORK
SLIDE 46
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
INFRASTRUCTURE TESTS
Serverspec
SLIDE 47
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
INFRASTRUCTURE TESTING STACK
/ Serverspec Linting tools
SLIDE 48
@burythehammer
TESTING PROGRAMMABLE INFRASTRUCTURE
SERVERSPEC SMOKE TESTS
▸ Run before everything else ▸ Really quick ▸ Catches obvious errors ▸ Not complex tasks
SLIDE 49
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
Background: Given environment has been created And the following user details: | user_alias | username | public_key | | userA | testuser | test | Scenario: IPA - Login via SSH Key authentication succeeds Given user "userA" is authorised to access environment vms When user "userA" starts ssh session in host "env" Then I should be able to echo "hello world"
CUCUMBER FOR ACCEPTANCE TESTING
SLIDE 50
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
Background: Given environment has been created And the following user details: | user_alias | username | public_key | | userA | testuser | test | Scenario: IPA - Login via SSH Key authentication succeeds Given user "userA" is authorised to access environment vms When user "userA" starts ssh session in host "env" Then I should be able to echo "hello world"
CUCUMBER FOR ACCEPTANCE TESTING
Cloud broker APIs
SLIDE 51
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
Background: Given environment has been created And the following user details: | user_alias | username | public_key | | userA | testuser | test | Scenario: IPA - Login via SSH Key authentication succeeds Given user "userA" is authorised to access environment vms When user "userA" starts ssh session in host "env" Then I should be able to echo "hello world"
CUCUMBER FOR ACCEPTANCE TESTING
Standard Ruby
SLIDE 52
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
Background: Given environment has been created And the following user details: | user_alias | username | public_key | | userA | testuser | test | Scenario: IPA - Login via SSH Key authentication succeeds Given user "userA" is authorised to access environment vms When user "userA" starts ssh session in host "env" Then I should be able to echo "hello world"
CUCUMBER FOR ACCEPTANCE TESTING
RSpec assertions
SLIDE 53
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
UNDER THE CUCUMBER, PLAIN RUBY
Then(/^I should be able to echo "([^"]*)"$/) do |text| cmd = "echo #{text}"
- utput = @session.exec!(cmd)
close_ssh(@session) expect(output.to_s.strip).to eql(text) end
SLIDE 54
@burythehammer
LESSONS LEARNED
THE GOOD, THE BAD, AND THE UGLY
SLIDE 55
@burythehammer
TESTING PROGRAMMABLE INFRASTRUCTURE
THE GOOD
▸ Specialised tests for each layer ▸ Quick, expressive ServerSpec
tests
▸ Power of a full programming
language for user tests
▸ ... it's also totally doable!
Credit: Overwatch, Blizzard Entertainment
SLIDE 56
@burythehammer
TESTING PROGRAMMABLE INFRASTRUCTURE
THE BAD
▸ Over reliance on
acceptance tests
▸ Context switching with
two suites
▸ Out of my comfort zone
Credit: Futurama, Fox
SLIDE 57
@burythehammer
TESTING PROGRAMMABLE INFRASTRUCTURE
THE UGLY
▸ Starting infrastructure is SLOW. ▸ It's expensive...
SLIDE 58
@burythehammer
IT WAS WORTH IT
DESPITE ALL THAT
SLIDE 59
@burythehammer
CONCLUSIONS
SLIDE 60
@burythehammer
TESTING IS IMPORTANT
BUT OFTEN IGNORED
SLIDE 61
@burythehammer
TESTERS AND OPS SHOULD WORK TOGETHER
WE NEED TO GET OUT OF OUR COMFORT ZONES
SLIDE 62
@burythehammer
TOOLS EXIST
BUT BE PREPARED TO HACK
SLIDE 63
@burythehammer
FINALLY...
SLIDE 64
TESTING PROGRAMMABLE INFRASTRUCTURE
@burythehammer
THE APPLICATION OF METHODS AND TOOLING FROM SOFTWARE DEVELOPMENT TO MANAGEMENT OF IT INFRASTRUCTURE PROGRAMMABLE INFRASTRUCTURE IS..
SLIDE 65
@burythehammer
TESTING IS A SOFTWARE DEVELOPMENT METHOD
WE SHOULD APPLY IT TO INFRASTRUCTURE
SLIDE 66
@burythehammer
THANKS! QUESTIONS?
matt.long@opencredo.com <-- sponsors booth on 5th floor! @burythehammer