"Specification-by-Example: A Cucumber Implementation" - - PDF document
AT10 Concurrent Session 11/8/2012 3:45 PM "Specification-by-Example: A Cucumber Implementation" Presented by: Mary Thorn Deutsche Bank Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888 268 8770 904
Concurrent Session 11/8/2012 3:45 PM
Presented by: Mary Thorn Deutsche Bank
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Mary Thorn Deutsche Bank VP of QA at Deutsche Bank Global Technologies in North Carolina, Mary Thorn has a broad testing background that spans automation, data warehouses, and web-based systems in a wide variety of technologies and testing techniques. During her more than fifteen years of experience in healthcare, HR, and SaaS-based products, Mary has held manager and contributor level positions in software development organizations. She has a strong interest in agile testing methodologies and direct experience leading agile teams through Scrum adoption and beyond. Mary is a Certified ScrumMaster and an active participant in Triangle Institute of Software Quality Assurance. .
1
Mary Thorn
QA Manager at Deutsche Bank. Over 15 years of QA experience working in the following
areas: Healthcare HR Agriculture Financial and SaaS areas: Healthcare, HR, Agriculture, Financial and SaaS- based products.
Have a broad testing background that spans
automation, data warehouses to web-based systems in a wide variety of technologies and testing techniques.
Heavy interest in Agile Testing Methodologies and direct Heavy interest in Agile Testing Methodologies and direct
experience leading Agile teams in the SCRUM Software Development Lifecycle (SDLC).
2
Feature Files
Goal: Presenter would like to have a successful presentation Acceptance Criteria:
presentation
3
The Challenge:
Today, the most difficult problem in software
development is knowing what to build, not how to build it.
Knowing what to build, requires knowing why it
is being built. In other words, understanding the goal. The Problem:
Stories typically concentrate on the what and
the how, not the why. Goals and examples are usually missing.
Stories are usually a list of imperative
requirements (acceptance criteria).
Imperative requirements are usually interpreted Imperative requirements are usually interpreted
in subtly different ways by each person. If goals and examples are absent, misinterpretation is much more likely.
4
Example:
Requirement: Draw a star with 10 points.
OR Solution: Specification by Example People understand requirements best using
People understand requirements best using
concrete examples.
Examples clear up ambiguity and
misunderstandings.
Examples expose missing requirements. Examples force everyone to think harder about
a problem.
5
Specification by Example This is the basis of acceptance testing and
This is the basis of acceptance testing and
Behavior Driven Design (BDD).
It is critical that everyone participates in creating
the examples so that the understanding is shared.
Ultimately, acceptance testing is less about the
example itself and more about the conversation required to create the example.
6
Acceptance Tests are Formal examples that define requirements
Formal examples that define requirements. Executable. Self-explanatory and readable by all team
members.
Tests of business rules Tests of business rules. A living specification of system behavior.
7
Acceptance Tests do NOT Test pieces of code in isolation They are not a
Test pieces of code in isolation. They are not a
substitute for unit and integration tests.
Verify the entire system. Verify all permutations of a piece of functionality. Use mock objects except to remove a Use mock objects except to remove a
dependency on an external 3rd party. Benefits:
Higher quality b/c everyone shares the same
understanding of the requirements and builds it right the first time.
Tests become the system's regression suite. Create an objective verification of “done-ness” for a
Create transparency into progress on a story.
p y p g y
Manual testers are part of the automation process. Allows for more exploratory testing b/c happy path is
automated
8
Acceptance Criteria: Transactions are rounded to the nearest cent. Original Value Rounded Value $0.021 $0 025 $0.02 $ $0.025 $0.029 $0.02 $0.02 Why was there confusion? The goal was missing... Goal: Create a currency conversion system. The Concrete Example clarified what was meant by the acceptance criteria by the acceptance criteria. Refined Acceptance Criteria: Transactions are always rounded down to the nearest cent.
9
This simple example is a REAL EXAMPLE in a
system that was well-tested and approved by all y pp y parties.
Attacker was able to steal > $15,000 starting
with a single cent using this process...
$0.01 → 0.0051 € rounded to 0.01 € 0.01 € → $0.0196 rounded to $0.02 A seemingly minor ambiguity in the
requirements was very expensive.
Concrete examples drive understanding and
consensus.
To create good representative examples,
everyone must understand the goal of the story.
For technical people to properly understand the
goal, they must understand the business domain domain.
10
describe how software should behave in plain
domain-specific language and serves as documentation of requirements, automated regression tests and development-aid - all rolled into one format.
y, , ,
has been translated to over 40 spoken languages.
11
Components:
Features) Features)
Command line – Rake TAGS - @abc Cucumber Framework - Tags Feature File - Sentence Step Definition File – Matching Sentence Terminal Browser Emulator Selinium/Watir – Terminal Result Output- Browser Emulator Selinium/Watir Execute Command
12
Process:
p feature file scenarios format in the story. Worst case, they at least start the Scenario:
scenarios and asks questions. Team can add/remove scenarios during grooming or team updates feature file after team updates feature file after.
getting the feature file 90% complete before the developer starts coding. Process:
The developers code the scenarios in the feature file in a TDD manner.
after development if there are new findings.
done by running the acceptance tests.
13
Two Components:
Scenarios.
“glue” code that connects the tests to the application. Feature File Overview: Written in plain text
Written in plain text Define the test suite and contain the actual tests
which are called Scenarios
Written collaboratively by everyone on the team
(PO, QA, and Dev)
QA can run these manually.
14
Feature: Suite is defined by keyword “Feature:” followed
Suite is defined by keyword Feature: followed
by a description. This states the goal for the Feature File. When the Feature File is complete, a tag is included at this level so that all of the test Scenarios contained within the file can be run from a single command. g Feature: This tests the currency conversion system Background: “Background:” is run before each test so that
you don’t have to repeat steps. Equal to a setup step. Background: Given I login with the test account And I navigate to the Transaction Screen And I navigate to the Transaction Screen Then I should see the Transaction logo
15
Scenario: Individual tests are defined by “Scenario:”
Individual tests are defined by Scenario:
followed by a unique name. Scenarios are concrete examples of how we want the software to behave. The first line states the goal of the given Scenario, while every line thereafter lists the steps needed to complete the test. p p
Scenario: Verify Transactions are rounded
to the nearest cent. Scenario Outline:
“Scenario Outline:” is used for data driven tests
Scenario Outline: Verify Transactions are rounded to the nearest cent. Given I am on the order screen When I make a <transaction> of values Then the <values> are rounded Examples: |t ti | l | |transaction|values| |.021|.02| |.025|.02| |.029|.02|
16
Keywords:
Tests have reserved words that begin each line: Given, When, Then, And, But.
Scenario Outline: Verify Transactions are rounded to the nearest cent. Given I am on the order screen When I make a transaction of 9.025 And save the transaction Then the value should equal 9.02
Tags:
many tags as you like. Just separate them with spaces.
@tag @smoke_test @component
17
@Cucumberpresentation Feature: User would like to have a successful presentation Tests the following acceptance criteria:
2 Verify the attendees are happy with the presentation
@Attendeeawake Scenario: Verify the attendees are awake Given that I am an attendee When I have drunk caffeine Then I will be able to listen to the presenter @Attendeeishappy S i V if Att d h Scenario: Verify Attendees are happy Given I am the presenter When I talk slow And my content is interesting Then the attendees will be happy
agile teams.
example and agile acceptance testing. p g p g
is easily readable and understandable to business users, developers and testers.
goal for development, acceptance criteria and functional regression checks for future changes.
authoritative source of information on system functionality that is always up-to-date.
as, frees them up from testing the “happy path” to be able to real exploratory testing.
18