Unit Tests Are Not Enough Practical Functional Testing in Java - - PowerPoint PPT Presentation

unit tests are not enough
SMART_READER_LITE
LIVE PREVIEW

Unit Tests Are Not Enough Practical Functional Testing in Java - - PowerPoint PPT Presentation

Unit Tests Are Not Enough Practical Functional Testing in Java Martin Klose klose brothers martin.klose@klosebrothers.de @martinklose 1 Agenda Why unit tests are not enough Characteristics of good tests scenarios The vision: an


slide-1
SLIDE 1

Unit Tests Are Not Enough

Practical Functional Testing in Java

Martin Klose klose brothers

martin.klose@klosebrothers.de @martinklose

1

slide-2
SLIDE 2

Agenda

  • Why unit tests are not enough
  • Characteristics of good tests scenarios
  • The vision: an Executable Specification
  • The tools we can use
  • Tool demo
  • Tips for functional testing
  • What can go wrong

2

slide-3
SLIDE 3

A Fairy Tale

  • Little John and the Air Balloons

3

slide-4
SLIDE 4

Customer not satisfied

4

slide-5
SLIDE 5

What’s covered

test scenarios automation layer (glue code) application test tool unit/ integration tests

functional testing, acceptance testing, specification by example

5

slide-6
SLIDE 6

Test scenarios should...

  • speak the customer’s language
  • cover only the important aspects
  • be complete and realistic examples
  • be self explaining
  • NOT be scripts!

6

slide-7
SLIDE 7

Test scenarios should...

  • speak the customer’s language
  • Uses business terms
  • About business functionality not SW design
  • Ideally customer should be able

to extend and change

  • Used for communication and clarification
  • Leading to a shared understanding
  • Spot inconsistencies

7

slide-8
SLIDE 8

Test scenarios should...

  • cover only the important aspects
  • Hide technical and implementation details
  • Only business related aspects
  • Do not cover all edge & corner cases
  • We still have testers
  • We have lots of unit tests, too

8

slide-9
SLIDE 9

Test scenarios should...

  • be complete and realistic example
  • Combinations of parameters

and expected outputs

  • Using real data helps

9

slide-10
SLIDE 10

Test scenarios should...

  • be self explaining
  • Understandable for users

with domain knowledge

  • All & only important information
  • Explaining underlying logic & business rules
  • Make default values explicit

when relevant for business logic

10

slide-11
SLIDE 11

Test scenarios should...

  • NOT be scripts
  • Describe what instead of how
  • Don’t make the user work backward from single

steps to understand what is illustrated

  • Scripts will cost a lot of time in the long term
  • Difficult to understand
  • Reason for test failure is difficult to find
  • Higher maintenance, UI & Workflows change

more often than business rules

11

slide-12
SLIDE 12

The vision: an Executable Specification

  • Frequently validated
  • Reliable information about system functionality
  • Easy to read and understand
  • Well organized and easy accessible
  • Updated when system functionality changes

12

slide-13
SLIDE 13

Tools for functional testing should allow...

  • to formulate test scenarios in the

appropriate abstraction/language

  • to connect to the system under test at

any layer or API

  • to use the VCS of your choice
  • to be easily integrated into your CI/

Build

13

slide-14
SLIDE 14

Tools to consider

  • Keyword- / table-centric frameworks
  • FitNesse, Robot Framework, Twist
  • Behaviour-Driven-Development tools
  • Cucumber, JBehave, EasyB
  • Free text tools
  • Concordion

14

slide-15
SLIDE 15

Demo

  • Source code is on github:
  • https://github.com/mklose/goto2011aar

15

slide-16
SLIDE 16

Hide technical details

  • Use the testing tool's set up / tear down /

abstraction mechanisms sparingly

  • Write dedicated fixture code which does

the complicated set up

16

slide-17
SLIDE 17

Choose the appropriate "point of attac"

Business Facade UI

AppModel View

Persistence

Credit Rules customer type monthly turnover

  • verdraft?

Private 2500 7500 Private 1200 5000 Private 99 Business 25000 25000 Business 50000 25000 expected 50000 actual

backdoor

Domain

Customer Account CreditRules 17

slide-18
SLIDE 18

Rules for Automation through the User Interface

  • Rule 1: Never automate through the UI
  • UI changes more often => Brittle tests
  • UI automation is technically complex
  • UI automation is slow

18

slide-19
SLIDE 19

Rules for Automation through the User Interface

  • Rule 2: Only automate through the UI

in case of emergency

  • Sometimes necessary to build customer

trust in test automation

  • Abstract the UI away
  • Be aware of what you're missing out on

19

slide-20
SLIDE 20

Decouple from systems not under your control

  • You cannot reliably include

external (test) systems in your regression testing

  • Use an additional set of tests for

checking real collaboration with external systems

  • Usually started by hand

20

slide-21
SLIDE 21

Organization of Tests

  • Goals
  • I want to see the state of current features at a

single glance

  • I want to organize all test scenarios in a

consistent manner

  • How to reach both goals
  • Organize test scenarios for current work by

stories/features

  • Reorganize regression tests by functional area

21

slide-22
SLIDE 22

What can go wrong ...

  • No close collaboration between

developers, testers & domain experts

  • Abuse of acceptance tests as

replacement for thorough unit testing

  • Neglecting clean up, refactoring and

rework of acceptance tests

➡ It's all about communication!

22

slide-23
SLIDE 23

Given a birthday balloon with a face on it When you hang it to the ceiling Then the face is smiling friendly

23

slide-24
SLIDE 24
  • Kudos goes to @johanneslink
  • shirt http://605644.spreadshirt.de/
  • global-day-of-coderetreat Sat, Dec 3, 2011

https://github.com/coreyhaines/coderetreat/wiki/Cities

  • Bielefeld*, Germany too

*and yes it exists!

Nodes

24

slide-25
SLIDE 25

References

  • Gojko Adzic: Specification by Example,How

Successful Teams Deliver the Right Software

  • Cunningham & Mugridge: FIT for

Developing Software: Framework for Integrated Tests

25