Software Testing Per Madsen Aalborg University Denmark Last - - PowerPoint PPT Presentation

I ntroduction to Programming - January 6 2004

1

Chair of Softw are Engineering

Software Testing

Per Madsen Aalborg University Denmark

Last revised 6 January 2004

I ntroduction to Programming - January 6 2004

2

Chair of Softw are Engineering

Where is Aalborg?

I ntroduction to Programming - January 6 2004

3

Chair of Softw are Engineering

Overview of This Lecture

Why do we need software testing? What is software testing? Testing techniques White box (structural testing) Black box (functional testing) Testing strategy Testing tools Testing and Eiffel

I ntroduction to Programming - January 6 2004

4

Chair of Softw are Engineering

Why Software Testing?

Software bugs affect our everyday life. Blue Screen of Death (Windows/ Office). Also bugs in Linux / Open Office / Mozilla etc. Traffic ☺ Software problems cost a lot of money. Ariane 5 Costs resulting from insufficient software testing: 22.2 to 59.5 billion dollars!

NIST, National Institute of Standards and Technology (May 2002)

Lots of other examples.

I ntroduction to Programming - January 6 2004

5

Chair of Softw are Engineering

What Is Software Testing?

Testing is the activity where we try to discover potential bugs in our program. Testing is not: Program proving Program debugging Program testing can be used to show the presence

• f defects, but never their absence!

Dijkstra, Structured Programming (1972)

I ntroduction to Programming - January 6 2004

6

Chair of Softw are Engineering

The Problem of Testing

In a perfect world we would test every possible execution of a program. But: Even very simple programs have a large number of possible executions. Programs with loops have an infinite number of executions.

I ntroduction to Programming - January 6 2004

7

Chair of Softw are Engineering

Testing Terminology

Test cases Test driver Test stub Test oracle

I ntroduction to Programming - January 6 2004

8

Chair of Softw are Engineering

Testing Activities

Testing software typically involves: Selecting what to test. Building test cases. Executing the test cases. Comparing actual results / expected results. Measuring execution characteristics (memory used, time consumed, etc.) Testing is also bug prevention.

I ntroduction to Programming - January 6 2004

9

Chair of Softw are Engineering

Testability

Testability: the ’price’ of testing a program Testability is a design issue Two key aspects of testability Controllable How can we control a program? Observable How can we observe the output of a program?

I ntroduction to Programming - January 6 2004

10

Chair of Softw are Engineering

Test Units

What can be tested? One instruction One feature One class One cluster One program One library We can do testing on all levels.

I ntroduction to Programming - January 6 2004

11

Chair of Softw are Engineering

Levels of Testing

Unit test Test a single unit/ module of a program. Integration test Put the units together and test them as a whole. Big bang or iterative integration. Acceptance test Determine whether the customer can accept the program. Regression test Re-run selected tests after updating the software.

I ntroduction to Programming - January 6 2004

12

Chair of Softw are Engineering

Black Box Testing

Testing “in the large” Also known as functional testing We test a program without using any knowledge about how the program is built. The primary idea is to discover bugs related to the functionality of the unit. Do we get the right output for a given input? Find bugs in interfaces. Find problems with efficiency, disk space etc. Examples: Equivalence testing Boundary testing

I ntroduction to Programming - January 6 2004

13

Chair of Softw are Engineering

Equivalence Partitioning

Equivalence partitioning is a black box technique based on the idea that we can split the input into a number of classes. Members of each class should behave in a similar way. Examples: The input is a interval [ a..b] Choose one sample less than a, one between a and b and one larger than b. The input is a set V Choose a sample in V and one not in V.

I ntroduction to Programming - January 6 2004

14

Chair of Softw are Engineering

Eiffel Example

feature swap_elements (a: ARRAY [INTEGER]; x: INTEGER; y: INTEGER) is

• - Swap the position of two elements x and y in an array a.

require a_not_void: a /= Void local tmp: INTEGER do tmp:= a.item (x) a.put (a.item (y), x) a.put (tmp, y) ensure x_is_old_y: a.item (x) = old a.item (y) y_is_old_x: a.item (y) = old a.item (x) end

I ntroduction to Programming - January 6 2004

15

Chair of Softw are Engineering

Test Cases

Candidates for test cases a is empty / not empty x< y, x= y, y> x x> a.size, y> a.size x= a.size, x= a.size+ 1, x= a.size-1 Observation: Static type checking and contracts can minimize the number of test cases we need. e.g add a precondition require x> 0 and x< = a.size

I ntroduction to Programming - January 6 2004

16

Chair of Softw are Engineering

Break

I ntroduction to Programming - January 6 2004

17

Chair of Softw are Engineering

White Box Testing

Testing “in the small”. Also known as structural testing. We use our knowledge about the internal structure

• f the program when testing the program.

Examples: Path testing Focuses on the control flow. Data flow Focuses on critical data values.

I ntroduction to Programming - January 6 2004

18

Chair of Softw are Engineering

Eiffel Example

feature testing (a: INTEGER; b: INTEGER) is

• - Use a loop and a branch to illustrate how to make a flow chart.

local c: INTEGER do from c:=0 until c=a loop if b>0 then c:=c + b else c:=c + 1 end end end

I ntroduction to Programming - January 6 2004

19

Chair of Softw are Engineering

Path Testing Example

• Path Testing:

Construct the flow chart Make a test case for every independent path

• The flow chart consists of:

Entry / exit points Branches Statements

I ntroduction to Programming - January 6 2004

20

Chair of Softw are Engineering

Number of Independent Paths

• Independent paths

1 1,2,3,5,1 1,2,4,5,1

• The number of independent paths is

equal to the number of regions in the graph.

• Cyclomatic complexity = number of

edges – number of nodes + 2

• CC = the number of test cases we

need to cover all edges.

• CC for the example 6 – 5 + 2 = 3

I ntroduction to Programming - January 6 2004

21

Chair of Softw are Engineering

White Box vs. Black Box

The typical use of white box testing: Used for unit testing. Used by the programmer. Not suitable for whole programs. The typical use of black box testing Can be used by the programmer and by others. Used in integration and system testing.

I ntroduction to Programming - January 6 2004

22

Chair of Softw are Engineering

Testing Strategy

How do we plan and structure the testing of a large program? Who is testing? Developers / special testing teams. It is hard to test your own code. What test levels do we need? Unit, integration, system, acceptance test. How do we do it in practice? Manual testing vs. testing tools.

I ntroduction to Programming - January 6 2004

23

Chair of Softw are Engineering

Testing and OO Technology

Observations Polymorphism / dynamic binding increases the number of test cases we need. A class is a natural unit for testing. The sequence of feature calls is important. Information hiding can make testing harder. It is hard to observe non-exported attributes. Deferred classes Cannot be testing directly. Inheritance Should inherited features be re-tested?

I ntroduction to Programming - January 6 2004

24

Chair of Softw are Engineering

Testing Tools

Automatic unit testing tools JUnit for Java Gobo EiffelTest for Eiffel Available for all major languages Capture and replay tools Testing coverage analysis tools Regression test tools Save the result of selected feature calls and compare the results with new versions of the features.

I ntroduction to Programming - January 6 2004

25

Chair of Softw are Engineering

Extreme Programming

Extreme Programming (XP) is a light weighted software development process. Test Driven Development Write test cases before you write the code. Let the test cases drive your design. Testability is the most important design issue. Assume simplicity Start out with the simplest thing that can work. Embrace change The automated testing approach allows you to make changes.

I ntroduction to Programming - January 6 2004

26

Chair of Softw are Engineering

‘Automated’ Unit Testing

Write test cases as features in the same language that you program. Put assertions in the test cases. Let a tool manage the test cases. Run and re-run the test cases. Keep track of bugs. Re-run the test cases after each update. Automated regression test. This approach automates the execution and the book-keeping of test cases but in the end the developer still has to write the test cases.

I ntroduction to Programming - January 6 2004

27

Chair of Softw are Engineering

Eiffel and Software Testing

Design by Contract can be a great help: Design by Contract is a bug prevention method. Contracts provide a good starting point for black box testing. Post conditions work as test oracles. Checked contracts speed up debugging. Eiffel programmers are familiar with writing assertions.

I ntroduction to Programming - January 6 2004

28

Chair of Softw are Engineering

References

Programmers Love Writing Tests http: / / junit.sourceforge.net/ doc/ testinfected/ tes ting.htm Gobo EiffelTest http: / / www.gobosoft.com/ eiffel/ gobo/ getest/ JUnit http: / / www.junit.org

I ntroduction to Programming - January 6 2004

29

Chair of Softw are Engineering

Summary

Testing is hard and time consuming. Testing cannot show the absence of bugs. Testability is a design issue. Various techniques exists Black box test White box test But there is no easy answer! We still need to think about testing, even if it is a hard task!