T14 November 18, 2004 1:30 PM A UTOMATED D ATABASE T ESTING WITH NU - - PDF document

BIO PRESENTATION

International Conference On Software Testing Analysis & Review November 15-19, 2004 Anaheim, CA USA

T14

November 18, 2004 1:30 PM

AUTOMATED DATABASE TESTING WITH NUNIT

Alan Corwin Process Builder, Inc.

Alan Corwin

Alan Corwin has been a software developer, trainer, and course designer for nearly thirty

• years. He is president and founder of Process Builder, CIO of OptionBots, and teaches

High Performance Data Integration in the University of Washington’s Advanced Web Development program.

1

Automated Database Testing with NUnit

Planned Test-Driven Development

2

Planned?

Isn’t test-driven development planned?

• Yes. Although planning is often decried as a

waste of time by the XP community, simply writing your tests before you write your code is a form of planning. It’s the next level of planning that few XPers are willing to take – planning to do a series of standard tests for an entire application.

3

Test Goals

Test the Generic Database Requirements Provide a Framework for More Complex Tests Support Rapid Development (Daily Deliverables!) Support Test-Driven Development Allow Tests to be run both by developers and testers. Test engineers and developers should be able to quickly and

easily add test cases as needed.

Eliminate unnecessary communication cycles.

4

Test Design Features

Tests are fully automated. Tests are black box (ignorant of the

underlying data structure)

Tests are infinitely expandable. Tests remain valid even when the

implementation changes.

5

Tools Required

VisualStudio.NET (Enterprise Architect

Edition)

NUnit XML Spy (Optional)

6

The Test Process

• Developer Builds the Test DataSet Classes
• Developer Creates the Test Data File
• The Test Data Files are placed in a common directory.
• Developer constructs and executes the basic test classes.
• Test Engineers develop test cases and add them to the test file.
• Tests are continually added.
• Each time the product is changed or a test case is added, all of the

tests are rerun.

• Developers run all tests when the code changes.
• Test engineers run all tests when new test cases are added.

7

Test Data

Create the Test Data Dataset Create the Test Data Data File

8

What is the Purpose

• f the test DataSet?

To hold the values necessary to robustly test

the Create, Read, Update, and Delete functions.

To provide a strongly typed DataSet to

increase reliability and reduce coding.

To simply the generation and initial

population of the test data file.

9

Why do you need a DataSet?

First, you don’t “need” a DataSet. They are

just a good idea.

DataSets reduce the amount of code required

to construct the tests.

DataSets increase the readability of code. DataSets eliminate many kinds of errors in

the test code itself.

10

Create the Test DataSet

Each DataSet is based on an underlying table, view,

• r stored procedure.

Start the DataSet Wizard by adding a new DataSet

to your test project.

Drag the underlying object to the designer. Switch to the XML view. Modify the primary key field. Duplicate the data fields. Add “Update” to the end of each new field name. Save. Demo Here!

11

What Just Happened?

The format for the test data file was defined. An XSD (XML schema definition file) was just

created.

A Strongly Typed DataSet class was

generated (3000 or more lines of highly reliable code) – requires that Generate DataSet be turned on (it is by default).

12

Generate the Data File

Here’s where XML Spy comes in handy. Open the Schema file you just created in XML Spy Choose the ‘Generate Sample XML File’ command

from the DTD/Schema menu.

When the dialog appears, select your options

including the number of rows to generate, and click OK.

Save the file in your public TestData directory as an

XML file.

Edit the data rows as needed. Demo Here!

13

The Test Data File

XML Format Means

Editable with a wide variety of tools (including

Notepad)

Built to a known standard supported by every

software vendor

Easy to transfer to any other data format Easy to read and understand. .NET-ready

For proper testing, you need both a green bar

data file and a red bar data file.

14

NUnit Basics

What is NUnit? NUnit Limitations Obtaining and Installing the Product Constructing A Simple Test Running Tests

15

What Is NUnit?

A Public Domain Test Harness for .NET

A Set of Classes to Facilitate Test Development

and Execution

A Command Line Interface A GUI to Run the Tests from Windows

A Tool For Developer-Centered Testing Developed as JUnit, Ported to .NET, then

Rewritten in C#

16

NUnit Limitations

You can’t test exception propagation easily.

Hard Code Tests Red Bar Tests

Can’t debug when you run from the UI

(This was a much longer list until version 2.1 which corrected most of the problems.)

Unlike user interfaces, database and business objects should throw rather than handle exceptions.

17

Obtaining and Installing NUnit

http://www.nunit.org Go to the Downloads Page Run the Executable

(That’s all there is to installing the program. Note, however that each project only knows about the test harness when it contains a reference to the NUnit libraries.)

18

Constructing a Simple Test

Adding the Harness Reference Creating a Test Class Adding Test Fixture Decorations Write the Test Logic

19

Adding the Harness Reference

Right-click the References icon for the test

project in the Solution Explorer and select Add Reference.

When the Open dialogue appears, navigate

to the NUnit Installation directory.

Select all of the DLLs and click Open.

(I’m not sure they are all needed, but the documentation does not specify which are needed.)

20

Creating a Test Class

Add a new class to the project. Import the NUnit Framework

Imports NUnit.Framework (VB.NET) using NUnit.Framework; (C#)

Add a TestFixture Decoration to the class

declaration.

<TestFixture()> (VB -- must be on the same line

as the class declaration)

[TestFixture] (C# -- Must be on the declaration

line or the line before)

21

Write the Test Logic

Get the Test Data For Each Test Case

Create the Record Verify the Record Update the Record Verify the Record Delete the Record Verify the Deletion The test logic is often in a different class than the test execution. That is the approach that we use because it makes debugging both the test and the application easier.

22

Running Tests

From the GUI

Green Bar Tests Red Bar Tests

From the Code

23

Process Change Results

As a result of adding NUnit and the basic database

test process demonstrated here, we were able to shrink delivery times significantly while testing our database applications hundreds of times more thoroughly than we could before.

To our credit and embarrassment, our prototypes

are now more robust and bug-free than our finished applications used to be. (And our finished applications were always more robust than the majority of what you see.)

24

Current Deliverables for One Programmer for One Four Hour Work Period

One DataSet Class One Command Helper

Class

Five Stored Procedures One Business Object Class

for Individual Objects

One Business Object Class

for the DataSet

One Test Class for Each

Business Object Class

One Test Data File

containing at least 3 test cases

A Data Entry form for the

individual object class.

A List form for the DataSet

Class

25

Summary

NUnit is free, flexible, and easy to use. XML test data files are easy to create and use. NUnit supports a simple, standard approach to

database testing that addresses the needs of both the test team and the development team.

The result of this approach is an automated test

suite that creates significant gains in quality and productivity.

26

Technical Appendix

Test Logic Set Tests Additional Tests Standardizing Test Classes Resources Automation of Test Automation

27

Test Logic: Get the Data

Three Details Make getting the Data Easy

Our test data is in an XML file. We have a strongly typed DataSet class that matches that

XML file.

Microsoft provides the ReadXML method.

VB Dim myTestData As New MyTestDataSetName() myTestData.ReadXML(fullPathOfTestFile) C# MyTestDataSetName myTestData = New MyTestDataSetName(); myTestData.ReadXML(fullPathOfTestFile);

28

Test Logic: Get Each Test Case

A Strongly Typed DataSet consists of

Strongly Typed DataRows.

VB

For Each rowInstance _ As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _ in _testSet.Tables[0].Rows _testRow = rowInstance ‘Assign to class variable for common use. Next rowInstance

C#

foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow rowInstance in _testSet.Tables[0].Rows) {_testRow = rowInstance;} // assign to a class variable for common use.

These Row classes were generated when the Test Data DataSet was created.

29

Test Logic: The CRUD Tests

The Logic of your CRUD tests will depend on the

structure of your application, but the set of tests will always have to be done within the test data loop.

VB

For Each rowInstance _ As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _ in _testSet.Tables[0].Rows _testRow = rowInstance ‘Assign to class variable for common use. RunCrudTestSuite() Next rowInstance

C#

foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow rowInstance in _testSet.Tables[0].Rows) { _testRow = rowInstance; // assign to a class variable for common use. RunCrudTestSuite(); }

A better design would pass the test row to the CRUD test routine..

30

Our Basic CRUD Tests

private void RunCrudTestSuite() { int recordID = theNewID(); TestInitialValues(recordID); UpdateRecord(recordID); TestUpdateValues(recordID); DeleteRecord(recordID); TestDeletion (recordID); }

The recordID ties the tests to the same database

• bject.

31

theNewID()

protected override int theNewID() { SampleBusinessObject testObject = new SampleBusinessObject(); testObject.Name = _testRow.name; testObject.OptionBotID = _testRow.optionBotID; testObject.Save(); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); return testObject.ID; }

Access to the database is allowed only through the business

• bjects.

A new business object is created. The properies are set from the test data row. The business object is saved. A check is made to see that no exceptions occurred. The id of the new record is returned.

Note the Assertion. This is an NUnit Assertion.

32

TestInitialValues()

protected override void TestInitialValues(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); Assert.AreEqual(testObject.Name, _testRow.name, “Wrong name."); Assert.AreEqual(testObject.OptionBotID, _testRow.optionBotID, “Wrong OptionBot ID!"); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); }

Tests are very simple.

You put a value into the field. You check to make sure that the same value was

recorded and retrieved.

33

UpdateRecord()

protected override void UpdateRecord(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); testObject.Name = _testRow.nameUpdate; testObject.OptionBotID = _testRow.optionBotIDUpdate; testObject.Save(); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); return testObject.ID; }

This is very similar to creating the record

except that we start with an existing object.

Note that we are populating the business

• bject with the update values from the test

data row.

34

TestUpdateValues()

protected override void TestUpdateValues(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); Assert.AreEqual(testObject.Name, _testRow.nameUpdate, “Wrong name."); Assert.AreEqual(testObject.OptionBotID, _testRow.optionBotIDUpdate, “Wrong OptionBot ID!"); }

Update test is almost identical to the initial

values test.

Note that we are testing against the update

values.

35

DeleteRecord()

protected override void DeleteRecord(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); testObject.Delete(); }

DeleteRecord is very simple. The logic for

the deletion is in the business object so our test code simply calls the objects Delete Method.

36

TestDeletion()

protected override void TestDeletion(int idOfObject) { SampleBusinessObject testObject = new SampleBusinessObject(idOfObject); Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred."); Assert.AreEqual(testObject.IsNew, true, “The object was not deleted."); }

Update test is almost identical to the initial

values test.

Note that we are testing against the update

values.

37

The Test Decoration

Note that although we had to add a

TestFixture decoration to the class declaration, none of the individual tests have had any additional decorations. These cannot be run from the NUnit GUI. That is intentional; the CRUD regression tests are designed to be run as a unit.

Add a test in your Test Fixture class to run

the whole set and add the declaration as shown on the next screen.

38

Test Decoration Example

The test decoration allows the NUnit GUI to

recognize a method as a test.

[Test] public void TestBackSpreadParameterSets() { SampleObjectTest _thisTest = new SampleObjectTest(@"H:\TestData\OptionBot\SampleObjects.xml"); _thisTest.RunAllTests(); }

In VB.NET, the correct decoration is ‘<Test()>’.

39

Test Initialization Example

NUnit allows you to specify a set of actions

that are performed when the Test Fixture class is initialized.

40

The Basic Persistent Set Tests

Get the Set Select Subsets Count the Set

41

Additional Tests

Are Driven by Customer Requirements Are added at the specification of the test

engineer.

Are added by the developer. Additional Test Example

42

Standardizing the Test Classes

Test Class Interfaces are used to standardize the language used

for the test methods.

Abstract Test Classes capture the common elements of test

classes. The latest versions of our test classes are currently available at http://www.ProcessBuilder.com for the following languages

• VB.NET
• C#

43

NUnit Resources

NUnit Home Page, http://www.nunit.org NUnit Resources

http://www.testdriven.com/modules/news/

44

Automating Test Automation

Note that this presentation is about how to

automate the execution of tests, not the creation of those tests. The creation of those tests is clearly automatable.

Extracting MetaData Code Generation Alternatives