[PPT] - A Primer on Testing Database-Driven Applications Gregory M. PowerPoint Presentation

SLIDE 1

A Primer on Testing Database-Driven Applications

Gregory M. Kapfhammer Department of Computer Science Allegheny College

3rd Biennial Intramural Faculty Conference May 18 - 19, 2004

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 1/23

SLIDE 2

Outline

Introduction and Motivation Testing Challenges Database-Driven Applications A Unified Representation Test Adequacy Criteria Test Suite Execution Test Coverage Monitoring Conclusions and Resources

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 2/23

SLIDE 3

Invocation

Thus spake the master, “Any program, no matter how small, contains bugs.” The novice did not believe the master’s words. “What if the program were so small that it performed a single function?” The master thoughtfully responded, “Such a program would have no meaning. But, if such a

ne existed, the operating system would fail

eventually, producing a bug.”

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 3/23

SLIDE 4

Invocation

But the novice was not satisfied. “What if the

perating system did not fail?”

The master responded, “There is no operating system that does not fail. But if such a one existed, the hardware would fail eventually, producing a bug.” The novice still was not satisfied. “What if the hardware did not fail?”

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 4/23

SLIDE 5

Invocation

The master gave a great sigh. “There is no hardware that does not fail. But if such a one existed, the user would want the program to do something different, and this too is a bug.”

A program without bugs would be an absurdity, a

nonesuch. If there were a program without any bugs then

the world would cease to exist.

Geoffrey James The Zen of Programming (Adaptation)

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 5/23

SLIDE 6

Life of a Program: Compilation P

Source Code

P

Byte Code

Compiler

The programming language compiler produces a representation of a program that can be executed

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 6/23

SLIDE 7

Life of a Program: Execution

P

Virtual Machine

Byte Code Input a 5 print ... exit Final Result: 45 Output

The virtual machine executes the program’s byte codes

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 7/23

SLIDE 8

Testing Challenges

I shall not deny that the construction of these testing programs has been a major intellectual effort: to convince oneself that one has not

verlooked “a relevant state” and to convince
neself that the testing programs generate them all

is no simple matter. The encouraging thing is that (as far as we know!) it could be done. Edsger W. Dijkstra, 1968

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 8/23

SLIDE 9

Where Bugs Live

Machine

P

Virtual File System System Operating Database

Byte Code Execution Environment Input a 5 print ... exit Final Result: 45 Output

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 9/23

SLIDE 10

Motivation

The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems

About 350 airspace boundaries contained in Jeppesen NavData are incorrect, the FAA has warned. The error

ccurred at Jeppesen after a software upgrade when

information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20.

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 10/23

SLIDE 11

More Testing Challenges

Should consider the environment in which software applications execute Must test a program and its interaction with a database Database-driven application’s state space is well-structured, but infinite (Chays et al.) Need to show program does not violate database integrity, where integrity = consistency + validity (Motro) Must locate program and database coupling points that vary in granularity

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 11/23

SLIDE 12

Database-Driven Applications

P m i m j Dl Dk

R R2

1

E F G H A B C D I

R3

J K L

Program P interacts with two relational databases

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 12/23

SLIDE 13

Database Interaction Levels

P

Database Level

1 n

D D

A program can interact with a database at different levels of granularity

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

SLIDE 14

Database Interaction Levels

UserInfo

user_name

4

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

Geoffrey Arnold 32142 41601 45322 56471 Marcus Bittman

Relation Level

P

D1 Dn

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

SLIDE 15

Database Interaction Levels

UserInfo

user_name

4

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

Geoffrey Arnold 32142 41601 45322 56471 Marcus Bittman

Record Level

P

n

D1 D

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

SLIDE 16

Database Interaction Levels

UserInfo

4

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

32142 41601 45322 56471

user_name

Attribute Level

Marcus Bittman Geoffrey Arnold

P

D1 Dn

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

SLIDE 17

Database Interaction Levels

UserInfo

4

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

Geoffrey Arnold 32142 41601 45322 56471

user_name

Attribute Value Level

Marcus Bittman

P

D1

n

D

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

SLIDE 18

Database Interaction Points

select A1, A2, . . . , Aq from r1, r2, . . . , rm where Q delete from r where Q insert into r(A1, A2, . . . , Aq) values(v1, v2, . . . , vq) update r set Al = F(Al) where Q

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 14/23

SLIDE 19

Refined Database-Driven Application

P m i m j

R R2

1

E F G H A B C D

l

D

k

D

where set J = 500 update L < 1000 R3 select

1

* from R R2 from ) select where A < ( avg(G)

I

R3

J K L

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 15/23

SLIDE 20

Data Flow Information

Interaction point: ‘‘UPDATE UserInfo SET acct lock=1 WHERE card number=’’ + card number + ‘‘;’’;

Database Level: define(BankDB) Attribute Level: define(acct_lock) and use(card_number)

Data flow information varies with respect to the granularity of the database interaction

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 16/23

SLIDE 21

Test Adequacy Criteria

all−attribute−value−DUs all−record−DUs all−attribute−DUs all−relation−DUs all−database−DUs

Database interaction association (DIA) involves the def and use of a database entity DIAs can be located in the DICFG with data flow analysis all-database-DUs requires tests to exercise all DIAs for all

f the accessed databases

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 17/23

SLIDE 22

Generating Test Requirements

Database Seeder

Database

(P)

Test Adequacy Criterion

(C)

System Under Test

Test Case Specification Relational Schema Requirements Test

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 18/23

SLIDE 23

Tracking Covered DIGs and DIAs

DB

P

DIGr m i m j DIGs

DIG # 1 DEF USE { ... } { ... } q { ... } { ... } 1 1 2 2 TEST { ... } { ... } COV?

DIG Coverage Table

DIA coverage can be tracked by recording which DIGs within a DICFG were executed during testing

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 19/23

SLIDE 24

Types of Test Suites

T T ∆

1 e

m1 ∆1

e

∆ 0 me ∆e−1

Independent

∆ m T1

1

∆1

ε−1

∆ Tε

ε

m

ε

∆

e

∆

e

m Te ∆e

Partially Independent

T T ∆

1 e

m1 ∆1

e

∆ 0 me ∆e−1

Non-restricted

– p. 20/23

SLIDE 25

Calculating Adequacy

mi mj

mi

DIA <def(e1), use(e1)> <def(e2), use(e2)> COV? <def(e3), use(e3)> <def(e4), use(e4)>

Test Requirements

DIA COV? <def(e5), use(e5)> <def(e6), use(e6)> <def(e7), use(e7)> <def(e8), use(e8)> <def(e9), use(e9)> <def(e10), use(e10)>

Test Requirements

mj

Tf

cov(mi) = 2

4

cov(mj) = 4

6

cov(Tf) = 6

10

– p. 21/23

SLIDE 26

Conclusions

Software testing is hard, especially when interaction with the application’s environment is considered Must test the program’s interaction with the database Many challenges associated with (1) unified program representation, (2) test adequacy criteria, (3) test coverage monitoring, (4) test suite execution Unique family of test adequacy criteria to ensure that test suites detect violations of database validity and completeness

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 22/23

SLIDE 27

Resources

Gregory M. Kapfhammer and Mary Lou Soffa. A Family of Test Adequacy Criteria for Database-Driven Applications. In Proceedings of the ACM SIGSOFT International Conference on the Foundations of Software Engineering, 2003. Gregory M. Kapfhammer. Software Testing. CRC Press Computer Science Handbook. June, 2004. http://cs.allegheny.edu/˜gkapfham/research/diatoms/

Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 23/23