A Family of Test Adequacy Criteria for Database-Driven Applications - - PowerPoint PPT Presentation

▶

Oct 28, 2023 146 likes •355 views

A Family of Test Adequacy Criteria for Database-Driven Applications Gregory M. Kapfhammer Department of Computer Science Mary Lou Soffa University of Pittsburgh Department of Computer Science University of Pittsburgh Department of Computer

SLIDE 1

A Family of Test Adequacy Criteria for Database-Driven Applications

Gregory M. Kapfhammer Department of Computer Science University of Pittsburgh Department of Computer Science Allegheny College Mary Lou Soffa Department of Computer Science University of Pittsburgh

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 1/15

SLIDE 2

Motivation

The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems

About 350 airspace boundaries contained in Jeppesen NavData are incorrect, the FAA has warned. The error

ccurred at Jeppesen after a software upgrade when

information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20.

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 2/15

SLIDE 3

Testing Challenges

Should consider the environment in which software applications execute Must test a program and its interaction with a database Database-driven application’s state space is well-structured, but infinite (Chays et al.) Need to show program does not violate database integrity, where integrity = consistency + validity (Motro) Must locate program and database coupling points that vary in granularity

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 3/15

SLIDE 4

Database-Driven Applications

P

Database Level

1 n

D D

A program can interact with a database at different levels of granularity

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

SLIDE 5

Database-Driven Applications

P

D D

1 n

UserInfo

user_name

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

Geoffrey Arnold 32142 41601 45322 56471 Marcus Bittman

Relation Level

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

SLIDE 6

Database-Driven Applications

P

D D

1 n

UserInfo

user_name

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

Geoffrey Arnold 32142 41601 45322 56471 Marcus Bittman

Record Level

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

SLIDE 7

Database-Driven Applications

P

D D

1 n

UserInfo

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

32142 41601 45322 56471

user_name

Attribute Level

Marcus Bittman Geoffrey Arnold Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

SLIDE 8

Database-Driven Applications

P

D D

1 n

UserInfo

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

Geoffrey Arnold 32142 41601 45322 56471

user_name

Attribute Value Level

Marcus Bittman Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

SLIDE 9

Test Adequacy Criteria

P violates a database Di’s validity when it:

(1-v) inserts entities into Di that do not reflect real

world P violates a database Di’s completeness when it:

(1-c) deletes entities from Di that still reflect real

world In order to verify (1-v) and (1-c), T must cause P to define and then use entities within D1, . . . , Dn!

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 5/15

SLIDE 10

Data Flow Information

Interaction point: ‘‘UPDATE UserInfo SET acct lock=1’’ + ‘‘WHERE card number=’’ + c n + ‘‘;’’;

Database Level: define(BankDB) Attribute Level: define(acct_lock) and use(card_number)

Data fl

w information varies with respect to

the granularity of the database interaction

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 6/15

SLIDE 11

Database Entities

UserInfo

user_name

acct_lock

1 Brian Zorman 2 Robert Roos 3

card_number pin_number

Marcus Bittman Geoffrey Arnold 41601 45322 56471 32142 v r

A (I ) = {

32142

}

1 Geoffrey Arnold

, , . . . , ,

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 7/15

SLIDE 12

The DICFG: A Unified Representation

entry lockAccount

temp1 = parameter0:c_n temp2 = LocalDatabaseEntity0:Bank temp3 = LocalDatabaseEntity1:acct_lock temp4 = LocalDatabaseEntity2:card_number

“Database-enhanced” CFG for lockAccount Define temporaries to represent the program’s interaction at the levels of database and attribute

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 8/15

SLIDE 13

The DICFG: A Unified Representation

exit G G G G

r r2 r 2 r 1

entry entry exit lockAccount update_lock = m_connect.createStatement() if( result_lock == 1) completed = true exit

D

qu_lck = "UPDATE UserInfo ..." + temp1 + ";" use(temp4) result_lock = update_lock.executeUpdate(qu_lck) define(temp2)

A Ir

define(temp3)

Database interaction graphs (DIGs) are placed before the database interaction point Ir Multiple DIGs can be integrated into a single CFG

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 8/15

SLIDE 14

Test Adequacy Criteria

all−attribute−value−DUs all−record−DUs all−attribute−DUs all−relation−DUs all−database−DUs

Database interaction association (DIA) involves the def and use of a database entity DIAs can be located in the DICFG with data flow analysis all-database-DUs requires tests to exercise all DIAs for all

f the accessed databases

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 9/15

SLIDE 15

Generating Test Requirements

Database Seeder

Database

(P)

Test Adequacy Criterion

(C)

System Under Test

Test Case Specification Relational Schema Requirements Test

Measured time and space overhead when computing family of test adequacy criteria Modifi ed ATM and mp3cd to contain appropriate database interaction points Soot 1.2.5 to calculate intraprocedural associations GNU/Linux workstation with kernel 2.4.18-smp and dual 1 GHz Pentium III Xeon processors

Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 10/15

SLIDE 16

Counting Associations and Definitions

D Rc Rl A Av Database Granularity 250 500 750 1000 1250 1500 1750 Assoc & Def Count D Rc Rl A Av

mp3cd HD ATM HD mp3cd DB ATM DB

DIAs at attribute value level represent 16.8% of mp3cd’s and 9.6% of ATM’s total number of intraprocedural associations

– p. 11/15

SLIDE 17

Measuring Time Overhead

None D Rc Rl A Av Database Granularity 22.5 25 27.5 30 32.5 35 37.5 System Time sec None D Rc Rl A Av Time Overhead

mp3cd ATM

Computing DIAs at the attribute value level incurs no more than a 5 second time overhead

– p. 12/15

SLIDE 18

Measuring Average Space Overhead

None D Rc Rl A Av Database Granularity 16 18 20 22 24 26 28 30 Node & Edge Count None D Rc Rl A Av

Emp3 Eatm Nmp3 Natm

mp3cd shows a more marked increase in the average number of nodes and edges than ATM

– p. 13/15

SLIDE 19

Measuring Maximum Space Overhead

None D Rc Rl A Av Database Granularity 200 400 600 800 1000 1200 1400 Node & Edge Count None D Rc Rl A Av

Emp3 Eatm Nmp3 Natm

mp3cd shows a signifi cantly greater maximum space

verhead than ATM

– p. 14/15

SLIDE 20

Conclusions

Must test the program’s interaction with the database Unique family of test adequacy criteria to detect type (1) violations

f database validity and completeness

Intraprocedural database interactions can be computed from a DICFG with minimal time and space overhead High number of hanging defi nitions indicates that the scope of data fl

w analysis could be broadened

This data fl

w-based test adequacy criteria can serve as the

foundation for algorithms that measure test suite quality, automatically generate test cases, and support regression testing http://cs.allegheny.edu/˜gkapfham/research/diatoms/

– p. 15/15