[PPT] - Using Dynamic Invariant Detection to Support the Testing and PowerPoint Presentation

SLIDE 1

Using Dynamic Invariant Detection to Support the Testing and Analysis

f Database Applications

Gregory M. Kapfhammer†

Department of Computer Science Allegheny College

http://www.cs.allegheny.edu/∼gkapfham/

University of Ulm – January 17, 2012

†Joint with Mary Jean Harrold and Jake Cobb (GA Tech), James A. Jones (UC Irvine), Jonathan Miller Kauffman (Allegheny)

SLIDE 2

Introduction Database Applications Dynamic Invariants Conclusion Important Points

Accessing the Presentation

Scan this QR Code with your smartphone! ... or, visit this Web site: http://www.cs.allegheny.edu/˜gkapfham/ulm2012.pdf ... or, ask me for a USB drive!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 3

Introduction Database Applications Dynamic Invariants Conclusion Important Points

Presenter Introduction: Gregory M. Kapfhammer

test

testing

software

suites

prioritization

components coverage empirically regression suite

applications evaluating

algorithm analysis data effectiveness empirical genetic performance understanding using

approach commercialofftheshelf comparison comprehensive constrained creation databasecentric environments execution finding framework identifying interactive java methods multiplots mutation party prioritized reduction relational study third timeaware towards

105 adequacy approaches array automatically building call challenges chapter communication compare compressing computer conditional cost cots covering creating criteria database databaseaware databasedriven databases declarative dependable detection devices distributed distributing duringsearchbased dynamic efficiency efficient engineering environment evaluate examination executing experimental family flow forward frameworks free generation greedy gui hamiltonian handbook heaps implementation improve incorporating increase information initial intranode invariant javaspace javaspacebased kernel knapsack linux measurement memory method monitoring

perators

paths poster potential preliminary primitives prioritizations prioritizers prioritizing problematic receive remote reports resource resourceconstrained results role runtime science searchbased selection solutions solvers space studies studying supported synthetic techniques transmission transparently trees tuple unstructured wrappers

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 4

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 5

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 6

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 7

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 8

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 9

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 10

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 11

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 12

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Program Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 13

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Program Program Program Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 14

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Program Program Program Program Program Program Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 15

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Everywhere

Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Program Program Program Program Program Program Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 16

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software Complexity and Data Enormity

Computer Software

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 17

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software Complexity and Data Enormity

Computer Software Lines of Code

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 18

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software Complexity and Data Enormity

Computer Software Lines of Code Numerous Features

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 19

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software Complexity and Data Enormity

Computer Software Lines of Code Numerous Features Feature Interactions

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 20

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software Complexity and Data Enormity

Computer Software Lines of Code Numerous Features Feature Interactions Execution Environments

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 21

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software Complexity and Data Enormity

Computer Software Lines of Code Numerous Features Feature Interactions Execution Environments Software entities are more complex for their size than per- haps any other human construct - Frederick P . Brooks, Jr.

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 22

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software Complexity and Data Enormity

Computer Software Lines of Code Numerous Features Feature Interactions Execution Environments Software entities are more complex for their size than per- haps any other human construct - Frederick P . Brooks, Jr. Prediction: in 2011, 1.8 zettabytes (i.e., 1.8 trillion giga- bytes) of data will be created - IDC Digital Universe Study

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 23

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Evolving

Execution Environment Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 24

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Evolving

Execution Environment Program Execution Environment Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 25

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Evolving

Execution Environment Program Execution Environment Program Program Changed because of the addition

f a new feature or the correction of a defect

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 26

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Evolving

Execution Environment Program Execution Environment Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 27

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Evolving

Execution Environment Program Execution Environment Program Execution Environment Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 28

Introduction Database Applications Dynamic Invariants Conclusion Software and Data Challenges

Software and Data are Evolving

Execution Environment Program Execution Environment Program Execution Environment Program Execution Environment Changed due to modification of a kernel, device driver, or relational database

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 29

Introduction Database Applications Dynamic Invariants Conclusion Relational Database Challenges

An Interesting Defect Report

Database Server Crashes

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 30

Introduction Database Applications Dynamic Invariants Conclusion Relational Database Challenges

An Interesting Defect Report

Database Server Crashes When you run a complex query against Microsoft SQL Server 2000, the SQL Server scheduler may stop respond-

ing. Additionally, you receive an error message that resem-

bles the following: Date Time server Error: 17883 Sever- ity: 1, State: 0 Date Time server Process 52:0 (94c) ...

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 31

Introduction Database Applications Dynamic Invariants Conclusion Relational Database Challenges

An Interesting Defect Report

Input-Dependent Defect

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 32

Introduction Database Applications Dynamic Invariants Conclusion Relational Database Challenges

An Interesting Defect Report

Input-Dependent Defect This problem occurs when one or more of the following con- ditions are true: The query contains a UNION clause or a UNIONALL clause that affects many columns. The query contains several JOIN statements. The query has a large estimated cost. BUG 473858 (SQL Server 8.0)

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 33

Introduction Database Applications Dynamic Invariants Conclusion Relational Database Challenges

Real-World Defective Database Application

The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems About 350 airspace boundaries contained in Jeppesen Nav- Data are incorrect, the FAA has warned. The error occurred at Jeppesen after a software upgrade when information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20.

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 34

Introduction Database Applications Dynamic Invariants Conclusion Relational Database Challenges

Real-World Defective Database Application

The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems About 350 airspace boundaries contained in Jeppesen Nav- Data are incorrect, the FAA has warned. The error occurred at Jeppesen after a software upgrade when information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20. Practically all use of databases occurs from within application programs [Silberschatz et al., 2006, pg. 311]

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 35

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Structured Query Language

The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS)

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 36

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Structured Query Language

The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) A schema is a collection of table definitions: CREATE TABLE person ( id INT, name VARCHAR(100) NOT NULL, age INT(3), PRIMARY KEY (id) )

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 37

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Structured Query Language

The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) The data manipulation language supports several operations:

SELECT name FROM person WHERE age >= 30 AND age <= 40

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 38

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Structured Query Language

The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) The data manipulation language supports several operations: UPDATE person SET name = Jan WHERE id = 2

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 39

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Structured Query Language

The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) The data manipulation language supports several operations:

INSERT INTO person (id, name, age) VALUES (1, John, 38)

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 40

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Structured Query Language

The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) The data manipulation language supports several operations: DELETE FROM person WHERE id = 2

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 41

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Relational Database Tables

id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 42

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Relational Database Tables

id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 43

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Relational Database Tables

id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 44

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Relational Database Tables

id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 45

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Relational Database Tables

id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 46

Introduction Database Applications Dynamic Invariants Conclusion Relational Databases

Relational Database Tables

id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 47

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Database Applications

Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 48

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Database Applications

Program Relational Database Management System Database State

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 49

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Database Applications

Program Relational Database Management System Relational Database Management System select Database State query Data Manipulation Language (DML) Statements

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 50

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Database Applications

Program Relational Database Management System Relational Database Management System update Database State modify Data Manipulation Language (DML) Statements

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 51

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Database Applications

Program Relational Database Management System Data Manipulation Language (DML) Statements Relational Database Management System insert Database State modify

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 52

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Database Applications

Program Relational Database Management System Data Manipulation Language (DML) Statements Relational Database Management System delete Database State modify

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 53

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Database Applications

Program Relational Database Management System Relational Database Management System create table Database Structure modify Data Definition Language (DDL) Statements

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 54

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Database Applications

Program Relational Database Management System Relational Database Management System create table Database Structure modify Data Definition Language (DDL) Statements Can we categorize the dif- ferent ways of implementing database applications?

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 55

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 56

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications Interaction Approach

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 57

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications Interaction Approach Program Location

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 58

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications Interaction Approach Program Location Interface

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 59

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications Interaction Approach Program Location Interface Embedded

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 60

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications Interaction Approach Program Location Interface Embedded Outside RDBMS

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 61

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications Interaction Approach Program Location Interface Embedded Outside RDBMS Inside RDBMS

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 62

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications Interaction Approach Program Location Interface Embedded Outside RDBMS Inside RDBMS Interface Outside RDBMS

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 63

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Categorizing Database Applications

Database Applications Interaction Approach Program Location Interface Embedded Outside RDBMS Inside RDBMS Interface Outside RDBMS Java application that submits SQL strings to HSQLDB using JDBC

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 64

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 65

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 66

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Only the database administrator can add new constraints to the schema!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 67

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure The programmers encode the constraints in the program’s source code! Constraint Ci

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 68

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure The programmers encode the constraints in the program’s source code! Constraint Ci Constraint Cj

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 69

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure The programmers encode the constraints in the program’s source code! Constraint Ci Constraint Cj Constraint Ck

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 70

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Constraint Ci Constraint Cj Constraint Ck Constraints Ci, Cj, Ck should be encoded in the schema!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 71

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Constraint Ci Constraint Cj Constraint Ck Goal: extract Ci, Cj, Ck from the source code of the program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 72

Introduction Database Applications Dynamic Invariants Conclusion Programs and Databases

Evolution of Database Applications

Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Goal: extract Ci, Cj, Ck from the source code of the program Database Structure

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 73

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariants

Definition An invariant is a mathematical property that holds through some set of transformations Motivating Examples

0 × y = 0
|x| ≥ 0
C

d = π

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 74

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariants

Definition An invariant is a mathematical property that holds through some set of transformations Motivating Examples

0 × y = 0
|x| ≥ 0
C

d = π

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 75

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariants

Definition An invariant is a mathematical property that holds through some set of transformations Motivating Examples

0 × y = 0
|x| ≥ 0
C

d = π

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 76

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariants

Definition An invariant is a mathematical property that holds through some set of transformations Motivating Examples

0 × y = 0
|x| ≥ 0
C

d = π

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 77

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Program Invariants

Invariant with respect to:

State
Input/Output

Simple Examples

0 ≤ x ≤ 10
1 ≤ nextX() ≤ 11
nextX() =

(x + 1) mod 11

1

class Invariant {

2

static int x = 0;

3

public static int nextX() {

4

if( ++x > 10 )

5

x = 0;

6

return x + 1;

7

}

8

}

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 78

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Program Invariants

Invariant with respect to:

State
Input/Output

Simple Examples

0 ≤ x ≤ 10
1 ≤ nextX() ≤ 11
nextX() =

(x + 1) mod 11

1

class Invariant {

2

static int x = 0;

3

public static int nextX() {

4

if( ++x > 10 )

5

x = 0;

6

return x + 1;

7

}

8

}

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 79

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Program Invariants

Invariant with respect to:

State
Input/Output

Simple Examples

0 ≤ x ≤ 10
1 ≤ nextX() ≤ 11
nextX() =

(x + 1) mod 11

1

class Invariant {

2

static int x = 0;

3

public static int nextX() {

4

if( ++x > 10 )

5

x = 0;

6

return x + 1;

7

}

8

}

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 80

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Program Invariants

Invariant with respect to:

State
Input/Output

Simple Examples

0 ≤ x ≤ 10
1 ≤ nextX() ≤ 11
nextX() =

(x + 1) mod 11

1

class Invariant {

2

static int x = 0;

3

public static int nextX() {

4

if( ++x > 10 )

5

x = 0;

6

return x + 1;

7

}

8

}

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 81

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Dynamic Invariants

Definition A dynamic invariant is a property that is observed to hold during a series of executions

Not guaranteed for all possible executions
May reflect property of:
Program
Inputs

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 82

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Dynamic Invariants

Definition A dynamic invariant is a property that is observed to hold during a series of executions

Not guaranteed for all possible executions
May reflect property of:
Program
Inputs

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 83

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Daikon Invariant Detector

Daikon [Ernst et al. 2001] is a dynamic invariant detection engine originally designed for traditional C and Java programs Detection Process

Collect data traces for variables at program points
Compare to pool of potential invariants
Output remaining invariants that meet confidence threshold

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 84

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Daikon Invariant Detector

Daikon [Ernst et al. 2001] is a dynamic invariant detection engine originally designed for traditional C and Java programs Detection Process

Collect data traces for variables at program points
Compare to pool of potential invariants
Output remaining invariants that meet confidence threshold

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 85

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 86

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Program Instrument Program’

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 87

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Program Instrument Program’ Execute Test Suite Trace File

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 88

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Program Instrument Program’ Execute Test Suite Trace File Daikon Potential Invariants Dynamic Invariants

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 89

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Program Instrument Program’ Execute Test Suite Trace File Daikon Potential Invariants Dynamic Invariants

Supporting Techniques

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 90

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Program Instrument Program’ Execute Test Suite Trace File Daikon Potential Invariants Dynamic Invariants

Supporting Techniques Instrumentation

Instrument

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 91

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Program Instrument Program’ Execute Test Suite Trace File Daikon Potential Invariants Dynamic Invariants

Supporting Techniques Instrumentation

Instrument

Test Suite Execution

Execute

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 92

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Program Instrument Program’ Execute Test Suite Trace File Daikon Potential Invariants Dynamic Invariants

Supporting Techniques Instrumentation

Instrument

Test Suite Execution

Execute

Statistical Analysis

Daikon

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 93

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Invariant Detection Process

Program Instrument Program’ Execute Test Suite Trace File Daikon Potential Invariants Dynamic Invariants

Supporting Techniques Instrumentation

Instrument

Test Suite Execution

Execute

Statistical Analysis

Daikon Refer to [Ernst et al. 2001] for additional details about these techniques

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 94

Introduction Database Applications Dynamic Invariants Conclusion Traditional Invariant Detection

Applications of Daikon

Applications of dynamic invariants in software engineering:

Programmer understanding
Run-time checking
Integration testing
Interface discovery
Test-input generation
Test-suite reduction
Many additional techniques

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 95

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Structural Mapping

Program Element Database Element Program Point Table Variable Column Occurence Row Detect invariants for:

Individual columns
Between columns in a given row

id name age employed . . . 1 ’John Smith’ 38 5 . . . 2 ’Jan Downing’ 22 2 . . .

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 96

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Structural Mapping

Program Element Database Element Program Point Table Variable Column Occurence Row Detect invariants for:

Individual columns
Between columns in a given row

id name age employed . . . 1 ’John Smith’ 38 5 . . . 2 ’Jan Downing’ 22 2 . . .

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 97

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Structural Mapping

Program Element Database Element Program Point Table Variable Column Occurence Row Detect invariants for:

Individual columns
Between columns in a given row

id name age employed . . . 1 ’John Smith’ 38 5 . . . 2 ’Jan Downing’ 22 2 . . .

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 98

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Data Mapping

Daikon Concepts

Representation type
int
double
String
int[]
Comparability

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 99

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Data Mapping

Group Name SQL Types Java Type 1 Text CHAR String VARCHAR TEXT 2 Integer INTEGER int NUMERIC BIT 3 Decimal FLOAT double DOUBLE REAL DECIMAL 4 Binary BLOB byte[] BIT 5 Text Set SET String[] 6 Datetime DATETIME String TIMESTAMP 7 Date DATE String 8 Time TIME String 9 Interval INTERVAL int 10 Primary Key INTEGER reference

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 100

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Data Mapping

Handling NULL Values

NULL is a possible value for any SQL type
Daikon does not accept null for primitive representation

types such as int

Introduce synthetic variable for each NULL-able column
Representation type is hashcode (reference)
Value is either null or a constant

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 101

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Data Mapping

Handling NULL Values

NULL is a possible value for any SQL type
Daikon does not accept null for primitive representation

types such as int

Introduce synthetic variable for each NULL-able column
Representation type is hashcode (reference)
Value is either null or a constant

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 102

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 103

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema Relational Database Management System

Schema Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 104

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema Relational Database Management System

Schema

Scan Database State

All Rows, Columns Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 105

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema Relational Database Management System

Schema

Scan Database State

All Rows, Columns

Instrumented Program

All Modified Rows Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 106

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema Relational Database Management System

Schema

Scan Database State

All Rows, Columns

Instrumented Program

All Modified Rows

Collect Trace

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 107

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema Relational Database Management System

Schema

Scan Database State

All Rows, Columns

Instrumented Program

All Modified Rows

Collect Trace Infer Invariants

Data Trace Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 108

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema Relational Database Management System

Schema

Scan Database State

All Rows, Columns

Instrumented Program

All Modified Rows

Collect Trace Infer Invariants

Data Trace

Relevant Invariants

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 109

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema Relational Database Management System

Schema

Scan Database State

All Rows, Columns

Instrumented Program

All Modified Rows

Collect Trace Infer Invariants

Data Trace

Relevant Invariants Present the Invariants to the Database Administrator

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 110

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Database-Aware Procedure

Read Schema Relational Database Management System

Schema

Scan Database State

All Rows, Columns

Instrumented Program

All Modified Rows

Collect Trace Infer Invariants

Data Trace

Relevant Invariants Present the Invariants to the Database Administrator Place Ci, Cj, Ck into the database schema

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 111

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Implementation

Trace Collector

Pythona program:
Input: Database connection information
Output: Daikon declarations and data trace files
Process:

1 Read schema metadata to determine tables, columns, and

data mapping

2 Write declarations file and serialize mapping info for reuse 3 SELECT table contents, transform data by mapping, write to

GZip’d trace file

Supports various RDBMS with the SQLAlchemy toolkit
a. . . plus a small amount of Cython

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 112

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Implementation

Trace Collector

Pythona program:
Input: Database connection information
Output: Daikon declarations and data trace files
Process:

1 Read schema metadata to determine tables, columns, and

data mapping

2 Write declarations file and serialize mapping info for reuse 3 SELECT table contents, transform data by mapping, write to

GZip’d trace file

Supports various RDBMS with the SQLAlchemy toolkit
a. . . plus a small amount of Cython

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 113

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Implementation

Trace Collector

Pythona program:
Input: Database connection information
Output: Daikon declarations and data trace files
Process:

1 Read schema metadata to determine tables, columns, and

data mapping

2 Write declarations file and serialize mapping info for reuse 3 SELECT table contents, transform data by mapping, write to

GZip’d trace file

Supports various RDBMS with the SQLAlchemy toolkit
a. . . plus a small amount of Cython

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 114

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Implementation

Instrumentation Wrapper

Modified P6Spy JDBC driver wrapper
On connection, capture information and initiate initial

metadata read and trace

On statement execution, append the trace file if the

database could be modified

INSERT statement
UPDATE statement
Unknown (e.g., a stored procedure call)
Ignore others, including DELETE and TRUNCATE

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 115

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Implementation

Instrumentation Wrapper

Modified P6Spy JDBC driver wrapper
On connection, capture information and initiate initial

metadata read and trace

On statement execution, append the trace file if the

database could be modified

INSERT statement
UPDATE statement
Unknown (e.g., a stored procedure call)
Ignore others, including DELETE and TRUNCATE

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 116

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Invariant Detection

Implementation

Instrumentation Wrapper

Modified P6Spy JDBC driver wrapper
On connection, capture information and initiate initial

metadata read and trace

On statement execution, append the trace file if the

database could be modified

INSERT statement
UPDATE statement
Unknown (e.g., a stored procedure call)
Ignore others, including DELETE and TRUNCATE

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 117

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Objects of Analysis

Fixed Data Sets Database Tables Columns Rows world 3 24 5302 sakila 23 131 50,086 menagerie 2 10 19 employees 6 24 3,919,015

MySQL sample databases commonly used for training,

certification, and testing

Trace the entire dataset during invariant detection

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 118

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Objects of Analysis

Database Applications

Program iTrust JWhoisServer JTrac Tables 30 7 13 Columns 177 57 126 KLOC 25.5 (Java), 8.6 (JSP) 6.7 12 Test Cases 787 67 41

Java applications that interact with a relational database
Wrap real database driver in a modified P6Spy driver
Execute the entire test suite during invariant detection

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 119

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Meaningful Invariants Invariants that capture a semantic relationship

dept_emp.from_date <= dept_emp.to_date
employees.gender one of { "F", "M" }
employees.birth_date < employees.hire_date
country.Population >= 0
icdcodes.Chronic one of { "no", "yes" }

All of these invariants were automatically generated!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 120

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Meaningful Invariants Invariants that capture a semantic relationship

dept_emp.from_date <= dept_emp.to_date
employees.gender one of { "F", "M" }
employees.birth_date < employees.hire_date
country.Population >= 0
icdcodes.Chronic one of { "no", "yes" }

All of these invariants were automatically generated!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 121

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Meaningful Invariants Invariants that capture a semantic relationship

dept_emp.from_date <= dept_emp.to_date
employees.gender one of { "F", "M" }
employees.birth_date < employees.hire_date
country.Population >= 0
icdcodes.Chronic one of { "no", "yes" }

All of these invariants were automatically generated!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 122

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Meaningful Invariants Invariants that capture a semantic relationship

dept_emp.from_date <= dept_emp.to_date
employees.gender one of { "F", "M" }
employees.birth_date < employees.hire_date
country.Population >= 0
icdcodes.Chronic one of { "no", "yes" }

All of these invariants were automatically generated!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 123

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Meaningful Invariants Invariants that capture a semantic relationship

dept_emp.from_date <= dept_emp.to_date
employees.gender one of { "F", "M" }
employees.birth_date < employees.hire_date
country.Population >= 0
icdcodes.Chronic one of { "no", "yes" }

All of these invariants were automatically generated!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 124

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Meaningful Invariants Invariants that capture a semantic relationship

dept_emp.from_date <= dept_emp.to_date
employees.gender one of { "F", "M" }
employees.birth_date < employees.hire_date
country.Population >= 0
icdcodes.Chronic one of { "no", "yes" }

All of these invariants were automatically generated!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 125

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Meaningful Invariants Invariants that capture a semantic relationship

dept_emp.from_date <= dept_emp.to_date
employees.gender one of { "F", "M" }
employees.birth_date < employees.hire_date
country.Population >= 0
icdcodes.Chronic one of { "no", "yes" }

All of these invariants were automatically generated!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 126

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Spurious Invariants

Vacuous invariants reflect a meaningless relationship.
patients.phone1 <= patients.BloodType
patients.lastName >= patients.address1
cptcodes.Description != cptcodes.Attribute
Lack-of-data invariants result from limited data samples.
mntnr.login == "mntnt"
inetnum.changed == "2006-10-14 16:21:09"
person.name one of { "no name company",

"persona non grata"}

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 127

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Spurious Invariants

Vacuous invariants reflect a meaningless relationship.
patients.phone1 <= patients.BloodType
patients.lastName >= patients.address1
cptcodes.Description != cptcodes.Attribute
Lack-of-data invariants result from limited data samples.
mntnr.login == "mntnt"
inetnum.changed == "2006-10-14 16:21:09"
person.name one of { "no name company",

"persona non grata"}

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 128

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Spurious Invariants

Vacuous invariants reflect a meaningless relationship.
patients.phone1 <= patients.BloodType
patients.lastName >= patients.address1
cptcodes.Description != cptcodes.Attribute
Lack-of-data invariants result from limited data samples.
mntnr.login == "mntnt"
inetnum.changed == "2006-10-14 16:21:09"
person.name one of { "no name company",

"persona non grata"}

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 129

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Number of Invariants

employees world menagerie JWhoisServer JTrac sakila iTrust 50 100 150 Type of Invariant

Vacuous Lack−of−data Meaningful

The majority of detected dynamic invariants are not spurious

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 130

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Number of Invariants

employees world menagerie JWhoisServer JTrac sakila iTrust 50 100 150 Type of Invariant

Vacuous Lack−of−data Meaningful

employees’ invariants are meaningful due to wealth of data

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 131

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Invariant Quality

Number of Invariants

employees world menagerie JWhoisServer JTrac sakila iTrust 50 100 150 Type of Invariant

Vacuous Lack−of−data Meaningful

JWhoisServer’s few meaningful invariants suggests poor tests

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 132

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Using Dynamic Invariants

Some invariants can be enforced by the schema definition
Schema enforcement:
Provides a stronger assurance of data integrity than

application enforcement

Enables easy long-term maintenance of the program and

the relational database

Analyze enforceable invariants:
Already enforced by the schema
Suggest modification to enforce the invariant

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 133

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Using Dynamic Invariants

Some invariants can be enforced by the schema definition
Schema enforcement:
Provides a stronger assurance of data integrity than

application enforcement

Enables easy long-term maintenance of the program and

the relational database

Analyze enforceable invariants:
Already enforced by the schema
Suggest modification to enforce the invariant

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 134

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Schema Enforced

Invariant Schema Definition

employees.gender one of { "F", "M" } ENUM(’F’,’M’) countrylanguage.IsOfficial one of { "F", "T" } ENUM(’F’,’T’) customer.active one of { 0, 1 } TINYINT(1) inventory.film id >= 1 SMALLINT(5) UNSIGNED spaces.guest allowed one of { 0, 1 } BIT(1)

Reverse engineered many constraints already enforced by the schema

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 135

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Schema Enforced

Invariant Schema Definition

employees.gender one of { "F", "M" } ENUM(’F’,’M’) countrylanguage.IsOfficial one of { "F", "T" } ENUM(’F’,’T’) customer.active one of { 0, 1 } TINYINT(1) inventory.film id >= 1 SMALLINT(5) UNSIGNED spaces.guest allowed one of { 0, 1 } BIT(1)

Reverse engineered many constraints already enforced by the schema

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 136

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Schema Enforceable Invariant Schema Modification

isnull(message.message) != null TEXT NOT NULL isnull(film text.description) != null TEXT NOT NULL isnull(history.time stamp) != null DATETIME NOT NULL user space roles.user id >= 1 BIGINT(20) UNSIGNED pet.sex one of { "f", "m" } CHAR(1) ENUM(’m’,’f’) country.Population >= 0 INT(11) UNSIGNED isnull(titles.to date) != null DATE NOT NULL

Many detected constraints can easily be added to the schema

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 137

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Schema Enforceable Invariant Schema Modification

isnull(message.message) != null TEXT NOT NULL isnull(film text.description) != null TEXT NOT NULL isnull(history.time stamp) != null DATETIME NOT NULL user space roles.user id >= 1 BIGINT(20) UNSIGNED pet.sex one of { "f", "m" } CHAR(1) ENUM(’m’,’f’) country.Population >= 0 INT(11) UNSIGNED isnull(titles.to date) != null DATE NOT NULL

Many detected constraints can easily be added to the schema

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 138

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Percentage of Meaningful Invariants

JWhoisServer menagerie JTrac employees iTrust world sakila 0.0 0.2 0.4 0.6 0.8 1.0 Type of Meaningful Invariant

Enforceable with Standards−Compliant Database Enforceable with Current Database Already Enforced

Percentages relative to the total number of non-spurious invariants

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 139

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Percentage of Meaningful Invariants

JWhoisServer menagerie JTrac employees iTrust world sakila 0.0 0.2 0.4 0.6 0.8 1.0 Type of Meaningful Invariant

Enforceable with Standards−Compliant Database Enforceable with Current Database Already Enforced

All constraints enforceable by a standards-compliant database

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 140

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Percentage of Meaningful Invariants

JWhoisServer menagerie JTrac employees iTrust world sakila 0.0 0.2 0.4 0.6 0.8 1.0 Type of Meaningful Invariant

Enforceable with Standards−Compliant Database Enforceable with Current Database Already Enforced

Three schemas can be enhanced with many new constraints

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 141

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Percentage of Meaningful Invariants

JWhoisServer menagerie JTrac employees iTrust world sakila 0.0 0.2 0.4 0.6 0.8 1.0 Type of Meaningful Invariant

Enforceable with Standards−Compliant Database Enforceable with Current Database Already Enforced

menagerie did not already enforce any of the meaningful invariants

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 142

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Schema Modification

Percentage of Meaningful Invariants

JWhoisServer menagerie JTrac employees iTrust world sakila 0.0 0.2 0.4 0.6 0.8 1.0 Type of Meaningful Invariant

Enforceable with Standards−Compliant Database Enforceable with Current Database Already Enforced

JWhoisServer’s MySQL doesn’t support constraint enforcement

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 143

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Empirical Conclusions and Future Research

Conclusions

Meaningful invariants may be mined from both relational

databases and database applications

Invariant quality depends on existence of diverse data
Data integrity may be enhanced by using invariants for

modification of the database’s schema Future Research

Invariants between multiple tables
Invariants for individual queries
Explore additional client applications

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 144

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Empirical Conclusions and Future Research

Conclusions

Meaningful invariants may be mined from both relational

databases and database applications

Invariant quality depends on existence of diverse data
Data integrity may be enhanced by using invariants for

modification of the database’s schema Future Research

Invariants between multiple tables
Invariants for individual queries
Explore additional client applications

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 145

Introduction Database Applications Dynamic Invariants Conclusion Empirical Results

Empirical Conclusions and Future Research

Conclusions

Meaningful invariants may be mined from both relational

databases and database applications

Invariant quality depends on existence of diverse data
Data integrity may be enhanced by using invariants for

modification of the database’s schema Future Research

Invariants between multiple tables
Invariants for individual queries
Explore additional client applications

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 146

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 147

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Input

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 148

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Input Output

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 149

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 150

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 151

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 152

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output Test Clean Up

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 153

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output Test Clean Up Test Oracle

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 154

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 155

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 156

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 157

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Test Verdict The test case passes and the code is correct!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 158

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 159

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Case?

Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Test Verdict The test case fails and a defect is found!

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 160

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 161

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 162

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 163

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3 T4

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 164

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3 T4 . . .

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 165

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3 T4 . . . Tn

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 166

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 167

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing?

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 168

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 169

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit Apache Ant

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 170

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

What is a Test Suite?

T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit Apache Ant Eclipse

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 171

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 172

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 173

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique What if Some Test Cases are More Effective? T3 Tn

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 174

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique What if Some Test Cases are More Effective? T3 Tn Prioritization T3 Tn T1 T4 . . . T2

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 175

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique What if Some Test Cases are More Effective? T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 176

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn What if Some Test Cases are Redundant? T1 T2 T3 T4 . . . Tn

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 177

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn What if Some Test Cases are Redundant? T1 T2 T3 T4 . . . Tn Reduction T4

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 178

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn What if Some Test Cases are Redundant? T1 T2 T3 T4 . . . Tn Reduction T4 T1 T2 T3 T4 . . . Tn

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 179

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn T1 T2 T3 T4 . . . Tn Reduction T4 T1 T2 T3 T4 . . . Tn What if Only Certain Tests are Needed? T2 Tn

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 180

Introduction Database Applications Dynamic Invariants Conclusion Regression Testing

Test Suite Management

T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn T1 T2 T3 T4 . . . Tn Reduction T4 T1 T2 T3 T4 . . . Tn What if Only Certain Tests are Needed? T2 Tn Selection T1 T2 T3 T4 . . . Tn

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 181

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 182

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 183

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 184

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program Test Suite

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 185

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program Test Suite Database

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 186

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program Test Suite Database Invariants

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 187

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program Test Suite Database Invariants

T1, . . . , T10

=

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 188

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program Test Suite Database Invariants

T1, . . . , T10

=

T41, . . . , T50

=

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 189

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program Test Suite Database Invariants

T1, . . . , T10

=

T41, . . . , T50

=

T21, . . . , T30

=

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 190

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program Test Suite Database Invariants

T1, . . . , T10

=

T41, . . . , T50

=

T21, . . . , T30

= Divide and Conquer

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 191

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Database-Aware Test Suite Reduction

T1, . . . , T10 T11, . . . , T20 T21, . . . , T30 T31, . . . , T40 T41, . . . , T50

Dynamic Invariant Detection Program Test Suite Database Invariants

T1, . . . , T10

=

T41, . . . , T50

=

T21, . . . , T30

Divide and Conquer =

T21, . . . , T30

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 192

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Conclusion

Databases are widely used in real-world applications
Database applications have complex state and structure
Programmers often encode constraints in program source
Dynamic invariant detection reverse engineers constraints
Detected invariants are meaningful and enforceable

Future Work

Further empirical studies of dynamic invariants
Implement and evaluate several client applications

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 193

Introduction Database Applications Dynamic Invariants Conclusion Database-Aware Method

Conclusion

Databases are widely used in real-world applications
Database applications have complex state and structure
Programmers often encode constraints in program source
Dynamic invariant detection reverse engineers constraints
Detected invariants are meaningful and enforceable

Future Work

Further empirical studies of dynamic invariants
Implement and evaluate several client applications

Kapfhammer Allegheny College Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

SLIDE 194

Using Dynamic Invariant Detection to Support the Testing and Analysis

f Database Applications

Gregory M. Kapfhammer

Department of Computer Science Allegheny College

http://www.cs.allegheny.edu/∼gkapfham/

Thank you for your attention! I welcome your questions and comments.