Fundamentals of (Static ic and Dynamic ic) ) Software Verif ific - - PowerPoint PPT Presentation

Fundamentals of (Static ic and Dynamic ic) ) Software Verif ific icatio ion

Control-flow Analysis Data-flow Analysis Static VS Dynamic Analysis Software Testing

Con Control

• l Flow
• w Gr

Graph

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 entry exit F T F T

Procedure AVG

S1 count = 0 S2 fread(fptr, n) S3 while (not EOF) do S4 if (n < 0) S5 return (error) else S6 nums[count] = n S7 count ++ endif S8 fread(fptr, n) endwhile S9 avg = mean(nums,count) S10 return(avg)

Domin inance/Postdomin inance

• A node n domin

inates a node m (n dom m) if every path from the entry to m includes n

• A node n postdomin

inates a node m (n pdom m) if every path from the m to the exit includes n

Domin inance

S1 S3 S4 S5 entry exit T S2 S6 F

CFG Dominance Tree

S1 S3 S4 S5 entry S2 S6 exit T F

Postdomin inance

S1 exit entry S2 S5 S6 S4 S3 S1 S3 S4 S5 entry exit T S2 S6 F

CFG

T F

Postdominance Tree

Basic ic Data-Fl Flow co conce ncepts

• Defin

init itio ion and Use Use

Consider statement X = Y + Z

• Definitions?
• Uses?
• Kil

ill and Re Reach

• A definition d of a variable x is killed

at a statement s iff s redefines x and the last assignment to x was d

• A definition d of x reaches s if there

is at least a path from d to s along which x is not killed (def-clear path)

Dependence Analysis is

• Important for
• Optimization

(e.g., instruction scheduling)

• Software engineering
• Program understanding
• Reverse engineering
• Debugging
• Two main kinds of dependences
• Data related
• Control related

Da Data ta-Depen Dependen dence e Gr Graph (DDG) DDG)

DDG DDG: : one node for every basic block and

• ne edge representing

the flow of data between two nodes

entry

Z > 1 X = 1 Z > 2 Y = X + 1 X = 2 Z = X – 3 X = 4 Z = X + 7

exit B1 B3 B2 B6 B5 B4

CF CFG

T F F T

Da Data ta-Depen Dependen dence e Gr Graph ph

entry

Z > 1 X = 1 Z > 2 Y = X + 1 X = 2 Z = X – 3 X = 4 Z = X + 7

exit B1 B3 B2 B6 B5 B4 entry

Z > 1 X = 1 Z > 2 Y = X + 1 X = 2 Z = X – 3 X = 4 Z = X + 7

exit B1 B3 B2 B6 B5 B4

X X X X X Z Z Z Y X Y X Z

DDG DDG CF CFG

T F F T

Con Control

• l-De

Dependence Gr Graph (CDG) G)

Y control-dependent on X iff

1. ∃ path P from X to Y with any Z in P postdominated by Y, and

• 2. X is not postdominated by Y

T F F T

entry

Z > 1 X = 1 Z > 2 Y = X + 1 X = 2 Z = X – 3 X = 4 Z = X + 7

exit B1 B3 B2 B6 B5 B4

CF CFG

Con Control

• l-De

Dependence Gr Graph (CDG) G)

T F F T

entry

Z > 1 X = 1 Z > 2 Y = X + 1 X = 2 Z = X – 3 X = 4 Z = X + 7

exit B1 B3 B2 B6 B5 B4

CF CFG

Intuit itiv ively: two edges out

• f X; traversing one edge

always leads to Y, the other may not lead to Y

Dependences:

B1, exit – entry B2 – B1T B3 – B2T B4 – B1F B5 – B2F, B1F B6 – B2F, B1F

Pr Program-De Dependence Graph h (PDG PDG)

• A PD

PDG for a program P is the combination of the DDG and CDG for P

• A PD

PDG contains nodes representing statements/basic blocks in P and edges representing either control or data dependence between nodes

Static ic VS Dynamic ic Analysis is

• Static

ic analysis ysis operates on a model

• f the SW (w/o executing it)
• Can produce definitive information that

holds for all inputs

• Dyn

ynamic ic analysis ysis operates on dynamic information collected by running the SW

• Produces “sampling information” that

holds for the inputs considered

• Combin

ined st static ic and dyn ynamic ic analyse yses leverage complementary strengths

Do We Need Dynamic ic Analysis is?

• Imprecision of static analysis
• Need to test for properties of

executions (e.g., debugging)

• Need to test assumptions about

the environment

• Need to determine performance

for the average case

• Need to test for non-functional

properties, such as usability

• …

Examples of Dynamic ic Analysis is

• Testing
• Profiling
• Coverage analysis
• Dynamic-invariant detection
• Assertions
• Dynamic tainting
• Dynamic slicing
• …

Issues in in Dynamic ic Analysis is

• Collecting dynamic data
• Instrumentation
• Runtime system
• Debugging interfaces
• Overhead
• Making sure observations don’t

change the behavior of the system

• Selecting the “right” inputs

So Softwa tware e Is Is Bu Buggy

• On average, 1-5 errors per 1KLOC
• Windows 2000
• 35M LOC
• 63,000 known bugs at the time of release
• 2 per 1,000 lines
• For mass market software 100%

correct is infeasible, but

• We must verify the SW as much as

possible

Fail ilure, Fault, Error

Fail ilure

Observable incorrect behavior of a

• program. Conceptually related to the

behavior of the program, rather than its code.

Fa Fault (bug) g)

Related to the code. Necessary (not sufficient!) condition for the occurrence

• f a failure.

Er Error

Cause of a fault. Usually a human error (conceptual, typo, etc.)

Fail ilure, Fault, Error: Example

1. 1. in int double(in int param) { 2. 2. in int result; 3. 3. re result = pa param * pa param; 4. 4. re return rn(re (result); 5. 5. }

A call to double(3) returns 9 Result 9 represents a failure Such failure is due to the fault at line 3 The error is a typo (hopefully)

Approaches to Verif ific icatio ion

• Testing: exercising software to try

and generate failures

• Static verification: identify (specific)

problems statically, that is, considering all possible executions

• Inspection/review/walkthrough:

systematic group review of program text to detect faults

• Formal proof: proving that the

program text implements the program specification

Comparis ison

Testing

• Pros: no false positives
• Limits: incomplete

Static verification

• P: complete (consider all program behaviors)
• L: false positives, expensive

Inspection

• P: systematic, thorough
• L: informal, subjective

Formal proof (of correctness)

• P: strong guarantees
• L: complex, expensive (requires a spec)

What is is Testin ing?

Testing == To execute a program with a sample of the input data

• Dynamic technique: program must be

executed

• Optimistic approximation:
• The program under test is exercised with

a (very small) subset of all the possible input data

• We assume that the behavior with any
• ther input is consistent with the

behavior shown for the selected subset

• f input data

Testin ing Techniq iques

There are a number of techniques

• Different processes
• Different artifacts
• Different approaches

There are no perfect techniques

• Testing is a best-effort activity

There is no best technique

• Different contexts
• Complementary strengths and weaknesses
• Trade-offs

Functio ional vs. Structural Testin ing

T est st-Data Selectio ion

232

32 x

x 232

32 = 2

= 264

64 ≅ 10

1019

19 test

sts s 1 test st per nanose second (109 test sts/ s/se sec) => => 1 1010

10 se

seconds

~60 600 y years rs

Systematic ic Partit itio ion Testin ing

Failure (valuable test case) No failure Failures are sparse in the space of possible inputs ... ... but dense in some parts of the space If we systematically test some cases from each part, we will include the dense parts Functional testing is one way

• f drawing lines to isolate

regions with likely failures The space of possible input values (the haystack)

His istoric ical models

His istoric ical models

printSum: test requir irements

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

Test this case and this one req #1 req #2

printSum: test specif ific icatio ions

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

a + b > 0 a + b < 0

printSum: : te test st case ses

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

Test Spec #1 a + b > 0 Test Spec #2 a + b < 0

3 9 12 12 re red

• 1
• 1

bl blue ue

printSum: : sta state tement t cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

printSum: : sta state tement t cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

Coverage: 0%

a == 3 b == 9

printSum: : sta state tement t cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

Coverage: 71%

a == 3 b == 9

printSum: : sta state tement t cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

Coverage: 71%

a == 3 b == 9 a == 0 b == -1

printSum: : sta state tement t cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

Coverage: 100%

a == 3 b == 9 a == 0 b == -1

printSum: : sta state tement t cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); }

Coverage: 100%

a == 3 b == 9 a == 0 b == -1

printSum: : sta state tement t cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); [else do nothing] }

Coverage: 100%

a == 3 b == 9 a == 0 b == -1

printSum: : sta state tement t cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); [else do nothing] }

Coverage: 100%

a == 3 b == 9 a == 0 b == -1

printSum: : branch cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); [else do nothing]}

a == 3 b == 9 a == 0 b == -1

Coverage: ?

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); [else do nothing]}

a == 3 b == 9 a == 0 b == -1

Coverage: 75% printSum: : branch cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); [else do nothing]}

a == 3 b == 9 a == 0 b == -1

Coverage: 75%

a == -5 b == 5

printSum: : branch cove verage ge

printSum(int a, int b) { int result = a + b; if (result > 0) printcol(“red”, result); else if (result < 0) printcol(“blue”, result); [else do nothing]}

a == 3 b == 9 a == 0 b == -1

Coverage: 100%

a == -5 b == 5

printSum: : branch cove verage ge

Writ ite a faulty y progr gram P and two test st su suit ites s T1 and T2 for P , su such th that: t: 1. 1. T1 achie ieves s 100% branch coverage ge but does s not reveal the fault in in P 2.

• 2. T2 achie

ieves s 100% st statement coverage ge, does s not achie ieve 100% branch coverage ge and reveals s the fault in in P

Writ ite a faulty y progr gram P su such that: 1. 1. Any y test st su suit ite that achie ieves s 100% st statement coverage ge reveals s the fault in in P 2.

• 2. It is

is possib ssible to writ ite a test st su suit ite that achie ieves s 100% branch coverage ge and does s not reveal the fault in in P

Writ ite a faulty y progr gram P su such that: 1. 1. Any y test st su suit ite that achie ieves s 100% path coverage ge reveals s the fault in in P 2.

• 2. It is

is possib ssible to writ ite a test st su suit ite that achie ieves s 100% branch coverage ge and does s not reveal the fault in in P