Sum m ary of Sum m ary of Test Case Design Techniques Brian Nielsen, Arne Skou {bnielsen | ask}@cs.auc.dk
Developm ent of Test Cases Complete testing is impossible ologi Testing cannot guarantee the absence of faults tekno How to select subset of test cases from all possible test cases How to select subset of test cases from all possible test cases tionst with a high chance of detecting most faults ? ormat Test Case Design Strategies Info CISS
W hitebox Testing Overview White box testing White box testing ologi Flowgraphs Test criteria/coverage tekno Statement / branch / decision / condition / path coverage St t t / b h / d i i / diti / th MC/DC coverage Looptesting tionst Def-use pairs Efficiency of different criteria ormat BEW ARE: Expected outcome cannot be determined from Info code! CISS
W hite-Box : Statem ent Testing Execute every statement of a program Relatively weak criterion Relatively weak criterion ologi Weakest white-box criterion tekno tionst ormat Info CISS
Exam ple : Statem ent Testing ( result = 0 + 1 + …+ | value| , if this < = m axint , error otherw ise) 1 PROGRAM maxsum ( maxint, value : INT ) ologi 2 INT result := 0 ; i := 0 ; tekno 3 IF value < 0 4 THEN value := - value ; tionst 5 5 WHILE WHILE ( i < value ) AND ( result <= maxint ) ( i < value ) AND ( result < maxint ) 6 DO i := i + 1 ; 7 7 result := result + i ; result := result + i ; ormat 8 OD; 9 9 IF result <= maxint IF result maxint Info 10 THEN OUTPUT ( result ) 11 ELSE OUTPUT ( “too large” ) 12 END. CISS
1 Flow grah Node = statement (blocks) 2 Edges = possible successor (control flow) ologi 1 PROGRAM maxsum ( maxint, value : INT ) 4 3 2 INT result := 0 ; i := 0 ; tekno 3 3 IF IF value < 0 l 0 4 THEN value := - value ; 5 WHILE ( i < value ) AND ( result <= maxint ) 5 6-7 tionst 6 6 DO DO i i := i + 1 ; i + 1 7 result := result + i ; 8 OD; ormat 9 9 IF result <= maxint IF result <= maxint 9 10 THEN OUTPUT ( result ) 11 ELSE OUTPUT ( “too large” ) Info 12 12 END END. 11 10 12 CISS
Flow graph: Cyclomatic complexity # edges - # nodes + 2 Defines the maximal number of test cases needed to provide statement coverage provide statement coverage ologi Mostly applicable for Unit testing Strategy for statement coverage: gy g tekno Derive flow graph 1. Find cyclomatic complexity # c 2. tionst Determine at most # c independent paths through the Determine at most # c independent paths through the 3. 3 program (add one new edge for each test case) Prepare test cases covering the edges for each path 4. ormat (possibly fewer than # c cases) ( ibl f th # ) Info CISS
1 Cyclomatic complexity? y p y 2 1 PROGRAM maxsum ( maxint, value : INT ) 2 2 INT INT result := 0 ; i := 0 ; lt 0 i 0 ologi 3 IF value < 0 4 3 4 THEN value := - value ; tekno 5 5 WHILE WHILE ( i < value ) AND ( result <= maxint ) ( i < l ) AND ( lt < i t ) 6 DO i := i + 1 ; 7 result := result + i ; 5 6-7 tionst 8 8 OD; OD; 9 IF result <= maxint 10 THEN OUTPUT ( result ) ormat 11 11 ELSE ELSE OUTPUT ( too large ) OUTPUT ( “too large” ) 9 12 END. Info 11 10 12 CISS
Exam ple : Statement Testing start yes yes value < 0 value:= -value; ologi no tekno Tests for complete i:=i+1; result:=result+i; statement coverage: g tionst (i<value) and yes (result<=maxint) maxint value ormat no 10 -1 0 -1 Info yes yes no no result<=maxint output(result); output(result); output(“ too large ”); output( too large ); exit CISS
W hite-Box : Path Testing Execute every possible path of a program, i e i.e., every possible sequence of statements every possible sequence of statements ologi Strongest white-box criterion tekno Usually impossible: infinitely many paths ( in case of loops ) So: not a realistic option tionst But note : enormous reduction w.r.t. all possible test cases ( each sequence of statements executed for only one value ) ( eac seque ce o s a e e s e ecu ed o o y o e a ue ) ormat Info CISS
W hite-Box : Branch Testing Branch testing == decision testing Execute every branch of a program : ologi each possible outcome of each decision occurs at least once tekno Example: IF b THEN s1 ELSE s2 tionst IF b THEN s1; s2 CASE x OF ormat 1 : …. 2 : …. Info 3 3 : …. CISS
Exam ple : Branch Testing start Tests for complete yes yes value < 0 value < 0 value:= -value; ologi statement coverage: maxint value no tekno 10 10 -1 1 0 -1 i:=i+1; result:=result+i; is not sufficient for is not sufficient for tionst (i<value) and yes branch coverage; (result<=maxint) ormat T k Take: no maxint value 10 10 3 3 Info yes yes no no result<=maxint 0 -1 for complete output(result); output(result); output( too large ); output(“ too large ”); branch coverage exit CISS
Exam ple : Branch Testing start yes yes maxint maxint value value value < 0 l < 0 value:= -value; l l ologi -1 -1 no -1 -1 -1 -1 tekno 10 10 3 3 10 3 i:=i+1; result:=result+i; tionst But: But: (i<value) and yes No green path ! (result<=maxint) ormat no no Needed : yes yes no o Info result<=maxint lt< i t Combination of decisions 10 -3 output(result); p ( ); output(“ too large ”); p ( ); g exit CISS
W hite-Box : Condition Testing Design test cases such that each possible outcome of each condition in each decision occurs at least once f ologi Example: tekno decision ( i < value ) AND (result <= maxint ) consists of two conditions : ( i < value ) AND (result <= maxint ) tionst t test cases should be designed such that each gets value t h ld b d i d h th t h t l true and false at least once ormat Info CISS
Exam ple : Condition Testing start yes yes value < 0 value:= -value; ologi ( i = result = 0 ) : no maxint value i<value result<=maxint tekno -1 1 true false i:=i+1; 1 0 false true result:=result+i; tionst gives condition coverage i diti (i<value) and yes for all conditions (result<=maxint) But it does not preserve But it does not preserve ormat no decision coverage Info yes yes no no result<=maxint always take care that condition coverage output(result); output(result); output(“ too large ”); output( too large ); preserves decision coverage : preserves decision coverage : decision / condition coverage exit CISS
W hite-Box : Multiple Condition Testing Testing Design test cases for each combination of conditions ologi Example: ( i < value ) ( i value ) (result <= maxint ) (result maxint ) tekno false false false true tionst t true f l false true true Implies decision- condition- decision/condition coverage Implies decision-, condition-, decision/condition coverage ormat But : exponential blow-up Info Again : some combinations may be infeasible Again : some combinations may be infeasible CISS
W hite-box: loop testing Statement and branch coverage are not sufficient Single loop strategy: Zero iterations Zero iterations ologi One iteration Two iterations tekno Typical number of iterations l b f n-1, n, and n+ 1 iterations (n maximum number of allowable iterations) tionst Nested loop strategy: d l Single loop strategy often intractable Select minimum values for outer loop(s) ormat Treat inner loop as a single loop Work ‘outwards’ and choose typical values for inner loops Concatenated loops: Concatenated loops: Info Treat as single, if independent Treat as nested, if dependent CISS
Exam ple : Loop testing Tests for complete start loop coverage: yes yes value < 0 value:= -value; ologi maxint value no 15 15 0 0 tekno 15 1 i:=i+1; result:=result+i; 15 2 tionst 15 3 (i<value) and yes (result<=maxint) 6 4 ormat no 15 5 Info yes yes no no result<=maxint output(result); output(result); output(“ too large ”); output( too large ); exit CISS
W hite-box testing: Data Flow Criteria Basic idea: For each variable definition (assignment), find a path (and a corresponding test case), to its use(s). A a path (and a corresponding test case), to its use(s). A ologi pair (definition,use) is often called a DU pair. Three dominant strategies: tekno All-defs (AD) strategy: follow at least one path from each ll d f ( ) f ll l h f h definition to some use of it All-uses (AU) strategy: follow at least one path for each DU tionst pair All-du-uses strategy (ADUP): follow all paths between a DU pair ormat Complements the testing power of decision coverage Info CISS
Recommend
More recommend
