CSC 509 Lecture Notes Week 2 Assignment 1 Ideas Concepts Underlying - - PowerPoint PPT Presentation

▶
csc 509 lecture notes week 2 assignment 1 ideas concepts
SMART_READER_LITE
LIVE PREVIEW

CSC 509 Lecture Notes Week 2 Assignment 1 Ideas Concepts Underlying - - PowerPoint PPT Presentation

Oct 24, 2023 148 likes •393 views

CSC509-S14-L2 Slide 1 CSC 509 Lecture Notes Week 2 Assignment 1 Ideas Concepts Underlying Testing Research CSC509-S14-L2 Slide 2 I. As described in assignment 1 writeup, ev eryone briefly presents their selected paper. CSC509-S14-L2 Slide

slide-1
SLIDE 1

CSC509-S14-L2 Slide 1

CSC 509 Lecture Notes Week 2 Assignment 1 Ideas Concepts Underlying Testing Research

slide-2
SLIDE 2

CSC509-S14-L2 Slide 2

  • I. As described in assignment 1 writeup,

ev eryone briefly presents their selected paper.

slide-3
SLIDE 3

CSC509-S14-L2 Slide 3

  • II. After the paper presentations, we’ll finish up

testing terminology from Lecture Notes Week 1

slide-4
SLIDE 4

CSC509-S14-L2 Slide 4

  • III. Some more specific testing terminology.
  • A. Consider last five years of ISSTA pubs.
  • B. There have been 140 papers over these years.
  • C. Here are the top five keywords used:
slide-5
SLIDE 5

CSC509-S14-L2 Slide 5

Top ISSTA Keywords, cont’d

  • 1. [automated] test [case] generation (27 times)
  • 2. static analysis (18 times)
  • 3. symbolic execution (11 times)
  • 4. dynamic analysis (10 times)
  • 5. coverage (8 times)
slide-6
SLIDE 6

CSC509-S14-L2 Slide 6

  • D. Majority of papers describe testing tools.
  • 1. Many tools generate executable tests.
  • 2. Others perform analysis before, during, after, or

instead of tests.

slide-7
SLIDE 7

CSC509-S14-L2 Slide 7

  • IV. So how does a testing tool do these things?
slide-8
SLIDE 8

CSC509-S14-L2 Slide 8

  • IV. So how does a testing tool do these things?
  • A. At the core is a program compiler.
slide-9
SLIDE 9

CSC509-S14-L2 Slide 9

  • IV. So how does a testing tool do these things?
  • A. At the core is a program compiler.
  • B. It generates an annotated parse tree,
  • r comparable structure.
slide-10
SLIDE 10

CSC509-S14-L2 Slide 10

  • IV. So how does a testing tool do these things?
  • A. At the core is a program compiler.
  • B. It generates an annotated parse tree,
  • r comparable structure.
  • C. It also generates an symbol table.
slide-11
SLIDE 11

CSC509-S14-L2 Slide 11

  • IV. So how does a testing tool do these things?
  • A. At the core is a program compiler.
  • B. It generates an annotated parse tree,
  • r comparable structure.
  • C. It also generates an symbol table.
  • D. The generation, analysis, execution, or coverage

procedure traverses the tree.

slide-12
SLIDE 12

CSC509-S14-L2 Slide 12

  • IV. So how does a testing tool do these things?
  • A. At the core is a program compiler.
  • B. It generates an annotated parse tree,
  • r comparable structure.
  • C. It also generates an symbol table.
  • D. The generation, analysis, execution, or coverage

procedure traverses the tree.

  • E. During the traversal, tool-specific rules are

applied for the problem at hand.

slide-13
SLIDE 13

CSC509-S14-L2 Slide 13

  • V. Consider the following example:

public class Example { /** * Return true if the given integer is * positive and even. */ /*@ ensures \result == i > 0 && i % 2 == 0 @*/ public static boolean isPositiveEven(int i) { if (i > 0 && i % 2 == 0) return true; else return false; } }

slide-14
SLIDE 14

CSC509-S14-L2 Slide 14

  • VI. (Symbolic) Execution
  • A. Parse program code.
  • B. At each node, apply execution rule.
  • C. E.g., to evaluate ’>’ operator, do this:
slide-15
SLIDE 15

CSC509-S14-L2 Slide 15

(Symbolic) Execution, cont’d

public Value evalGreaterThan( TreeNode expr, SymbolTable symtab) { Value v1 = eval(expr.child1, symtab); Value v2 = eval(expr.child2, symtab); return new BooleanValue(v1.val > v2.val) }

slide-16
SLIDE 16

CSC509-S14-L2 Slide 16

(Symbolic) Execution, cont’d

  • D. Difference between regular and symbolic --
  • 1. For regular, lookup var value in symbol table.
  • 2. For symbolic, use string var name, produce

result with string concatenation.

  • 3. E.g., to symbolically evaluate ’>’ operator:
slide-17
SLIDE 17

CSC509-S14-L2 Slide 17

(Symbolic) Execution, cont’d

public Value evalGreaterThan( TreeNode expr, SymbolTable symtab) { Value v1 = eval(expr.child1, symtab); Value v2 = eval(expr.child2, symtab); if (isLiteral(v1) && isLiteral(v2) return new BooleanValue(v1.val > v2.val) else return new StringValue( v1.val + ">" + v2.val); }

slide-18
SLIDE 18

CSC509-S14-L2 Slide 18

  • VII. Blackbox Test Case Generation
  • A. Apply well-known rules for test cases.
  • B. E.g., five cases for a numeric range:
  • 1. well below bound
  • 2. 1 below bound
  • 3. at bound
  • 4. 1 above bound
  • 5. well above bound
slide-19
SLIDE 19

CSC509-S14-L2 Slide 19

Blackbox Test Case Generation, cont’d

  • C. To implement, e.g. range tests
  • 1. parse pre and post-conditions
  • 2. traverse tree
  • 3. for inequality of the form "x < C", eject test

code like this for each rule-based value:

x = nextRuleBasedValue(); assertTrue(validatePostcond(x,C));

slide-20
SLIDE 20

CSC509-S14-L2 Slide 20

  • VIII. Whitebox Test Case Generation
  • A. Apply similar well-know rules to blackbox.
  • B. Parse code & traverse tree.
  • C. Similar code ejection to blackbox.
slide-21
SLIDE 21

CSC509-S14-L2 Slide 21

  • IX. Coverage
  • A. Annotate program tree with line numbers.
  • B. Execute tree.
  • C. At each tree node with line number, increment

it’s execution count annotation.

  • D. Do post-execution analysis for coverage report.
slide-22
SLIDE 22

CSC509-S14-L2 Slide 22

  • X. Static Analysis, e.g., for Smart Regression
  • A. Trav

erse changed portion of parse tree.

  • B. Determine for each method in symbol table if it

is reachable.

  • C. If so, mark its tests as requiring re-execution.
slide-23
SLIDE 23

CSC509-S14-L2 Slide 23

  • XI. Dynamic Analysis, E.g., Smart Regression
  • A. Parse test code.
  • B. For each called test method, memoize its results.
  • C. If test method called again, use memoized value.