Using Conditional Mutation to Increase the Efficiency of Mutation - - PowerPoint PPT Presentation

▶
using conditional mutation to increase the efficiency of
SMART_READER_LITE
LIVE PREVIEW

Using Conditional Mutation to Increase the Efficiency of Mutation - - PowerPoint PPT Presentation

Jul 02, 2023 206 likes •953 views

Using Conditional Mutation to Increase the Efficiency of Mutation Analysis e Just 1 & Gregory M. Kapfhammer 2 & Franz Schweiggert 1 Ren 1 Ulm University, Germany 2 Allegheny College, USA 6th International Workshop on the Automation of

slide-1
SLIDE 1

Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

Ren´ e Just1 & Gregory M. Kapfhammer2 & Franz Schweiggert1

1Ulm University, Germany 2Allegheny College, USA

6th International Workshop on the Automation of Software Test Waikiki, Honolulu, Hawaii, USA May 23 - 24, 2011

slide-2
SLIDE 2

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-3
SLIDE 3

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis Challenges

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-4
SLIDE 4

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis Challenges Solutions

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-5
SLIDE 5

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis Challenges Solutions Conditional Mutation

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-6
SLIDE 6

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-7
SLIDE 7

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-8
SLIDE 8

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-9
SLIDE 9

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated Comprehensive Empirical Study

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-10
SLIDE 10

Introduction Conditional Mutation Implementation Conclusion

Overview of the Presentation

Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated Comprehensive Empirical Study Efficient Technique - Fully Integrated into the Java 6 SE Compiler

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-11
SLIDE 11

Introduction Conditional Mutation Implementation Conclusion

Overview of Mutation Analysis

public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; }

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-12
SLIDE 12

Introduction Conditional Mutation Implementation Conclusion

Overview of Mutation Analysis

public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; }

Methodically inject small syntactical faults into the program under test

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-13
SLIDE 13

Introduction Conditional Mutation Implementation Conclusion

Overview of Mutation Analysis

public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; }

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-14
SLIDE 14

Introduction Conditional Mutation Implementation Conclusion

Overview of Mutation Analysis

public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; }

= ⇒ = ⇒ y = a - x; y = a + x; y = a / x; if(b < a) if(b != a) if(b == a)

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-15
SLIDE 15

Introduction Conditional Mutation Implementation Conclusion

Overview of Mutation Analysis

public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; }

Unbiased and powerful method for assessing

  • racles and

input values

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-16
SLIDE 16

Introduction Conditional Mutation Implementation Conclusion

Overview of Mutation Analysis

public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; }

Unbiased and powerful method for assessing

  • racles and

input values Useful method for fault seeding during the empirical study

  • f testing

techniques

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-17
SLIDE 17

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-18
SLIDE 18

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-19
SLIDE 19

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-20
SLIDE 20

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program Mutants

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-21
SLIDE 21

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-22
SLIDE 22

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-23
SLIDE 23

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-24
SLIDE 24

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-25
SLIDE 25

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-26
SLIDE 26

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results Individually Executing the Mutants is Too Expensive

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-27
SLIDE 27

Introduction Conditional Mutation Implementation Conclusion

Mutation Analysis Challenges

Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results Individually Executing the Mutants is Too Expensive Prior Solutions?

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-28
SLIDE 28

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-29
SLIDE 29

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis Offutt and Untch

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-30
SLIDE 30

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis Offutt and Untch Do Fewer

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-31
SLIDE 31

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis Offutt and Untch Do Fewer Sampling Selection

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-32
SLIDE 32

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-33
SLIDE 33

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Distributed Weak Mutation

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-34
SLIDE 34

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-35
SLIDE 35

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster Compiler Integrated Bytecode Transformation Mutant Schemata

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-36
SLIDE 36

Introduction Conditional Mutation Implementation Conclusion

Prior Work in Mutation Analysis

Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster Higher Order Mutation Jia and Harman

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-37
SLIDE 37

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation

Conditional Mutation

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-38
SLIDE 38

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation

Conditional Mutation Encapsulates all mutants within the same block

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-39
SLIDE 39

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation

Conditional Mutation Encapsulates all mutants within the same block Can be inte- grated within the compiler

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-40
SLIDE 40

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation

Conditional Mutation Encapsulates all mutants within the same block Transforms the abstract syntax tree (AST) Can be inte- grated within the compiler

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-41
SLIDE 41

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation

Conditional Mutation Encapsulates all mutants within the same block Transforms the abstract syntax tree (AST) Stmt → Conditional Stmt (if-then-else, switch) Expr → Conditional Expr (conditional operator ?:) Can be inte- grated within the compiler

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-42
SLIDE 42

Introduction Conditional Mutation Implementation Conclusion

Working Example

public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; }

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-43
SLIDE 43

Introduction Conditional Mutation Implementation Conclusion

Working Example

public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; }

⇓

ASSIGN IDENT y BINARY ∗ a x

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-44
SLIDE 44

Introduction Conditional Mutation Implementation Conclusion

Working Example

public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; }

⇓

ASSIGN IDENT y BINARY ∗ a x

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-45
SLIDE 45

Introduction Conditional Mutation Implementation Conclusion

Working Example

public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; }

⇓

ASSIGN IDENT y BINARY ∗ a x

= ⇒ = ⇒

ASSIGN IDENT y COND-EXPR THEN BINARY + a x COND (M NO ==2) ELSE COND-EXPR THEN BINARY

  • a

x COND (M NO ==1) ELSE BINARY ∗ a x

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-46
SLIDE 46

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation Algorithm

public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; }

1

Define mutation operators

MOP(x ∗ y) = {x − y, x + y, x/y}

2

Determine whether current expression or statement is affected by mutation

3

Apply mutation operators

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-47
SLIDE 47

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation Algorithm

public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; }

1

Define mutation operators

MOP(x ∗ y) = {x − y, x + y, x/y}

2

Determine whether current expression or statement is affected by mutation

3

Apply mutation operators

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-48
SLIDE 48

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation Algorithm

public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; }

1

Define mutation operators

MOP(x ∗ y) = {x − y, x + y, x/y}

2

Determine whether current expression or statement is affected by mutation

3

Apply mutation operators

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-49
SLIDE 49

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation Algorithm

public int eval(int x){ int a=3, b=1, y; y = (M_NO==1)? a - x : a * x ; y += b; return y; }

1

Define mutation operators

MOP(x ∗ y) = {x − y, x + y, x/y}

2

Determine whether current expression or statement is affected by mutation

3

Apply mutation operators

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-50
SLIDE 50

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation Algorithm

public int eval(int x){ int a=3, b=1, y; y = (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; }

1

Define mutation operators

MOP(x ∗ y) = {x − y, x + y, x/y}

2

Determine whether current expression or statement is affected by mutation

3

Apply mutation operators

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-51
SLIDE 51

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation Algorithm

public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; }

1

Define mutation operators

MOP(x ∗ y) = {x − y, x + y, x/y}

2

Determine whether current expression or statement is affected by mutation

3

Apply mutation operators Versatile approach, can be combined with prior solutions Formal description and implementation details in the paper

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-52
SLIDE 52

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation Algorithm

public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; }

1

Define mutation operators

MOP(x ∗ y) = {x − y, x + y, x/y}

2

Determine whether current expression or statement is affected by mutation

3

Apply mutation operators Versatile approach, can be combined with prior solutions Formal description and implementation details in the paper

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-53
SLIDE 53

Introduction Conditional Mutation Implementation Conclusion

Conditional Mutation Algorithm

public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; }

1

Define mutation operators

MOP(x ∗ y) = {x − y, x + y, x/y}

2

Determine whether current expression or statement is affected by mutation

3

Apply mutation operators Versatile approach, can be combined with prior solutions Formal description and implementation details in the paper

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-54
SLIDE 54

Introduction Conditional Mutation Implementation Conclusion

Mutation Coverage

public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; }

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-55
SLIDE 55

Introduction Conditional Mutation Implementation Conclusion

Mutation Coverage

public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; }

Mutants not exe- cuted cannot be killed

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-56
SLIDE 56

Introduction Conditional Mutation Implementation Conclusion

Mutation Coverage

public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : (M_NO==0 && COVERED(1,3))? a * x : a * x ; y += b; return y; }

Mutants not exe- cuted cannot be killed Determine covered mutants with addi- tional instrumentation

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-57
SLIDE 57

Introduction Conditional Mutation Implementation Conclusion

Mutation Coverage

public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : (M_NO==0 && COVERED(1,3))? a * x : a * x ; y += b; return y; }

Mutants not exe- cuted cannot be killed Determine covered mutants with addi- tional instrumentation Only execute and investi- gate the covered mutants

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-58
SLIDE 58

Introduction Conditional Mutation Implementation Conclusion

MAJOR: Mutation Analysis in a Java Compiler

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-59
SLIDE 59

Introduction Conditional Mutation Implementation Conclusion

MAJOR: Mutation Analysis in a Java Compiler

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-60
SLIDE 60

Introduction Conditional Mutation Implementation Conclusion

MAJOR: Mutation Analysis in a Java Compiler

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-61
SLIDE 61

Introduction Conditional Mutation Implementation Conclusion

MAJOR: Mutation Analysis in a Java Compiler

Configuration: Common compiler options Domain specific language (DSL) http://www.mathematik.uni-ulm.de/sai/major

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-62
SLIDE 62

Introduction Conditional Mutation Implementation Conclusion

Performance Analysis

1 2 3 4 5 6 7 8 9 10 11 12 20000 40000 60000 80000 100000 120000 140000 Compiler runtime in seconds Number of mutants apache ant jfreechart itext java pathfinder commons math commons lang numerics4j

Overhead for generating and compiling mutants is negligible

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-63
SLIDE 63

Introduction Conditional Mutation Implementation Conclusion

Performance Analysis

1 2 3 4 5 6 7 8 9 10 11 12 20000 40000 60000 80000 100000 120000 140000 Compiler runtime in seconds Number of mutants apache ant jfreechart itext java pathfinder commons math commons lang numerics4j

Overhead for generating and compiling mutants is negligible

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-64
SLIDE 64

Introduction Conditional Mutation Implementation Conclusion

Performance Analysis

Application Mutants Runtime of test suite Memory consumption

  • riginal

instrumented

  • riginal

instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90

Runtime overhead is application dependent

Larger for CPU-bound applications Small for I/O-bound applications

Even for large projects, applicable on commodity workstations

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-65
SLIDE 65

Introduction Conditional Mutation Implementation Conclusion

Performance Analysis

Application Mutants Runtime of test suite Memory consumption

  • riginal

instrumented

  • riginal

instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90

Runtime overhead is application dependent

Larger for CPU-bound applications Small for I/O-bound applications

Even for large projects, applicable on commodity workstations

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-66
SLIDE 66

Introduction Conditional Mutation Implementation Conclusion

Performance Analysis

Application Mutants Runtime of test suite Memory consumption

  • riginal

instrumented

  • riginal

instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90

Runtime overhead is application dependent

Larger for CPU-bound applications Small for I/O-bound applications

Even for large projects, applicable on commodity workstations

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-67
SLIDE 67

Introduction Conditional Mutation Implementation Conclusion

Performance Analysis

Application Mutants Runtime of test suite Memory consumption

  • riginal

instrumented

  • riginal

instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90

Runtime overhead is application dependent

Larger for CPU-bound applications Small for I/O-bound applications

Even for large projects, applicable on commodity workstations

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-68
SLIDE 68

Introduction Conditional Mutation Implementation Conclusion

Performance Analysis

Application Mutants Runtime of test suite Memory consumption

  • riginal

instrumented

  • riginal

instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90

Runtime overhead is application dependent

Larger for CPU-bound applications Small for I/O-bound applications

Even for large projects, applicable on commodity workstations

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-69
SLIDE 69

Introduction Conditional Mutation Implementation Conclusion

Enabling Efficient Mutation Analysis

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-70
SLIDE 70

Introduction Conditional Mutation Implementation Conclusion

Enabling Efficient Mutation Analysis

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-71
SLIDE 71

Introduction Conditional Mutation Implementation Conclusion

Conclusion

Conclusion: Largest empirical study of mutation analysis to date Mutant generation time reduced to a minimum Mutation coverage provides runtime optimization Versatilely applicable in every Java-based environment Arbitrary conditions enable support for higher order mutation Future Work: Implement new mutation operators Enhance the domain specific language

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-72
SLIDE 72

Introduction Conditional Mutation Implementation Conclusion

Conclusion

Conclusion: Largest empirical study of mutation analysis to date Mutant generation time reduced to a minimum Mutation coverage provides runtime optimization Versatilely applicable in every Java-based environment Arbitrary conditions enable support for higher order mutation Future Work: Implement new mutation operators Enhance the domain specific language

Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

slide-73
SLIDE 73

Using Conditional Mutation to Increase the Efficiency of Mutation Analysis Thank you for your attention! Questions?

http://www.mathematik.uni-ulm.de/sai/major