efficient and effective mutation testing supporting the
play

Efficient and Effective Mutation Testing: Supporting the - PowerPoint PPT Presentation

Nov 14, 2022 •573 likes •2.33k views

Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/ gkapfham/

  1. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Expected Expected Test Output Output Set Up Method Test Input Output Output Under Test Oracle Test Test Test Clean Up Verdict Verdict The test case fails and a defect is found! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  2. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  3. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 T 3 T 4 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  4. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 T 3 T 4 T 5 T 6 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  5. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  6. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  7. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  8. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  9. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  10. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  11. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  12. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  13. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  14. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  15. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 B 2 B 2 B 2 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  16. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 B 2 B 2 B 2 B 2 B 2 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  17. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  18. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  19. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 How Good is Test Suite T ? R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  20. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 How Good is Test Suite T ? Coverage Analysis R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  21. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 How Good is Test Suite T ? Coverage Analysis Mutation Analysis R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  22. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults if(a > 10) Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  23. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults if(a > 10) if(a >= 10) Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  24. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults if(a > 10) if(a >= 10) Implemented Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  25. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults if(a > 10) if(a >= 10) Implemented Potential Fault Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  26. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a if(a > 10) if(a >= 10) true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  27. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a if(a > 10) if(a >= 10) true false true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  28. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 0 if(a > 10) if(a >= 10) true false true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  29. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 0 if(a > 10) if(a >= 10) true false false true false false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  30. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 15 if(a > 10) if(a >= 10) true false true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  31. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 15 if(a > 10) if(a >= 10) true true false true true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  32. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  33. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  34. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false Can the tests differentiate between implemented and potential fault ? Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  35. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false If yes , then the tests are adequate ! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  36. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false If no , then the tests must be improved ! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  37. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false Purposefully insert faults in order to implement quality software! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  38. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  39. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Operator Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  40. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  41. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  42. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  43. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  44. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  45. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  46. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  47. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  48. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  49. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  50. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  51. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  52. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  53. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  54. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  55. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  56. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 The test suite Execute the cannot kill the test suite after mutant – either enabling a a test suite single mutant weakness or in the program an equivalent under test mutant! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  57. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Repeat this Execute the process for test suite after all of the enabling a test cases single mutant and mutants in the program – calculate mutation score under test when finished Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  58. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  59. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  60. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  61. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Conditional Mutation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  62. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Conditional Mutation Syntax Tree Transformation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  63. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  64. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Conditional Mutation Compiler Syntax Tree Integrated Transformation Expressions and Statements Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  65. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Comprehensive Mutation Empirical Study Analysis Solutions Conditional Mutation Compiler Syntax Tree Integrated Transformation Expressions and Statements Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  66. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Technique - Fully Integrated into the Java 6 SE Compiler Efficient Challenges Comprehensive Mutation Empirical Study Analysis Solutions Conditional Mutation Compiler Syntax Tree Integrated Transformation Expressions and Statements Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  67. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max( int a, int b){ int max = a; if (b>a){ max=b; } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  68. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ int a=3, b=1, y; y = a * x; Methodically y += b; inject small return y; } syntactical public int max( int a, int b){ faults into int max = a; the program under test if (b>a){ max=b; } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  69. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max( int a, int b){ int max = a; if (b>a){ max=b; } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  70. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ int a=3, b=1, y; • y = a - x; • y = a + x; ⇒ y = a * x; = • y = a / x; y += b; return y; } public int max( int a, int b){ int max = a; • if(b < a) • if(b != a) if (b>a){ ⇒ = max=b; • if(b == a) } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  71. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ Unbiased int a=3, b=1, y; and powerful y = a * x; method for assessing y += b; oracles and return y; } input values public int max( int a, int b){ int max = a; if (b>a){ max=b; } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  72. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ Unbiased int a=3, b=1, y; and powerful y = a * x; method for assessing y += b; oracles and return y; } input values public int max( int a, int b){ Useful method int max = a; for fault seeding if (b>a){ during the max=b; empirical study } of testing return max; techniques } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  73. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  74. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutation Mutant Operators Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  75. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Program Mutation Mutant Operators Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  76. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Program Mutation Mutant Mutants Operators Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  77. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Often Yields a Substantial Num- Program ber of Mutants Mutation Mutant Mutants Operators Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  78. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Often Yields a Substantial Num- Program ber of Mutants Mutation Mutant Mutants Operators Generation High Time Over- head for Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  79. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Often Yields a Substantial Num- Program ber of Mutants Mutation Mutant Mutants Operators Generation Mutation Analysis High Time Over- head for Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  80. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Often Yields a Substantial Num- Program ber of Mutants Mutation Mutant Mutants Operators Generation Mutation Tests Analysis High Time Over- head for Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with Mark Harman and William Langdon Agenda Why Higher Order Mutation Testing? Search for interesting HOMs Milu mutation testing tool Scalability and

669 views • 38 slides
Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and

Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and

Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and Scalable Mutation Analysis Ren Just 1 , 2 &amp; Gregory M. Kapfhammer 3 &amp; Franz Schweiggert 2 1 University of Washington, USA 2 Ulm University,

1.18k views • 85 slides
Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings College, London W. B. Langdon, Crest 1 Introduction What is mutation testing 2 objectives: Hard to kill, little change to source Higher

417 views • 29 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

350 views • 6 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

306 views • 7 slides
Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Introduction Mutation Operators Conclusion Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015 P. Delgado-Prez UCASE (University of Cdiz) Advances in Mutation Testing Research for C++ TAROT:

694 views • 45 slides
Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis Department of Informatics Athens University of Economics and Business 1 Mutation Testing White-box, fault-based testing technique Considered as

540 views • 22 slides
Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15 2 Sunday, October 4, 15 3 Sunday, October 4, 15 Mutation Testing 4 Sunday, October 4, 15 What is mutation testing? Code under test + test

449 views • 32 slides
MAJOR: An Efficient and Extensible Tool for Mutation Analysis in a Java Compiler e Just 1 , Franz

MAJOR: An Efficient and Extensible Tool for Mutation Analysis in a Java Compiler e Just 1 , Franz

MAJOR: An Efficient and Extensible Tool for Mutation Analysis in a Java Compiler e Just 1 , Franz Schweiggert 1 , and Gregory M. Kapfhammer 2 Ren 1 Ulm University, Germany 2 Allegheny College, USA 26th International Conference on Automated

743 views • 32 slides
Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff, David R. White and John A. Clark. The 13th CREST Open Workshop Thursday 12 May 2011 Outline Invariants Using GP to find Invariants Identifying

417 views • 38 slides
Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common way to determine the quality of a test suite How good are my tests? is to measure its coverage - we ve already seen quite a number of

423 views • 38 slides
GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra, J. de Lara Modelling&amp;Software Engineering Research Group http://miso.es Universidad Autnoma de Madrid (Spain) Mercedes G. Merayo Design

350 views • 13 slides
The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

Software Testing Challenges Structural Testing Regression Testing Mutation Testing Future Work Conclusion The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer Department of Computer Science

578 views • 23 slides
Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi

Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi

Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi ciently deep ? 2 Test suite depth Mutation testing 3 Program Generate Mutants 4 Program Generate Mutants 5 Program Generate Mutants Mutant

653 views • 38 slides
Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa

Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa

Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa Misailovic 2 , August Shi 2 ICSE NIER 2017 Buenos Aires, Argentina May 24, 2017 1 2 CCF-1409423, CCF-1421503, CCF-1566363, CCF-1629431, CCF-1319688,

445 views • 31 slides
1 Mutation - selection equilibrium 1. Mutation pressure: ( ) ( ) = Let = the

1 Mutation - selection equilibrium 1. Mutation pressure: ( ) ( ) = Let = the

Population Genetics 7: transient verses equilibrium polymorphism Mutation - selection equilibrium Mutation pressure and selection can operate in opposite directions as a force for change in allele frequencies in populations. Note that

359 views • 9 slides
Thursday September 10, 2015 Started with OKIE811 in June 2013 Previous Work: COO for Girl

Thursday September 10, 2015 Started with OKIE811 in June 2013 Previous Work: COO for Girl

Thursday September 10, 2015 Started with OKIE811 in June 2013 Previous Work: COO for Girl Scouts Western Oklahoma - 4 years IT Director for State of Oklahoma / OKDHS 10 years Risk &amp; Information Manager for Fleming Companies, Inc.

656 views • 26 slides
Munitions Supply Chain Modelling: A Top-Down Approach Crispin Allard &amp; Louise Carver The

Munitions Supply Chain Modelling: A Top-Down Approach Crispin Allard &amp; Louise Carver The

Munitions Supply Chain Modelling: A Top-Down Approach Crispin Allard &amp; Louise Carver The Barbican, East Street, Farnham, Surrey GU9 7TB 01252 738500 www.Advantage-Business.co.uk Contents of Presentation 1. Project MASS 2.

496 views • 20 slides
Contents Double Data Rate Interfaces DDR SDRAM Architecture and Functionality DDR

Contents Double Data Rate Interfaces DDR SDRAM Architecture and Functionality DDR

D ESIGN AND I MPLEMENTAION OF A DDR SDRAM C ONTROLLER FOR S YSTEM ON C HIP Magnus Sjlander 2002-12-13 Contents Double Data Rate Interfaces DDR SDRAM Architecture and Functionality DDR Memory Controller Data Resynchronization

404 views • 21 slides
How to fix bugs automatically? Martin Monperrus University of Lille &amp; Inria Lille, France

How to fix bugs automatically? Martin Monperrus University of Lille &amp; Inria Lille, France

How to fix bugs automatically? Martin Monperrus University of Lille &amp; Inria Lille, France October 2014 Martin Monperrus, University of Lille &amp; Inria 1 www.gustonomie.fr Le Merveilleux, specialty of Lille Martin Monperrus,

161 views • 15 slides
Visualising Data and Dashboards By Simranjit Singh Deol Agenda/Objectives What are Dashboards?

Visualising Data and Dashboards By Simranjit Singh Deol Agenda/Objectives What are Dashboards?

Introduction to best design practices for Visualising Data and Dashboards By Simranjit Singh Deol Agenda/Objectives What are Dashboards? Why are they effective? Visualizing Basics Value-Based Design Dashboard Navigation Design and Layout

1.34k views • 111 slides
Site related Issues Andreas S. Schwarz XFEL Project Meeting, November 5 th , 2003 1. Site

Site related Issues Andreas S. Schwarz XFEL Project Meeting, November 5 th , 2003 1. Site

Site related Issues Andreas S. Schwarz XFEL Project Meeting, November 5 th , 2003 1. Site Decision 2. Plan Approval Procedure 3. Next Steps XFEL Project Meeting, November 5 th , 2003 1 Site related Issues A.S. Schwarz 1. Site Decision

308 views • 14 slides
Brihaspti-3 Project Yatindra Nath Singh IIT Kanpur http://home.iitk.ac.in/~ynsingh URLs

Brihaspti-3 Project Yatindra Nath Singh IIT Kanpur http://home.iitk.ac.in/~ynsingh URLs

Brihaspti-3 Project Yatindra Nath Singh IIT Kanpur http://home.iitk.ac.in/~ynsingh URLs http://brihaspati.nmeict.in/ http://brihsvn.iitk.ernet.in/repos http://sourceforge.net/projects/brihaspati

285 views • 12 slides
An Introduction to Omega SAT Swimming Victoria Saturday October 18 th 2014 Regional Development

An Introduction to Omega SAT Swimming Victoria Saturday October 18 th 2014 Regional Development

An Introduction to Omega SAT Swimming Victoria Saturday October 18 th 2014 Regional Development Day Presenter: Greg Smith Peter Krenz Leisure Centre Central Victoria Swimming Inc. Napier St, Eaglehawk An Introduction to Omega SAT The

433 views • 21 slides

More recommend