Advances in Mutation Testing Research for C++ Pedro Delgado-Prez - - PowerPoint PPT Presentation

▶
advances in mutation testing research for c
SMART_READER_LITE
LIVE PREVIEW

Advances in Mutation Testing Research for C++ Pedro Delgado-Prez - - PowerPoint PPT Presentation

Apr 11, 2024 233 likes •699 views

Introduction Mutation Operators Conclusion Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015 P. Delgado-Prez UCASE (University of Cdiz) Advances in Mutation Testing Research for C++ TAROT:

slide-1
SLIDE 1

Introduction Mutation Operators Conclusion

Advances in Mutation Testing Research for C++

Pedro Delgado-Pérez TAROT: Intro Talk June 2015

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 1 / 24

slide-2
SLIDE 2

Introduction Mutation Operators Conclusion

Index

1

Introduction Mutation testing Research line

2

Mutation Operators Class mutation operators Mutation operator implementation Correct mutations

3

Conclusion Conclusion

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 2 / 24

slide-3
SLIDE 3

Introduction Mutation Operators Conclusion

Index

1

Introduction Mutation testing Research line

2

Mutation Operators Class mutation operators Mutation operator implementation Correct mutations

3

Conclusion Conclusion

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 3 / 24

slide-4
SLIDE 4

Introduction Mutation Operators Conclusion

Mutation testing

A brief definition A fault injection testing technique. x > 1 → x < 1 Involves inserting simple syntactic changes in the program using mutation operators. Mutation operators are based on typical mistakes. This modification creates a new version called mutant. Goals

1

Measure how good is a test suite detecting faults affecting the program.

2

Improve the test suite through the results of the mutants.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 4 / 24

slide-5
SLIDE 5

Introduction Mutation Operators Conclusion

Mutation testing

A brief definition A fault injection testing technique. x > 1 → x < 1 Involves inserting simple syntactic changes in the program using mutation operators. Mutation operators are based on typical mistakes. This modification creates a new version called mutant. Goals

1

Measure how good is a test suite detecting faults affecting the program.

2

Improve the test suite through the results of the mutants.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 4 / 24

slide-6
SLIDE 6

Introduction Mutation Operators Conclusion

Mutation testing

Mutant classification

1

Dead: The output of the original program and the mutant is different.

2

Alive: The change has not been detected:

Equivalence: The change cannot be detected by any input. A new test case is needed to detect the change.

3

Invalid: The mutant does not comply with the grammar rules.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 5 / 24

slide-7
SLIDE 7

Introduction Mutation Operators Conclusion

Mutation testing

Mutant classification

1

Dead: The output of the original program and the mutant is different.

2

Alive: The change has not been detected:

Equivalence: The change cannot be detected by any input. A new test case is needed to detect the change.

3

Invalid: The mutant does not comply with the grammar rules.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 5 / 24

slide-8
SLIDE 8

Introduction Mutation Operators Conclusion

Index

1

Introduction Mutation testing Research line

2

Mutation Operators Class mutation operators Mutation operator implementation Correct mutations

3

Conclusion Conclusion

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 6 / 24

slide-9
SLIDE 9

Introduction Mutation Operators Conclusion

Mutation testing research

History First ideas in 1970s. Early years: around procedural languages → traditional operators From 1990s onwards: around other kind of languages and domains. Mutation tools developed Mothra - FORTRAN MuJava - Java GAmera - WS-BPEL ...

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 7 / 24

slide-10
SLIDE 10

Introduction Mutation Operators Conclusion

Mutation testing research

Motivation One of the most important programming languages → 4th position in TIOBE index. Research regarding C++ was pending. Obtaining results about the usefulness of this technique in C++. Possible reasons Complexity of the language. The technique to inject mutations in the code. Dependency analysis of source code files.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 8 / 24

slide-11
SLIDE 11

Introduction Mutation Operators Conclusion

Mutation testing research

Motivation One of the most important programming languages → 4th position in TIOBE index. Research regarding C++ was pending. Obtaining results about the usefulness of this technique in C++. Possible reasons Complexity of the language. The technique to inject mutations in the code. Dependency analysis of source code files.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 8 / 24

slide-12
SLIDE 12

Introduction Mutation Operators Conclusion

The C++ programming language

Achievements

1

State of the art [1].

2

Definition of a set of class-level mutation operators [2].

3

Implementation of class mutation operators. P . Delgado-Pérez, I. Medina-Bulo and J. J. Domínguez-Jiménez. Analysis of the development process of a mutation testing tool for the C++ language. In The Ninth International Multi-Conference on Computing in the Global Information Technology, ICCGI 2014. Seville, Spain, 2014. P . Delgado-Pérez, I. Medina-Bulo, J. Domínguez-Jiménez,

  • A. García-Domínguez and F. Palomo-Lozano.

Class mutation operators for C++ object-oriented systems. Annals of Telecommunications, April 2015. ISSN 0003-4347.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 9 / 24

slide-13
SLIDE 13

Introduction Mutation Operators Conclusion

Index

1

Introduction Mutation testing Research line

2

Mutation Operators Class mutation operators Mutation operator implementation Correct mutations

3

Conclusion Conclusion

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 10 / 24

slide-14
SLIDE 14

Introduction Mutation Operators Conclusion

Categories

Summary Definition of 37 operators at the class level. Operators grouped into 7 categories. Adapted and new operators.

1

Access control

2

Inheritance

3

Polymorphism and dynamic binding

4

Method overloading

5

Exception handling

6

Object and member replacement

7

Miscellany

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 11 / 24

slide-15
SLIDE 15

Introduction Mutation Operators Conclusion

Categories

Summary Definition of 37 operators at the class level. Operators grouped into 7 categories. Adapted and new operators.

1

Access control

2

Inheritance

3

Polymorphism and dynamic binding

4

Method overloading

5

Exception handling

6

Object and member replacement

7

Miscellany

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 11 / 24

slide-16
SLIDE 16

Introduction Mutation Operators Conclusion

Example

Example “Inheritance” block: IOD (Overriding method deletion)

Original: class A { class B: public A{ ... ... ... ... int method(){... ...}; int method(){... ...}; }; }; Mutant: class A { class B: public A{ ... ... ... ... int method(){... ...}; /*IOD*/ }; };

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 12 / 24

slide-17
SLIDE 17

Introduction Mutation Operators Conclusion

Example

Example “Inheritance” block: IOD (Overriding method deletion)

Original: class A { class B: public A{ ... ... ... ... int method(){... ...}; int method(){... ...}; }; }; Mutant: class A { class B: public A{ ... ... ... ... int method(){... ...}; /*IOD*/ }; };

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 12 / 24

slide-18
SLIDE 18

Introduction Mutation Operators Conclusion

Example

Example “Inheritance” block: IOD (Overriding method deletion)

Original: class A { class B: public A{ ... ... ... ... int method(){... ...}; int method(){... ...}; }; }; Mutant: class A { class B: public A{ ... ... ... ... int method(){... ...}; /*IOD*/ }; };

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 12 / 24

slide-19
SLIDE 19

Introduction Mutation Operators Conclusion

Example

Example “Polymorphism” block: PVI (virtual modifier insertion)

Original: class A { class B: public A{ ... ... ... ... int method(){... ...}; int method(){... ...}; }; }; Mutant: class A { ... ... virtual int method(){... ...}; };

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 13 / 24

slide-20
SLIDE 20

Introduction Mutation Operators Conclusion

Example

Example “Polymorphism” block: PVI (virtual modifier insertion)

Original: class A { class B: public A{ ... ... ... ... int method(){... ...}; int method(){... ...}; }; }; Mutant: class A { ... ... virtual int method(){... ...}; };

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 13 / 24

slide-21
SLIDE 21

Introduction Mutation Operators Conclusion

Index

1

Introduction Mutation testing Research line

2

Mutation Operators Class mutation operators Mutation operator implementation Correct mutations

3

Conclusion Conclusion

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 14 / 24

slide-22
SLIDE 22

Introduction Mutation Operators Conclusion

Mutation operator implementation

AST transformations Abstract Syntax Tree (AST): Simplified structure of the code. Language elements are represented with different kind of nodes. Traversal of the AST through pattern matching. Useful to determine the mutation locations and transform the code.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 15 / 24

slide-23
SLIDE 23

Introduction Mutation Operators Conclusion

Mutation operator implementation

Steps

1

Creation of the pattern using the DSL in Clang.

2

The source code is converted to the form of AST.

3

AST is traversed searching for every mutation target.

4

The nodes retrieved are analyzed, ensuring that the injection of a fault is possible at that point.

5

Depending on the nature of the operator, one or more variants can be introduced in each location.

6

The mutation is inserted.

7

The source code containing the mutant is saved.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 16 / 24

slide-24
SLIDE 24

Introduction Mutation Operators Conclusion

Mutation operator implementation

Steps

1

Creation of the pattern using the DSL in Clang.

2

The source code is converted to the form of AST.

3

AST is traversed searching for every mutation target.

4

The nodes retrieved are analyzed, ensuring that the injection of a fault is possible at that point.

5

Depending on the nature of the operator, one or more variants can be introduced in each location.

6

The mutation is inserted.

7

The source code containing the mutant is saved.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 17 / 24

slide-25
SLIDE 25

Introduction Mutation Operators Conclusion

Mutation operator implementation

Steps

1

Creation of the pattern using the DSL in Clang.

2

The source code is converted to the form of AST.

3

AST is traversed searching for every mutation target.

4

The nodes retrieved are analyzed, ensuring that the injection of a fault is possible at that point.

5

Depending on the nature of the operator, one or more variants can be introduced in each location.

6

The mutation is inserted.

7

The source code containing the mutant is saved.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 17 / 24

slide-26
SLIDE 26

Introduction Mutation Operators Conclusion

Mutation operator implementation

Steps

1

Creation of the pattern using the DSL in Clang.

2

The source code is converted to the form of AST.

3

AST is traversed searching for every mutation target.

4

The nodes retrieved are analyzed, ensuring that the injection of a fault is possible at that point.

5

Depending on the nature of the operator, one or more variants can be introduced in each location.

6

The mutation is inserted.

7

The source code containing the mutant is saved.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 17 / 24

slide-27
SLIDE 27

Introduction Mutation Operators Conclusion

Mutation operator implementation

Steps

1

Creation of the pattern using the DSL in Clang.

2

The source code is converted to the form of AST.

3

AST is traversed searching for every mutation target.

4

The nodes retrieved are analyzed, ensuring that the injection of a fault is possible at that point.

5

Depending on the nature of the operator, one or more variants can be introduced in each location.

6

The mutation is inserted.

7

The source code containing the mutant is saved.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 17 / 24

slide-28
SLIDE 28

Introduction Mutation Operators Conclusion

Mutation operator implementation

Steps

1

Creation of the pattern using the DSL in Clang.

2

The source code is converted to the form of AST.

3

AST is traversed searching for every mutation target.

4

The nodes retrieved are analyzed, ensuring that the injection of a fault is possible at that point.

5

Depending on the nature of the operator, one or more variants can be introduced in each location.

6

The mutation is inserted.

7

The source code containing the mutant is saved.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 17 / 24

slide-29
SLIDE 29

Introduction Mutation Operators Conclusion

Mutation operator implementation

Steps

1

Creation of the pattern using the DSL in Clang.

2

The source code is converted to the form of AST.

3

AST is traversed searching for every mutation target.

4

The nodes retrieved are analyzed, ensuring that the injection of a fault is possible at that point.

5

Depending on the nature of the operator, one or more variants can be introduced in each location.

6

The mutation is inserted.

7

The source code containing the mutant is saved.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 17 / 24

slide-30
SLIDE 30

Introduction Mutation Operators Conclusion

Mutation operator implementation

Steps

1

Creation of the pattern using the DSL in Clang.

2

The source code is converted to the form of AST.

3

AST is traversed searching for every mutation target.

4

The nodes retrieved are analyzed, ensuring that the injection of a fault is possible at that point.

5

Depending on the nature of the operator, one or more variants can be introduced in each location.

6

The mutation is inserted.

7

The source code containing the mutant is saved.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 17 / 24

slide-31
SLIDE 31

Introduction Mutation Operators Conclusion

Index

1

Introduction Mutation testing Research line

2

Mutation Operators Class mutation operators Mutation operator implementation Correct mutations

3

Conclusion Conclusion

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 18 / 24

slide-32
SLIDE 32

Introduction Mutation Operators Conclusion

Can we insert the mutation?

1

Check appropriate conditions for the mutation.

2

Reduce unproductive mutations. Example “Inheritance” block: IOD (Overriding method deletion)

Original: class A { class B: public A{ ... ... ... ... int method(){... ...}; int method(){... ...} }; };

Goals Prevent errors in the syntax of the code. Avoid noise and silence in the pattern matching.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 19 / 24

slide-33
SLIDE 33

Introduction Mutation Operators Conclusion

Can we insert the mutation?

1

Check appropriate conditions for the mutation.

2

Reduce unproductive mutations. Example “Inheritance” block: IOD (Overriding method deletion)

Original: class A { class B: public A{ ... ... ... ... int method(){... ...}; int method(){... ...} }; };

Goals Prevent errors in the syntax of the code. Avoid noise and silence in the pattern matching.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 19 / 24

slide-34
SLIDE 34

Introduction Mutation Operators Conclusion

Can we insert the mutation?

1

Check appropriate conditions for the mutation.

2

Reduce unproductive mutations. Example “Inheritance” block: IOD (Overriding method deletion)

Original: class A { class B: public A{ ... ... ... ... int method(){... ...}; int method(){... ...} }; };

Goals Prevent errors in the syntax of the code. Avoid noise and silence in the pattern matching.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 19 / 24

slide-35
SLIDE 35

Introduction Mutation Operators Conclusion

Is the mutation valuable?

1

Check appropriate conditions for the mutation.

2

Reduce unproductive mutations. Unproductive mutants Those mutants which do not help the purpose of mutation testing. Mutants:

1

Invalid mutants.

2

Equivalent mutants

3

Trivial mutants.

Detect situations always producing an unproductive mutant.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 20 / 24

slide-36
SLIDE 36

Introduction Mutation Operators Conclusion

Is the mutation valuable?

1

Check appropriate conditions for the mutation.

2

Reduce unproductive mutations. Unproductive mutants Those mutants which do not help the purpose of mutation testing. Mutants:

1

Invalid mutants.

2

Equivalent mutants

3

Trivial mutants.

Detect situations always producing an unproductive mutant.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 20 / 24

slide-37
SLIDE 37

Introduction Mutation Operators Conclusion

Is the mutation valuable?

1

Check appropriate conditions for the mutation.

2

Reduce unproductive mutations.

Original: class A { class B: public A{ ... ... ... ... virtual int method() = 0; int method(){... ...}; }; }; Mutant: class A { class B: public A{ ... ... ... ... virtual int method() = 0; /*IOD*/ }; };

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 21 / 24

slide-38
SLIDE 38

Introduction Mutation Operators Conclusion

Is the mutation valuable?

1

Check appropriate conditions for the mutation.

2

Reduce unproductive mutations.

Original: class A { class B: public A{ ... ... ... ... virtual int method() = 0; int method(){... ...}; }; }; Mutant: class A { class B: public A{ ... ... ... ... virtual int method() = 0; /*IOD*/ }; };

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 21 / 24

slide-39
SLIDE 39

Introduction Mutation Operators Conclusion

Index

1

Introduction Mutation testing Research line

2

Mutation Operators Class mutation operators Mutation operator implementation Correct mutations

3

Conclusion Conclusion

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 22 / 24

slide-40
SLIDE 40

Introduction Mutation Operators Conclusion

Conclusion

Summary Goal: apply mutation testing to C++. First step: definition of 37 class mutation operators. Second step: automation of mutation operators. Third step: evaluation and improvement of operators. Future work Measure quality metrics. Improve the mutation tool:

Test coverage. New standards of the language.

Evolutionary Mutation Testing. Contribution of class operators.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 23 / 24

slide-41
SLIDE 41

Introduction Mutation Operators Conclusion

Conclusion

Summary Goal: apply mutation testing to C++. First step: definition of 37 class mutation operators. Second step: automation of mutation operators. Third step: evaluation and improvement of operators. Future work Measure quality metrics. Improve the mutation tool:

Test coverage. New standards of the language.

Evolutionary Mutation Testing. Contribution of class operators.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 23 / 24

slide-42
SLIDE 42

Introduction Mutation Operators Conclusion

Conclusion

Summary Goal: apply mutation testing to C++. First step: definition of 37 class mutation operators. Second step: automation of mutation operators. Third step: evaluation and improvement of operators. Future work Measure quality metrics. Improve the mutation tool:

Test coverage. New standards of the language.

Evolutionary Mutation Testing. Contribution of class operators.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 23 / 24

slide-43
SLIDE 43

Introduction Mutation Operators Conclusion

Conclusion

Summary Goal: apply mutation testing to C++. First step: definition of 37 class mutation operators. Second step: automation of mutation operators. Third step: evaluation and improvement of operators. Future work Measure quality metrics. Improve the mutation tool:

Test coverage. New standards of the language.

Evolutionary Mutation Testing. Contribution of class operators.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 23 / 24

slide-44
SLIDE 44

Introduction Mutation Operators Conclusion

Conclusion

Summary Goal: apply mutation testing to C++. First step: definition of 37 class mutation operators. Second step: automation of mutation operators. Third step: evaluation and improvement of operators. Future work Measure quality metrics. Improve the mutation tool:

Test coverage. New standards of the language.

Evolutionary Mutation Testing. Contribution of class operators.

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 23 / 24

slide-45
SLIDE 45

Introduction Mutation Operators Conclusion

Thank you for your attention

TAROT-2015

Pedro Delgado-Pérez pedro.delgado@uca.es

  • P. Delgado-Pérez

UCASE (University of Cádiz) Advances in Mutation Testing Research for C++ TAROT: Intro Talk 24 / 24