mrstudyr Retrospective Mutant Reduction Colton J. McCurdy - - PowerPoint PPT Presentation

▶
mrstudyr
SMART_READER_LITE
LIVE PREVIEW

mrstudyr Retrospective Mutant Reduction Colton J. McCurdy - - PowerPoint PPT Presentation

Jan 10, 2023 365 likes •964 views

mrstudyr Retrospective Mutant Reduction Colton J. McCurdy McCurdyColton ICSME 2016 Gregory M. Kapfhammer and Phil McMinn Motivation Quality software? Motivation Regression Test Suite Motivation Regression Test Suite T= t 1 , t 2 ,

slide-1
SLIDE 1

mrstudyr

Retrospective Mutant Reduction Colton J. McCurdy

McCurdyColton

ICSME 2016 Gregory M. Kapfhammer and Phil McMinn

slide-2
SLIDE 2

Motivation

Quality software?

slide-3
SLIDE 3

Motivation Regression Test Suite

slide-4
SLIDE 4

Motivation Regression Test Suite

T= t1, t2, . . . , tn

slide-5
SLIDE 5

Motivation

✓Pass ✗Fail

slide-6
SLIDE 6

Motivation

✓Pass ✗Fail

slide-7
SLIDE 7

Motivation

✓Pass ✗Fail

slide-8
SLIDE 8

Motivation

Use testing to assess quality?

slide-9
SLIDE 9

Motivation

Trivial test suite?

slide-10
SLIDE 10

Background

Mutation Testing!

slide-11
SLIDE 11

Mutation Testing

Original Mutant

slide-12
SLIDE 12

Mutation Testing

Original Mutant

slide-13
SLIDE 13

Mutation Testing

a > b

slide-14
SLIDE 14

Mutation Testing

a > b Relational Operator Replacement (ROR)

slide-15
SLIDE 15

Mutation Testing

a > b Relational Operator Replacement (ROR) a ≥ b a ≤ b a == b a != b

slide-16
SLIDE 16

Mutation Testing

✓Alive ✗Killed

slide-17
SLIDE 17

Mutation Testing

✓Alive ✗Killed

slide-18
SLIDE 18

Mutation Testing

✓Alive ✗Killed ✗Killed

slide-19
SLIDE 19

Mutation Testing

MST = Killed Total

slide-20
SLIDE 20

Mutation Testing

MST = Killed Total

slide-21
SLIDE 21

Mutation Testing

MST = Killed Total

slide-22
SLIDE 22

Mutation Testing

MST = Killed Total

MST ∈ [0, 1]

slide-23
SLIDE 23

Mutation Testing

MST = Killed Total

MST ∈ [0, 1]

HIB

slide-24
SLIDE 24

Mutation Testing

Silver Bullet?

slide-25
SLIDE 25

Mutation Testing

Limitations

slide-26
SLIDE 26

Mutation Testing

slide-27
SLIDE 27

Mutation Testing T= t1, t2, . . . , tn

slide-28
SLIDE 28

Mutation Testing T= t1, t2, . . . , tn

Weak

slide-29
SLIDE 29

Mutation Testing T= t1, t2, . . . , tn

Weak Strong

slide-30
SLIDE 30

Mutation Testing T= t1, t2, . . . , tn

Weak Strong Redundant

slide-31
SLIDE 31

Mutation Testing T= t1, t2, . . . , tn

Weak Strong Redundant

slide-32
SLIDE 32

Mutation Testing

Still . . .

slide-33
SLIDE 33

Mutation Testing

Still . . . limitations

slide-34
SLIDE 34

Mutation Testing

slide-35
SLIDE 35

Mutation Testing

Understand system intricacies

slide-36
SLIDE 36

Mutation Testing

Understand system intricacies

slide-37
SLIDE 37

Mutation Testing

Retrospective Mutant Reduction

slide-38
SLIDE 38

Mutation Testing

Retrospective Mutant Reduction

slide-39
SLIDE 39

mrstudyr ✗ ✓

slide-40
SLIDE 40

mrstudyr ✗ ✓

mccurdyc/mrstudyr

slide-41
SLIDE 41

mrstudyr ✗ ✓

mccurdyc/mrstudyr

Data

slide-42
SLIDE 42

mrstudyr ✗ ✓

mccurdyc/mrstudyr

Reduce Data

slide-43
SLIDE 43

mrstudyr ✗ ✓

mccurdyc/mrstudyr

Reduce Data Remove

slide-44
SLIDE 44

Input

Database Schema Mutants

slide-45
SLIDE 45

Demonstration

Demonstration!

slide-46
SLIDE 46

Motivation

Random Sampling

slide-47
SLIDE 47

Output

Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90 2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00

slide-48
SLIDE 48

Output

Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90 2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00

slide-49
SLIDE 49

Output

Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90 2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00

slide-50
SLIDE 50

Output

Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90 2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00

slide-51
SLIDE 51

Conclusion

Program Quality

slide-52
SLIDE 52

Conclusion

Program Quality Regression Test Suite

slide-53
SLIDE 53

Conclusion

Program Quality Regression Test Suite Mutation Testing

slide-54
SLIDE 54

Conclusion

Program Quality Regression Test Suite Mutation Testing Reduction Techniques

slide-55
SLIDE 55

Conclusion

Program Quality Regression Test Suite Mutation Testing Reduction Techniques mrstudyr

slide-56
SLIDE 56

Conclusion

Reduced

slide-57
SLIDE 57

Conclusion

Reduced

10%

slide-58
SLIDE 58

Conclusion

Reduced

10%