mutation testing of memory related operators
play

Mutation Testing of Memory- Related Operators Jay Nanavati, Fan Wu, - PowerPoint PPT Presentation

Mar 29, 2024 •312 likes •606 views

Mutation Testing of Memory- Related Operators Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL, UK Mutation Testing Test case Test case Test case Test case Test case Test case Test case Jay Nanavati, Fan Wu, Mark

  1. Mutation Testing of Memory- Related Operators Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL, UK

  2. Mutation Testing Test case Test case Test case Test case Test case Test case Test case Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  3. Mutation Testing Test case Test case Test case Test case Test case Test case Test case Mutants Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  4. Motivation Bug #68942 Use after free Submitted: 2015-01-29 07:20 UTC Reference: https://bugs.php.net/bug.php?id=68942 if (zend_hash_find(...) == SUCCESS) { if (zend_hash_find(...) == SUCCESS) { convert_to_long(*z_timezone_type); if (SUCCESS == timezone_initialize(...)) { return SUCCESS; } Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  5. Motivation Bug #68942 Use after free Submitted: 2015-01-29 07:20 UTC Reference: https://bugs.php.net/bug.php?id=68942 if (zend_hash_find(...) == SUCCESS) { if (zend_hash_find(...) == SUCCESS) { convert_to_long(*z_timezone_type); if (SUCCESS == timezone_initialize(...)) { return SUCCESS; } Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  6. Motivation - if (zend_hash_find(...) == SUCCESS) { + if (zend_hash_find(...) == SUCCESS && Z_TYPE_PP(z_timezone_type) == IS_LONG) { if (zend_hash_find(...) == SUCCESS) { - convert_to_long(*z_timezone_type); if (SUCCESS == timezone_initialize(...)) { return SUCCESS; } Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  7. Motivation Test case Test case Test case Test case Test case Test case Test case Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  8. Motivation Test case Test case Test case Test case Test case Test case Test case Mutants Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  9. Motivation Test case Test case Test case Test case Test case Test case Test case weakpoint Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  10. Memory Mutation Operators Uninitialized Memory Access Faulty Memory Allocation Faulty Heap Management Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  11. Memory Mutation Operators Uninitialized Memory Access Uninitialized memory REC2M calloc(k, sizeof(T)) malloc(k*sizeof(T)) Use-after-free RMNA str = NULL str Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  12. Memory Mutation Operators Faulty Memory Allocation Use-before-allocation REDAWN malloc(k*sizeof(T)) NULL Buffer overflow REDAWZ malloc(k*sizeof(T)) malloc(0) RESOTPE malloc(k*sizeof(T)) malloc(k*sizeof(T*)) REMSOTP malloc(k*sizeof(T*)) malloc(k*sizeof(T)) Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  13. Memory Mutation Operators Faulty Heap Management Memory leaks RMFS free(str) REM2A malloc(k*sizeof(T)) alloc(k*sizeof(T)) REC2A calloc(k, sizeof(T)) alloc(k*sizeof(T)) Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  14. Weakly Killing Criteria Memory Fault Detection Control Flow Deviation Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  15. Weakly Killing Criteria Memory Fault Detection (MFD)   t, MFD(M, t) MFD(P, t) Valgrind Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  16. Weakly Killing Criteria Control Flow Deviation (CFD) P M Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  17. Weakly Killing Criteria Control Flow Deviation (CFD) P M   t, CFD(M, t) CFD(P, t) Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  18. Research Questions RQ1 What are the characteristics of the proposed Memory Mutation Operators? RQ1a What is the prevalence of Memory Mutants? RQ1b How effective is each Memory Mutation Operator in inserting memory faults? RQ1c What is the Mutation Score for the Traditional criterion applied against the Memory Mutants? Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  19. Research Questions RQ2 What is the reduction rate of survived mutants after introducing Memory Fault Detection and Control Flow Deviation criteria? RQ3 What is the relation between MFD and CFD criteria? All Mutants MFD T CFD Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  20. Experiments No. Program LoC 1 PeerWireProtocol 1547 2 Craft 731 3 CfixedArraylist 497 4 ChashMapViaLinkedList 488 5 CAVLTree 405 6 CpseudoLRU 384 7 CHashMapViaQuadraticProbing 1097 8 CtextureAtlas 745 9 Csplaytree 834 10 CstreamingBencodeReader 371 11 CSparseCounter 328 12 Cheap 207 13 CcircularBuffer 118 14 ClinkedListQueue 200 15 CbipBuffer 118 16 Cbitfield 87 Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  21. Results (RQ1) RQ1a What is the prevalence of Memory Mutants? Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  22. Results (RQ1) RQ1b How effective is each Memory Mutation Operator in inserting memory faults? RQ1c What is the Mutation Score for the Traditional criterion applied against the Memory Mutants? Category Mutation Generated Survived Mutation Operator Mutants Mutants Score Uninitialized REC2M 30 25 0.167 Memory Access RMNA 39 21 0.462 Faulty Memory REDAWN 65 12 0.815 Allocation REDAWZ 63 35 0.444 RESOTPE 48 28 0.417 REMSOTP 5 5 0.000 Faulty Heap RMFS 53 53 0.000 Management REM2A 27 16 0.407 REC2A 29 6 0.793 All 359 201 0.440 Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  23. Results (RQ2) RQ2 What is the reduction rate of survived mutants after introducing Memory Fault Detection and Control Flow Deviation criteria? Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  24. Results (RQ3) RQ3 What is the relation between MFD and CFD criteria? All Mutants MFD T CFD     MFD T CFD CFD T MFD   c c MFD CFD   MFD  CFD T MFD  CFD T Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  25. Results (RQ3) RQ3 What is the relation between MFD and CFD criteria? All Mutants MFD T CFD Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  26. Conclusion & Extension Proposed Memory Mutation Operators Introduced MFD & CFD, reduced survived mutants Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  27. Conclusion & Extension Compare with traditional operators Extend the comparison between traditional strong killing criterion and MFD/CFD All Mutants MFD T CFD Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  28. All Mutants MFD T CFD Category Mutation Generated Survived Mutation Operator Mutants Mutants Score Uninitialized REC2M 30 25 0.167 Memory Access RMNA 39 21 0.462 Faulty Memory REDAWN 65 12 0.815 Allocation REDAWZ 63 35 0.444 RESOTPE 48 28 0.417 REMSOTP 5 5 0.000 Faulty Heap RMFS 53 53 0.000 Management REM2A 27 16 0.407 REC2A 29 6 0.793 All 359 201 0.440

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Introduction Mutation Operators Conclusion Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015 P. Delgado-Prez UCASE (University of Cdiz) Advances in Mutation Testing Research for C++ TAROT:

694 views • 45 slides
Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and

Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and

Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and Scalable Mutation Analysis Ren Just 1 , 2 & Gregory M. Kapfhammer 3 & Franz Schweiggert 2 1 University of Washington, USA 2 Ulm University,

1.18k views • 85 slides
Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with Mark Harman and William Langdon Agenda Why Higher Order Mutation Testing? Search for interesting HOMs Milu mutation testing tool Scalability and

669 views • 38 slides
Escaping Large Deceptive Basins of Attraction with Heavy-Tailed Mutation Operators Tobias

Escaping Large Deceptive Basins of Attraction with Heavy-Tailed Mutation Operators Tobias

Escaping Large Deceptive Basins of Attraction with Heavy-Tailed Mutation Operators Tobias Friedrich, Francesco Quinzan, Markus Wagner How to mutate? I mean: mutation rate, ? Many packages do this: if n is the length of a solution, then

654 views • 26 slides
Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings College, London W. B. Langdon, Crest 1 Introduction What is mutation testing 2 objectives: Hard to kill, little change to source Higher

417 views • 29 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

306 views • 7 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

350 views • 6 slides
Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis Department of Informatics Athens University of Economics and Business 1 Mutation Testing White-box, fault-based testing technique Considered as

540 views • 22 slides
Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15 2 Sunday, October 4, 15 3 Sunday, October 4, 15 Mutation Testing 4 Sunday, October 4, 15 What is mutation testing? Code under test + test

449 views • 32 slides
Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff, David R. White and John A. Clark. The 13th CREST Open Workshop Thursday 12 May 2011 Outline Invariants Using GP to find Invariants Identifying

417 views • 38 slides
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by

Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by

Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/ gkapfham/

2.33k views • 174 slides
Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common way to determine the quality of a test suite How good are my tests? is to measure its coverage - we ve already seen quite a number of

423 views • 38 slides
GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra, J. de Lara Modelling&Software Engineering Research Group http://miso.es Universidad Autnoma de Madrid (Spain) Mercedes G. Merayo Design

350 views • 13 slides
The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

Software Testing Challenges Structural Testing Regression Testing Mutation Testing Future Work Conclusion The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer Department of Computer Science

578 views • 23 slides
Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi

Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi

Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi ciently deep ? 2 Test suite depth Mutation testing 3 Program Generate Mutants 4 Program Generate Mutants 5 Program Generate Mutants Mutant

653 views • 38 slides
Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa

Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa

Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa Misailovic 2 , August Shi 2 ICSE NIER 2017 Buenos Aires, Argentina May 24, 2017 1 2 CCF-1409423, CCF-1421503, CCF-1566363, CCF-1629431, CCF-1319688,

445 views • 31 slides
The No Gap Conjecture: Proof by Pictures Stephen Hermes Wellesley College, Wellesley, MA

The No Gap Conjecture: Proof by Pictures Stephen Hermes Wellesley College, Wellesley, MA

The No Gap Conjecture: Proof by Pictures Stephen Hermes Wellesley College, Wellesley, MA Maurice Auslander Distinguished Lectures and International Conference Woods Hole May, 2016 Preliminaries Joint With Kiyoshi Igusa. arXiv:1601.04054

515 views • 35 slides
Thread synchronization David Hovemeyer 2 December 2019 David Hovemeyer Computer Systems

Thread synchronization David Hovemeyer 2 December 2019 David Hovemeyer Computer Systems

Thread synchronization David Hovemeyer 2 December 2019 David Hovemeyer Computer Systems Fundamentals: Thread synchronization 2 December 2019 A program 1 const int NUM_INCR=100000000, NTHREADS=2; typedef struct { volatile int count; }

658 views • 36 slides
Grap aphQL hQL forget (the) REST? Christian Schwendtner

Grap aphQL hQL forget (the) REST? Christian Schwendtner

Grap aphQL hQL forget (the) REST? Christian Schwendtner https://www.pexels.com/photo/man-person-people-emotions-1990/ https://www.pexels.com/photo/man-person-people-emotions-1990/ REST Feelings Im lovin it Well, What else?

482 views • 46 slides
Buffer Overflow overflows Defenses and other memory safety vulnerabilities Everything

Buffer Overflow overflows Defenses and other memory safety vulnerabilities Everything

This time We will continue By looking at Buffer Overflow overflows Defenses and other memory safety vulnerabilities Everything youve always wanted to know about gdb but were too afraid to ask Overflow defenses Other memory

1.23k views • 95 slides
Software Security: Defenses & Principles CS 161: Computer Security Prof. Vern Paxson TAs:

Software Security: Defenses & Principles CS 161: Computer Security Prof. Vern Paxson TAs:

Software Security: Defenses & Principles CS 161: Computer Security Prof. Vern Paxson TAs: Devdatta Akhawe, Mobin Javed & Matthias Vallentin http://inst.eecs.berkeley.edu/~cs161/ January 25, 2011 Testing for Software Security Issues

444 views • 42 slides
Using abstract interpretation to correct synchronization faults Pietro Ferrara Omer Tripp Peng

Using abstract interpretation to correct synchronization faults Pietro Ferrara Omer Tripp Peng

Using abstract interpretation to correct synchronization faults Pietro Ferrara Omer Tripp Peng Liu Eric Koskinen JuliaSoft Google IBM Research Yale University VMCAI 17 January 2017 Concurrency is here to stay. Thread 1 Thread 2

578 views • 46 slides
Drift, mutation, selection, and the evolutionary dispersion of mean phenotypes recombination

Drift, mutation, selection, and the evolutionary dispersion of mean phenotypes recombination

Drift, mutation, selection, and the evolutionary dispersion of mean phenotypes recombination random mutation genetic drift The Population-genetic Environment 1000x Range of Variation in the Mutation Rate The mutation rate per nucleotide

480 views • 27 slides
Evolutionary Testing for Crash Reproduction Mozhan Soltani Annibale Panichella Arie van Deursen

Evolutionary Testing for Crash Reproduction Mozhan Soltani Annibale Panichella Arie van Deursen

Evolutionary Testing for Crash Reproduction Mozhan Soltani Annibale Panichella Arie van Deursen Bugs are everywhere Cost of Bug Fixings Major Bug for Apache Commons Collections https://issues.apache.org/jira/browse/COLLECTIONS-70 Cost

653 views • 37 slides

More recommend