mutation testing at saab
play

Mutation testing at SAAB Niklas Pettersson - PowerPoint PPT Presentation

May 07, 2023 •255 likes •743 views

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 1 Hkan Anderwall | LN-030676| Issue 1 Hkan Anderwall | LN-030676| Issue

  1. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 1 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Mutation testing at SAAB Niklas Pettersson niklas.pettersson2@saabgroup.com Joakim Brännström joakim.k.brannstrom@saabgroup.com

  2. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 2 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Agenda • What is SAAB? (~25 min) ‒ Some PR-videos, our products and more. ‒ Safety-critical software. • Why use mutation testing? (~20 min) ‒ Practical examples. ‒ Problem with code coverage. ‒ What is mutation testing? • Pitfalls of Mutation testing (~30 min) ‒ From academic to industry. • Short tool introduction (~10 min) ‒ Dextool Mutate (will be used in the laboration). • Questions and wrap-up (~ 5min)

  3. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 3 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 In 1937 we took off 6:07

  5. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 5 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 FAA RTCA/DO-178C Provides detailed guidelines for the production of software for airborne systems: - Objectives for the life cycle processes. - Activities and design considerations for achieving those objectives. - Descriptions of the evidence indicating the objectives have been satisfied.

  6. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 6 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Responsibility - Saab has a certificate issued by the Military Aviation Safety Inspectorate that allows us to have an organization to design and build military aircrafts. - Saab is responsible for the safety of the aircraft, including the software. - Other companies are not trusted to be responsible for the safety (e.g. external authority signs the software development plan). - In case of an accident, Saab will be reviewed: - Saab must be able to show that we did enough to avoid the failure. - Saab will be measured against the best practices of the aviation industry. RTCA/DO-178B/C is best practice for software.

  7. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 7 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Design Assurance Levels (DAL)

  8. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 8 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Design Assurance Levels - Examples

  9. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 9 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 How can we effectively test software? ? Program Tests ● How effective are ● What to test? the tests? ● How to test? ● What is missing? ● When to stop? Source: G. Petrovic, M Ivanokovic, et. al.

  10. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 10 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Coverage metrics Many types may be interesting: ● Coverage of requirements ● Coverage of functions ● Coverage of use cases ● Coverage of code structure ○ Function coverage ○ Statement coverage ○ Branch coverage ○ Condition coverage ○ MCDC

  11. RTCA/DO-178C: Table A-7 - Verification of Verification Process Results Applicability by Software Level Objective A B C D 1 Test procedures are correct 2 Test results are correct and discrepancies explained. 3 Test coverage of high-level requirements is achieved. 4 Test coverage of low-level requirements is achieved 5 Test coverage of software structure (modified condition/decision coverage) is achieved 6 Test coverage of software structure (decision coverage) is achieved. 7 Test coverage of software structure (statement coverage) is achieved. 8 Test coverage of software structure (data coupling and control coupling) is achieved The objective should be satisfied with independence. The objective should be satisfied. Blank Satisfaction of the objective is at application’s discretion.

  12. RTCA/DO-178C: Table A-7 - Verification of Verification Process Results Applicability by Software Level Objective A B C D 1 Test procedures are correct 2 Test results are correct and discrepancies explained. 3 Test coverage of high-level requirements is achieved. 4 Test coverage of low-level requirements is achieved 5 Test coverage of software structure (modified condition/decision coverage) is achieved 6 Test coverage of software structure (decision coverage) is achieved. 7 Test coverage of software structure (statement coverage) is achieved. 8 Test coverage of software structure (data coupling and control coupling) is achieved The objective should be satisfied with independence. The objective should be satisfied. Blank Satisfaction of the objective is at application’s discretion.

  13. RTCA/DO-178C: Figure 6-1 - Software Testing Process Software Requirements-Based Test Generation Hardware/ Software Low-Level Software Integration Tests Integration Tests Tests Software Requirements-Based Test Coverage Analysis Software Structural Coverage Analysis Additional Direct Path Verification End of Testing Conditional Path

  14. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 14 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Working with code coverage

  15. Practical example

  16. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 16 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Practical example • Small program

  17. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 17 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Practical example • Small program • Test suite

  18. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 18 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Practical example • Small program • Test suite • Test result

  19. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 19 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Practical example • Small program • Test suite • Test result • Are we testing everything?

  20. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 20 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Practical example • Small program • Test suite • Test result • Are we testing everything?

  21. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 21 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Practical example • Small program • Test suite • Test result • Are we testing everything? ‒ Obviously not …

  22. COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIED OPEN | NOT EXPORT CONTROLLED | NOT CLASSIFIED 22 Håkan Anderwall | LN-030676| Issue 1 Håkan Anderwall | LN-030676| Issue 1 Niklas Pettersson | Issue 1 Practical example • Small program • Test suite • Test result • Are we testing everything? ‒ Obviously not … • Add test case

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

FAR & DFARS compliance projects Saab Dynamics and Saab Aeronautics Fredrik Genbo

FAR & DFARS compliance projects Saab Dynamics and Saab Aeronautics Fredrik Genbo

This document and the information contained herein is the property of Saab AB and must not be used, disclosed or altered without Saab AB prior written consent. FAR & DFARS compliance projects Saab Dynamics and Saab Aeronautics Fredrik

432 views • 9 slides
Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with Mark Harman and William Langdon Agenda Why Higher Order Mutation Testing? Search for interesting HOMs Milu mutation testing tool Scalability and

669 views • 38 slides
Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings College, London W. B. Langdon, Crest 1 Introduction What is mutation testing 2 objectives: Hard to kill, little change to source Higher

417 views • 29 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

306 views • 7 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

350 views • 6 slides
Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Introduction Mutation Operators Conclusion Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015 P. Delgado-Prez UCASE (University of Cdiz) Advances in Mutation Testing Research for C++ TAROT:

694 views • 45 slides
Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis Department of Informatics Athens University of Economics and Business 1 Mutation Testing White-box, fault-based testing technique Considered as

540 views • 22 slides
SAAB ONEFACT CONFERENCE Wilhelm Isoz, Saab Aeronautics This document and the information

SAAB ONEFACT CONFERENCE Wilhelm Isoz, Saab Aeronautics This document and the information

COMPANY UNCLASSIFIED | NOT EXPORT CONTROLLED | NOT CLASSIFIED 1 Wilhelm Isoz, OTTDHA-m | Issue 1 SAAB ONEFACT CONFERENCE Wilhelm Isoz, Saab Aeronautics This document and the information contained herein is the property of Saab AB and must

598 views • 24 slides
SAAB PRESENTATION TO DEBT INVESTORS 19 SEPTEMBER 2017 Magnus rnberg, EVP and CFO Saab AB 2

SAAB PRESENTATION TO DEBT INVESTORS 19 SEPTEMBER 2017 Magnus rnberg, EVP and CFO Saab AB 2

SAAB PRESENTATION TO DEBT INVESTORS 19 SEPTEMBER 2017 Magnus rnberg, EVP and CFO Saab AB 2 AGENDA 1. Transaction overview 2. Overview of Saab 3. Market & Q2 financial update Sales of SEK 30,130m LTM Q2 2017 Total assets

485 views • 29 slides
Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15 2 Sunday, October 4, 15 3 Sunday, October 4, 15 Mutation Testing 4 Sunday, October 4, 15 What is mutation testing? Code under test + test

449 views • 32 slides
Interoperability by design IUC The success story Hans Lindgren Saab Training & Simulation

Interoperability by design IUC The success story Hans Lindgren Saab Training & Simulation

This document and the information contained herein is the property of Saab AB and must not be used, disclosed or altered without Saab AB prior written consent. Interoperability by design IUC The success story Hans Lindgren Saab Training &

431 views • 17 slides
Saab experience with application of composites in aerospace structures Pontus Nordin, Saab

Saab experience with application of composites in aerospace structures Pontus Nordin, Saab

Saab experience with application of composites in aerospace structures Pontus Nordin, Saab Aeronautics 2011-09-05 ICAS Biennial workshop 2011, Stockholm Advanced Materials and Manufacturing - Certification and Operational Challenges New,

619 views • 28 slides
Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff, David R. White and John A. Clark. The 13th CREST Open Workshop Thursday 12 May 2011 Outline Invariants Using GP to find Invariants Identifying

417 views • 38 slides
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by

Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by

Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/ gkapfham/

2.33k views • 174 slides
Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common way to determine the quality of a test suite How good are my tests? is to measure its coverage - we ve already seen quite a number of

423 views • 38 slides
GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra, J. de Lara Modelling&Software Engineering Research Group http://miso.es Universidad Autnoma de Madrid (Spain) Mercedes G. Merayo Design

350 views • 13 slides
Hierarchical Assembly of Stellar Envelopes in Galaxy Clusters Meng Gu (Harvard University)

Hierarchical Assembly of Stellar Envelopes in Galaxy Clusters Meng Gu (Harvard University)

Hierarchical Assembly of Stellar Envelopes in Galaxy Clusters Meng Gu (Harvard University) Collaborators: Charlie Conroy, Gabriel Brammer, David Law, Pieter van Dokkum, Renbin Yan, David Wake, Alexa Villaume, Allison Merritt, Roberto Abraham,

762 views • 19 slides
Lessons learned from planning and preparing a distributed ISR LVC Environment - and there were

Lessons learned from planning and preparing a distributed ISR LVC Environment - and there were

UNCLASSIFIED Lessons learned from planning and preparing a distributed ISR LVC Environment - and there were lots John W. Diem Test Technology Director, US Army Operational Test Command ITEA Systems of Systems Engineering Workshop January

442 views • 15 slides
THE RIVA METER AND PLATFORM 3 rd International Workshop on Non-Intrusive Load Monitoring NILM2016,

THE RIVA METER AND PLATFORM 3 rd International Workshop on Non-Intrusive Load Monitoring NILM2016,

THE RIVA METER AND PLATFORM 3 rd International Workshop on Non-Intrusive Load Monitoring NILM2016, May 14-15, 2016 Robert Sonderegger, Director and Engineering Advisor, Itron, Inc. OpenWay Riva v2.0 Solution Overview March 2015, Itron Inc.,

334 views • 17 slides
BRCA Founder Mutations Ashkenazi Jewish (Hungarian and Russian): BRCA1 - 185delAG and

BRCA Founder Mutations Ashkenazi Jewish (Hungarian and Russian): BRCA1 - 185delAG and

Development of a Founder Mutation Development of a Founder Mutation A high frequency of a specific gene mutation in a A high frequency of a specific gene mutation in a population founded by a small ancestral group population founded by a small

278 views • 8 slides
Simultaneous Identification of Multiple Driver Pathways in Cancer Mark D. M. Leiserson, et.al

Simultaneous Identification of Multiple Driver Pathways in Cancer Mark D. M. Leiserson, et.al

Simultaneous Identification of Multiple Driver Pathways in Cancer Mark D. M. Leiserson, et.al Goal To distinguish the functional driver mutations responsible for cancer development from the random passenger mutations that have no

537 views • 22 slides
Mutation Analysis for Coq Pengyu Nie 1 Ahmet Celik 1 , Karl Palmskog 1 , Marinela Parovic 1 , us

Mutation Analysis for Coq Pengyu Nie 1 Ahmet Celik 1 , Karl Palmskog 1 , Marinela Parovic 1 , us

Mutation Analysis for Coq Pengyu Nie 1 Ahmet Celik 1 , Karl Palmskog 1 , Marinela Parovic 1 , us Gallego Arias 2 , and Milos Gligoric 1 Emilio Jes ASE 2019 1 2 1 / 24 Program Verification Using Proof Assistants Verified software: encode

682 views • 42 slides
Maximal green sequences of minimal mutation-infinite quivers John Lawson joint with Matthew

Maximal green sequences of minimal mutation-infinite quivers John Lawson joint with Matthew

Maximal green sequences of minimal mutation-infinite quivers John Lawson joint with Matthew Mills Durham University Oct 2016 Spoilers Theorem. All minimal mutation-infinite quivers have a maximal green sequence. Theorem. Any cluster algebra

584 views • 32 slides
Accepted Point Mutation (Dayhoff et al. 68,72,78) An APM in a protein is a replacement of

Accepted Point Mutation (Dayhoff et al. 68,72,78) An APM in a protein is a replacement of

1 Accepted Point Mutation (Dayhoff et al. 68,72,78) An APM in a protein is a replacement of one AA by another accepted by evolution We want to estimate the probability that given a site with AA A has udergone an APM, the new AA

513 views • 8 slides

More recommend