how high will it be using machine learning models to
play

How High Will It Be? Using Machine Learning Models to Predict - PowerPoint PPT Presentation

Oct 26, 2022 •102 likes •333 views

How High Will It Be? Using Machine Learning Models to Predict Branch Coverage in Automated Testing G. Grano, T. Titov, S. Panichella, H. Gall MaLTeSQuE@SANER 2018, 20, Campobasso (Italy) grano@ifi.uzh.ch giograno90 U.S. Economy Impact 1

  1. How High Will It Be? Using Machine Learning Models to Predict Branch Coverage in Automated Testing G. Grano, T. Titov, S. Panichella, H. Gall MaLTeSQuE@SANER 2018, 20, Campobasso (Italy) grano@ifi.uzh.ch giograno90

  2. U.S. Economy Impact 1 $59.5 billion $22.2 billion avoidable 0.6% GDP 1 http://www.abeacha.com/NIST_press_release_bugs_cost.html 2 — Giovanni Grano @ s.e.a.l.

  3. Continuous Integration The way we develop and release code is rapidly changing In an ideal world, at every single commit the entire test suite should be executed ... ... but companies like Google commit about 16.000 changes per day! 3 — Giovanni Grano @ s.e.a.l.

  4. Test Data Generation Pretty active research area in the last years Mature tools able to generate test suites with high coverage: > EvoSuite > Randoop > ... > and many more! How do they fit in such a CI/CD environment? 4 — Giovanni Grano @ s.e.a.l.

  5. Continuous Testing Generation 2 Continuous integration enhanced with automated test generation It raises many questions: > testing order > how much time to spend per class 2 Campos et al - Continuous Test Generation: Enhancing Continuous Integration with Automated Test Generation 5 — Giovanni Grano @ s.e.a.l.

  6. Test Suite Augmentatio n 3 Automatic generation considering code changes and their effect on the previous codebase Hardy doable to the expensive amount of time needed to generate tests 3 Xu et al - Directed Test Suite Augmentation: Techniques and Tradeoffs 6 — Giovanni Grano @ s.e.a.l.

  7. Coverage Prediction Knowing a priori the coverage achieved by test data generation tools > maximize the coverage for the entire system given an amount of time > budget allocation for critical components 7 — Giovanni Grano @ s.e.a.l.

  8. Research Questions

  9. > RQ1: Which type of features can we leverage to train machine learning models to predict the branch coverage achieved by test data generation tools? > RQ2: To what extend can we predict the coverage achieved by test data generation tools? 9 — Giovanni Grano @ s.e.a.l.

  10. Project Selection Open Source Projects from Defect4j > Apache Cassandra > Apache Ivy > Google Guava > Google Dagger Guava Cassandr Dagger Ivy a LOC 78,525 220,573 848 50,430 Java 538 1,474 43 464 Files 10 — Giovanni Grano @ s.e.a.l.

  11. Training Set > Run EvoSuite > Labeled data (with coverage) > Computed metrics = input variables Coverage Threats: Metrics Extractor > we did it once 11 — Giovanni Grano @ s.e.a.l.

  12. Feature Selection Goal: capture complexity

  13. Package Level Features Computed with JDepend 4 Name Description Ca indicator of the package's responsibility Ce indicator of the package's independence A abstract classes / total number of classes I indicator of the package's resilience to change ... ... 4 https://github.com/clarkware/jdepend 13 — Giovanni Grano @ s.e.a.l.

  14. CK and OO Feature ck tool provided by Aniche 5 Name Description CBO coupling between objects DIT depth of inheritance tree NOC number of children NOSF number of static field ... ... 5 https://github.com/mauricioaniche/ck 14 — Giovanni Grano @ s.e.a.l.

  15. Java Reserved Keywords To capture additional complexity Previously used in Information Retrieval as a feature 6 52 Java reserved keywords 6 Sanderson et al The history of information retrieval research 15 — Giovanni Grano @ s.e.a.l.

  16. Grid Selection best hyper-parameters 3-cross fold validation Feature Transformation z-score 16 — Giovanni Grano @ s.e.a.l.

  17. Algorithms Huber Regression Support Vector Regression Multi-Layer Perception 17 — Giovanni Grano @ s.e.a.l.

  18. Results RQ1 Huber SVR MLP Tool's Average EvoSuit 0.255 0.216 0.242 0.238 e Randoop 0.172 0.088 0.139 0.131 Average 0.213 0.152 0.191 18 — Giovanni Grano @ s.e.a.l.

  19. Results RQ2 Time Math Lang Tool's Aver. EvoSuit 0.255 0.330 0.289 0.291 e Randoo 0.168 0.262 0.246 0.225 p Aver. 0.211 0.296 0.267 19 — Giovanni Grano @ s.e.a.l.

  20. Conclusion Knowing a priori the coverage achieved by test data generation tools might ease important decisions; > we took the first steps, investigating well known features > well know features (from gut feeling) give reasonable results 20 — Giovanni Grano @ s.e.a.l.

  21. Future work Improvements > larger dataset > branch-level features > feature analysis > differences between employed tools 21 — Giovanni Grano @ s.e.a.l.

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead Cynthia Rudin, Duke University Presenters : Sreya Dutta Roy, Ziqian Lin 1 Overview Introduction Explainable ML Vs

1.04k views • 39 slides
Deploying Machine Learning Models on The Edge Deploying Machine Learning Models on The Edge Yan

Deploying Machine Learning Models on The Edge Deploying Machine Learning Models on The Edge Yan

Deploying Machine Learning Models on The Edge Deploying Machine Learning Models on The Edge Yan Zhang, Mathew Salvaris Microsoft https://github.com/microsoft/deploy-MLmodels-on-iotedge Cloud Analytics Edge Analytics Device/Sensor Analytics

373 views • 34 slides
The role of models@run.time in self-explanation in the era of Machine Learning Antonio Garcia,

The role of models@run.time in self-explanation in the era of Machine Learning Antonio Garcia,

The role of models@run.time in self-explanation in the era of Machine Learning Antonio Garcia, Juan Marcelo Parra-Ullauri and Nelly Bencomo 14th International Workshop on Models@run.time September 17th, 2019 Introduction Machine learning is

616 views • 34 slides
Good Machine Learning = Generalization Goal of machine learning: build models that generalize

Good Machine Learning = Generalization Goal of machine learning: build models that generalize

Good Machine Learning = Generalization Goal of machine learning: build models that generalize well to predicting new data Overfitting: fitting the training data too well, so we lose generality of model o Example: linear regression vs.

120 views • 11 slides
Auditing Machine Learning Models for Individual Bias and Unfairness Songkai Xue Department of

Auditing Machine Learning Models for Individual Bias and Unfairness Songkai Xue Department of

Auditing Machine Learning Models for Individual Bias and Unfairness Songkai Xue Department of Statistics, University of Michigan Joint work with Mikhail Yurochkin and Yuekai Sun Introduction High-stakes decision making involves Recidivism

478 views • 30 slides
Moven Machine/Deep Learning Models Distribution Relying on the Maven Infrastructure Sergio

Moven Machine/Deep Learning Models Distribution Relying on the Maven Infrastructure Sergio

Moven Machine/Deep Learning Models Distribution Relying on the Maven Infrastructure Sergio Fernndez (Redlink GmbH) November 14th, 2016 - Sevilla SSIX aims to exploit the predictive power of Social Media on Financial Markets High-Level

724 views • 13 slides
Efficient Algorithms for Learning Sparse Models from High Dimensional Data Yoram Singer Google

Efficient Algorithms for Learning Sparse Models from High Dimensional Data Yoram Singer Google

Efficient Algorithms for Learning Sparse Models from High Dimensional Data Yoram Singer Google Research Machine Learning Summer School, UC Santa Cruz, July 18, 2012 1 Sparsity ? THE HIGHER MINIMUM WAGE THAT WAS SIGNED INTO LAW ... WILL BE

855 views • 74 slides
CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map

CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map

CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map Machine Learning Approaches Supervised Semi-supervised Unsupervised Reinforcement Learning Learning Learning Learning Classification Regression

1.06k views • 57 slides
Learning in Graphical Models Andrea Passerini passerini@disi.unitn.it Machine Learning Learning

Learning in Graphical Models Andrea Passerini passerini@disi.unitn.it Machine Learning Learning

Learning in Graphical Models Andrea Passerini passerini@disi.unitn.it Machine Learning Learning in Graphical Models Learning graphical models Parameter estimation We assume the structure of the model is given We are given a dataset of

268 views • 23 slides
Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning:

Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning:

Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning: instances, concepts, and hypotheses Specific learners Learning Hypothesis/ Labeled algorithm Model h Decision trees data New example

519 views • 41 slides
Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University April 5, 2011 Today: Readings: Kernels: Bishop Ch. 6.1 Latent Dirichlet Allocation topic models optional: Social network analysis

298 views • 13 slides
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 18, 2015 Today: Readings: Bishop chapter 8, through 8.2 Graphical models Bayes Nets: Representing distributions

852 views • 31 slides
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University October 13, 2011 Today: Readings: Required: Graphical models Bishop chapter 8, through 8.2 Bayes Nets: Conditional

149 views • 10 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Dhruv Batra Virginia Tech What is this class about? Some of the most exciting developments in Machine Learning, AI, Statistics &

925 views • 61 slides
Logistic regression on Sonar Machine Learning Toolbox Classification models Categorical

Logistic regression on Sonar Machine Learning Toolbox Classification models Categorical

MACHINE LEARNING TOOLBOX Logistic regression on Sonar Machine Learning Toolbox Classification models Categorical (i.e. qualitative) target variable Example: will a loan default? Still a form of supervised learning Use a

315 views • 27 slides
Stealing Machine Learning Models via Prediction APIs Florian Tramr, Fan Zhang, Ari Juels,

Stealing Machine Learning Models via Prediction APIs Florian Tramr, Fan Zhang, Ari Juels,

Stealing Machine Learning Models via Prediction APIs Florian Tramr, Fan Zhang, Ari Juels, Michael K. Reiter, Thomas Ristenpart Usenix Security Symposium Austin, Texas, USA August, 11 th 2016 Machine Learning (ML) Systems (1) Gather labeled

213 views • 17 slides
Microservices Are your Frameworks ready? Martin Eigenbrodt | martin.eigenbrodt@innoq.com

Microservices Are your Frameworks ready? Martin Eigenbrodt | martin.eigenbrodt@innoq.com

Microservices Are your Frameworks ready? Martin Eigenbrodt | martin.eigenbrodt@innoq.com Alexander Heusingfeld | alexander.heusingfeld@innoq.com Microservices? Levels of Architecture Domain architecture Domain architecture Macro (technical)

1.18k views • 97 slides
You, me and jigsaw Tom Schindl <tom.schindl@bestsolution.at> Twitter: @tomsontom Blog:

You, me and jigsaw Tom Schindl <tom.schindl@bestsolution.at> Twitter: @tomsontom Blog:

You, me and jigsaw Tom Schindl <tom.schindl@bestsolution.at> Twitter: @tomsontom Blog: http://tomsondev.bestsolution.at Website: http://www.bestsolution.at About Tom CTO BestSolution.at Systemhaus GmbH Eclipse Committer e4

935 views • 59 slides
Generating Readable Unit Tests for Guava Ermira Daka, Jos Campos, and Gordon Fraser University

Generating Readable Unit Tests for Guava Ermira Daka, Jos Campos, and Gordon Fraser University

Generating Readable Unit Tests for Guava Ermira Daka, Jos Campos, and Gordon Fraser University of Sheffield Jonathan Dorn and Westley Weimer University of Virginia ESEC/FSE 2015 [Developers] read tests [] 77% of the total time they

568 views • 20 slides
Web media initiative and media support at Columbia University Brian O'Hagan October 8, 2010 3rd

Web media initiative and media support at Columbia University Brian O'Hagan October 8, 2010 3rd

Web media initiative and media support at Columbia University Brian O'Hagan October 8, 2010 3rd TF-Media Task Force meeting Digital media production support Podcasting & Course-casting Course-related video platform Columbia on iTunes U,

481 views • 25 slides
Project Jigsaw_ Florian Trobach whoami codecentric Karlsruhe Plain Old Java Dev

Project Jigsaw_ Florian Trobach whoami codecentric Karlsruhe Plain Old Java Dev

Project Jigsaw_ Florian Trobach whoami codecentric Karlsruhe Plain Old Java Dev Currently trying to tame the SMACK stack 2 Services 3 Modules can provide services java.util.ServiceLoader 4 Modules can provide services

349 views • 22 slides
Selective Monitoring Radu Grigore Stefan Kiefer Concur 2018 Beijing, 4 September 2018 Radu

Selective Monitoring Radu Grigore Stefan Kiefer Concur 2018 Beijing, 4 September 2018 Radu

Selective Monitoring Radu Grigore Stefan Kiefer Concur 2018 Beijing, 4 September 2018 Radu Grigore, Stefan Kiefer Selective Monitoring 1 Labelled Markov Chains and DFAs 1 1 2 a 2 a s 1 s 0 s 2 1 1 1 1 2 a 2 c 2 a 2 b b q 0 f a , c a ,

531 views • 23 slides
Announcements/Follow-ups P7 is posted, due Friday August 2 at 11pm No late period You

Announcements/Follow-ups P7 is posted, due Friday August 2 at 11pm No late period You

Announcements/Follow-ups P7 is posted, due Friday August 2 at 11pm No late period You can choose either FishPond or YearGoogol Secret Tests Follow-ups Bidirectional example fix: remove guava from build- path

577 views • 15 slides
Writing Datomic in Clojure Rich Hickey Overview What is Datomic? Architecture

Writing Datomic in Clojure Rich Hickey Overview What is Datomic? Architecture

Writing Datomic in Clojure Rich Hickey Overview What is Datomic? Architecture Implementation - Clojure Applied Summary What is Datomic? A database A sound model of information, with time Provides database as a value

778 views • 57 slides

More recommend