applying computational learning theory to software testing
play

Applying Computational Learning Theory to Software Testing Neil - PowerPoint PPT Presentation

Dec 13, 2022 •1.08k likes •1.14k views

Applying Computational Learning Theory to Software Testing Neil Walkinshaw Computational Learning Theory (CLT) Theoretical analysis of questions of learnability . Is it feasible to learn a given type of concept from a set of examples?

  1. Applying Computational Learning Theory to Software Testing Neil Walkinshaw

  2. Computational Learning Theory (CLT) • Theoretical analysis of questions of learnability . • Is it feasible to learn a given type of concept from a set of examples? • What are the performance bounds? Time complexity? Gold, E. Mark. "Language • Several theoretical learning frameworks to identification in the limit." Information and control 10.5 (1967) characterise a learning problem. • Identification In the Limit (IIL) Mitchell, Tom M. "Generalization as search." Artificial intelligence(1982) • Version Spaces • Probably Approximately Correct (PAC) learning Valiant, Leslie G. "A theory of the learnable." Communications of the ACM (1984) • Vapnik-Chervonenkis Theory • … Vapnik, Vladimir Naumovich, and Vlamimir Vapnik. Statistical learning theory, Wiley, 1998.

  3. Machine Learning Insights from CLT • Numerous positive / negative learnability results. Gold, E. Mark. "Language identification in the limit." Information and control 10.5 (1967): • Finite languages, supra-finite languages from positive 447-474. data, regular grammars with a helpful teacher, Angluin, Dana. "Queries and conjunctive concepts (in PAC), disjunctions of concept learning." Machine learning 2.4 (1988): 319-342. conjunctive concepts (in PAC), … Valiant, Leslie G. "A theory of the learnable." Communications of the • The number of required examples can be ACM 27.11 (1984): 1134-1142. bounded explicitly … Haussler, David. "Quantifying • … in PAC if we can bound the size of the inductive bias: AI learning Version Space or … algorithms and Valiant's learning framework." Artificial intelligence 36.2 (1988): 177-221. • … in PAC if we can bound the size of the Vapnik Chervonenkis dimension … Blumer, Anselm, et al. "Learnability and the Vapnik-Chervonenkis dimension." Journal of the ACM • Derived by close examination of relationship (JACM) 36.4 (1989): 929-965. between learning algorithm and subject system.

  4. Applications to Testing? Weyuker "Assessing test data adequacy through program inference." ACM TOPLAS (1983) • Several theoretical frameworks that link testing and ML Budd, and Angluin. "Two notions of correctness and their relation to • Learnability ➔ Testability testing." Acta Informatica 18.1 (1982): 31-45. • Bounds on training set size ➔ Bounds on test set size. Walkinshaw, "Assessing test adequacy for black-box systems without specifications." ICTSS • Current results can rarely simply be carried over. (2011) • Software representations are necessarily complex. Romanik, "Approximate testing and its relationship to learning." • Rarely valid to represent software as (e.g.) a FSM. Theoretical Computer Science 188.1 (1997): 79-99. • CLT results assume that learning is `distribution free’ - i.e. apply to any distribution over the training sample Romanik and Vitter (1996). Using Vapnik–Chervonenkis Dimension to • Reasoning wrt. distributions of test inputs is Analyze the Testing Complexity of Program Segments. Information notoriously problematic. and Computation, 128(2), 87-108. Hamlet, R. (1994). Random testing. Goal: Improved theoretical frameworks to reason Encyclopedia of software about learnability (and testability) of generic Engineering. software systems

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization

411 views • 28 slides
The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

Software Testing Challenges Structural Testing Regression Testing Mutation Testing Future Work Conclusion The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer Department of Computer Science

578 views • 23 slides
Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides

Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides

Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others 1 This lecture: Computational Learning Theory The Theory of Generalization

845 views • 49 slides
Computational Learning Theory: Positive and negative learnability results Machine Learning 1

Computational Learning Theory: Positive and negative learnability results Machine Learning 1

Computational Learning Theory: Positive and negative learnability results Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization Probably

534 views • 40 slides
Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

653 views • 44 slides
Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Checkpoint: The bigger picture Supervised learning: instances, concepts, and

895 views • 42 slides
Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on

Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on

Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

1.09k views • 81 slides
Software Testing Software testing 1 V model Software testing 2 Program testing goals To

Software Testing Software testing 1 V model Software testing 2 Program testing goals To

Software Testing Software testing 1 V model Software testing 2 Program testing goals To demonstrate to the developer and the customer that the software meets its requirements. => leads to validation testing To discover situations

264 views • 11 slides
S V V .lu software verification & validation Achieving Scalability in Software Testing

S V V .lu software verification & validation Achieving Scalability in Software Testing

S V V .lu software verification & validation Achieving Scalability in Software Testing with Machine Learning and Metaheuristic Search Lionel Briand Definition of Software Testing ISTQB: Software testing is a process of executing

925 views • 77 slides
Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from

Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from

Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

585 views • 35 slides
Collective Annotation: Applying Voting Theory to Computational Linguistics Ulle Endriss

Collective Annotation: Applying Voting Theory to Computational Linguistics Ulle Endriss

Collective Annotation FNWI Student Colloquium 2015 Collective Annotation: Applying Voting Theory to Computational Linguistics Ulle Endriss Institute for Logic, Language and Computation University of Amsterdam joint work with Raquel

461 views • 7 slides
Computational Learning Theory For which tasks is successful learning possible? Under what

Computational Learning Theory For which tasks is successful learning possible? Under what

Computational Learning Theory For which tasks is successful learning possible? Under what conditions is successful learning guaranteed? What is successful learning? Probably approximately correct (PAC) framework Bounds on number

422 views • 27 slides
Correctness? Principles of Software Construction: Learning Goals Objects, Design, and

Correctness? Principles of Software Construction: Learning Goals Objects, Design, and

Correctness? Principles of Software Construction: Learning Goals Objects, Design, and Concurrency Integrating unit testing into the development (Part 2: Designing (Sub-)Systems) process Understanding and applying coverage metrics More

293 views • 8 slides
CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom

CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom

CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom Mitchells slides) September 8, 2003 1 Introduction Combines machine learning with: Algorithm design and analysis Computational complexity

393 views • 27 slides
Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch.

Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch.

0. Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 7 Acknowledgement: The present slides are an adaptation of slides drawn by T. Mitchell 1. Main Questions in Computational Learning Theory

682 views • 42 slides
Computational Learning Theory 1 / 22 Decidability Computation Decidability which

Computational Learning Theory 1 / 22 Decidability Computation Decidability which

Computational Learning Theory 1 / 22 Decidability Computation Decidability which problems have algorithmic solutions Machine Learning Feasibility what assumptions must we make to trust that we can learn an unknown target

246 views • 22 slides
Testing Convexity Properties of Tree Colorings Eldar Fischer and Orly Yahalom Technion IIT,

Testing Convexity Properties of Tree Colorings Eldar Fischer and Orly Yahalom Technion IIT,

STACS 2007, RWTH Aachen University, Germany Testing Convexity Properties of Tree Colorings Eldar Fischer and Orly Yahalom Technion IIT, Haifa, Israel Convex colorings A tree coloring is convex convex convex convex if it induces

729 views • 48 slides
Generative Adversarial Networks Stefano Ermon, Aditya Grover Stanford University Lecture 9

Generative Adversarial Networks Stefano Ermon, Aditya Grover Stanford University Lecture 9

Generative Adversarial Networks Stefano Ermon, Aditya Grover Stanford University Lecture 9 Stefano Ermon, Aditya Grover (AI Lab) Deep Generative Models Lecture 9 1 / 22 Recap Model families Autoregressive Models: p ( x ) = n i =1 p

248 views • 22 slides
Layer 2 VPN(L2VPN) Service Model (L2SM) Interim meetjng Wednesday 27th September 2017

Layer 2 VPN(L2VPN) Service Model (L2SM) Interim meetjng Wednesday 27th September 2017

Layer 2 VPN(L2VPN) Service Model (L2SM) Interim meetjng Wednesday 27th September 2017 2pm-3:30pm London Time Webex: htups://ietg.webex.com/ietg/j.php?MTID=m269230aaa9b65847700af143006c1a55 Adrian Farrel (adrian@olddog.co.uk ) Qin WU

943 views • 22 slides
Computer Vision Dr. Danna Gurari September 8, 2015 1 Todays Outline Aligning two

Computer Vision Dr. Danna Gurari September 8, 2015 1 Todays Outline Aligning two

9/6/2015 Computer Vision Dr. Danna Gurari September 8, 2015 1 Todays Outline Aligning two images Chamfer distance Applications Analyze binary images Thresholding Morphological operators Connected

796 views • 32 slides
WoLA19: Open Problems July, 2019 Abstract Some questions suggested during the Open Problems

WoLA19: Open Problems July, 2019 Abstract Some questions suggested during the Open Problems

WoLA19: Open Problems July, 2019 Abstract Some questions suggested during the Open Problems session of the 3 rd Workshop on Local Algorithms (WoLA), held in July 2019 at ETH, Zurich. Non-Adaptive Group Testing Suggested by Oliver Gebhard. In

55 views • 4 slides
Sparse multiple testing: can one estimate the null distribution? Etienne Roquain 1 Joint work with

Sparse multiple testing: can one estimate the null distribution? Etienne Roquain 1 Joint work with

Sparse multiple testing: can one estimate the null distribution? Etienne Roquain 1 Joint work with A. Carpentier 2 , S. Delattre 3 , N. Verzelen 4 , 1 LPSM, Sorbonne Universit, France 2 Otto-von-Guericke-Universitt Magdeburg, Allemagne 3 LPSM,

911 views • 61 slides
Inspections for Decision Makers (or: you may fool me, but not hurt me) Federico Echenique and

Inspections for Decision Makers (or: you may fool me, but not hurt me) Federico Echenique and

Inspections for Decision Makers (or: you may fool me, but not hurt me) Federico Echenique and Eran Shmaya California Institute of Technology November 10, 2007 SISHOO Nov. 10th Inspections for decision makers Existing result: Inspections are

616 views • 23 slides
Which sample is more diverse? RNA Sequencing Study 700 600 Number of Unique Sequences 500 400

Which sample is more diverse? RNA Sequencing Study 700 600 Number of Unique Sequences 500 400

Diversity Indices Usually used as biodiversity (as in species), but can apply to other data where you count the number of times a particular category occurs (e.g. nest shape). Comprises two components Species Richness: the number of

312 views • 10 slides

More recommend