a systematic study of automated program repair fixing 55
play

A SYSTEMATIC STUDY OF AUTOMATED PROGRAM REPAIR: FIXING 55 OUT OF - PowerPoint PPT Presentation

Feb 09, 2023 •17 likes •647 views

A SYSTEMATIC STUDY OF AUTOMATED PROGRAM REPAIR: FIXING 55 OUT OF 105 BUGS FOR $8 EACH Claire Michael Stephanie Westley Le Goues Dewey-Vogt Forrest Weimer 1 http://genprog.cs.virginia.edu Everyday, almost 300 Annual cost of bugs

  1. A SYSTEMATIC STUDY OF AUTOMATED PROGRAM REPAIR: FIXING 55 OUT OF 105 BUGS FOR $8 EACH Claire Michael Stephanie Westley Le Goues Dewey-Vogt Forrest Weimer 1 http://genprog.cs.virginia.edu

  2. “Everyday, almost 300 Annual cost of bugs appear […] far too many for only the Mozilla software errors in the programmers to handle.” US: $59.5 billion – Mozilla Developer, (0.6% of GDP). 2005 PROBLEM: BUGGY SOFTWARE 10%: Everything Else Average time to fix a security-critical error: 28 days. 90%: Maintenance 2 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  3. HOW BAD IS IT? 3 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  4. 4 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  5. 5 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  6. …REALLY? Tarsnap: 125 spelling/style 63 harmless 11 minor + 1 major 75/200 = 38% TP rate $17 + 40 hours per TP 6 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  7. …REALLY? Tarsnap: 125 spelling/style 63 harmless 11 minor + 1 major 75/200 = 38% TP rate $17 + 40 hours per TP 7 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  8. …REALLY? 8 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  9. SOLUTION: PAY STRANGERS 9 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  10. SOLUTION: PAY STRANGERS 10 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  11. SOLUTION: AUTOMATE 11 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  12. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC 1 , SCALABLE, COMPETITIVE BUG REPAIR. 1 C. Le Goues, T . Nguyen, S. Forrest, and W. Weimer, “GenProg: A generic method for automated software repair,” Transactions on Software Engineering, vol. 38, no. 1, pp. 54– 72, 2012. W. Weimer, T . Nguyen, C. Le Goues, and S. Forrest, “Automatically finding patches using genetic programming,” in International Conference on Software Engineering, 2009, pp. 364–367. 12 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  13. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC 1 , SCALABLE, COMPETITIVE BUG REPAIR. 1 C. Le Goues, T . Nguyen, S. Forrest, and W. Weimer, “GenProg: A generic method for automated software repair,” Transactions on Software Engineering, vol. 38, no. 1, pp. 54– 72, 2012. W. Weimer, T . Nguyen, C. Le Goues, and S. Forrest, “Automatically finding patches using genetic programming,” in International Conference on Software Engineering, 2009, pp. 364–367. 13 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  14. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC, SCALABLE, COMPETITIVE BUG REPAIR. 14 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  15. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC, SCALABLE, COMPETITIVE BUG REPAIR. 15 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  16. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC, SCALABLE, COMPETITIVE BUG REPAIR. 16 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  17. INPUT EVALUATE FITNESS DISCARD ACCEPT OUTPUT MUTATE Claire Le Goues, ICSE 2012

  18. INPUT EVALUATE FITNESS DISCARD ACCEPT OUTPUT MUTATE Claire Le Goues, ICSE 2012

  19. BIRD’S EYE VIEW Search: random (GP) search through nearby patches. Approach: compose small random edits. • Where to change? • How to change it? 19 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  20. Input: 1 2 3 4 5 6 7 8 9 10 11 12 20 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  21. Input: 1 2 3 4 5 6 7 Legend: � High change probability. 8 9 10 � Low change probability. � Not changed. 11 12 21 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  22. 1 2 3 4 5 6 7 An edit is: • Replace statement X with statement Y 8 9 10 • Insert statement X after statement Y • Delete statement X 11 12 22 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  23. 1 2 3 4 5 6 7 An edit is: • Replace statement X with statement Y 8 9 10 • Insert statement X after statement Y • Delete statement X 11 12 23 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  24. 1 2 3 4 5 6 7 An edit is: • Replace statement X with statement Y 8 9 10 • Insert statement X after statement Y • Delete statement X 11 12 24 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  25. 1 2 4 3 5 6 7 An edit is: • Replace statement X with statement Y 9 10 8 • Insert statement X after statement Y • Delete statement X 11 12 25 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  26. 1 2 4 3 4 5 6 7 An edit is: • Replace statement X with statement Y 9 10 8 • Insert statement X after statement Y • Delete statement X 11 12 26 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  27. 1 2 4 3 4 5 6 7 An edit is: • Replace statement X with statement Y 9 10 8 • Insert statement X after statement Y • Delete statement X 11 12 27 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  28. 1 2 4 3 4 5 6 7 An edit is: • Replace statement X with statement Y 9 10 4’ • Insert statement X after statement Y • Delete statement X 11 12 28 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  29. 1 2 4 3 4 5 6 7 An edit is: • Replace statement X with statement Y 9 10 4’ • Insert statement X after statement Y • Delete statement X 11 12 29 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  30. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC, SCALABLE, COMPETITIVE BUG REPAIR. 30 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  31. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC, SCALABLE, COMPETITIVE BUG REPAIR. 31 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  32. SCALABLE: SEARCH SPACE 1 2 4 3 5 6 7 8 9 10 11 12 32 32 http://genprog.cs.virginia.edu http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  33. SCALABLE: SEARCH SPACE 1 2 4 3 5 6 7 8 9 10 11 12 33 33 http://genprog.cs.virginia.edu http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  34. SCALABLE: SEARCH SPACE 1 2 4 3 5 6 7 8 9 10 11 12 34 34 http://genprog.cs.virginia.edu http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  35. SCALABLE: SEARCH SPACE 1 2 4 3 Fix localization: 5 6 7 intelligently choose code to 8 9 10 move. 11 12 35 35 http://genprog.cs.virginia.edu http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  36. SCALABLE: REPRESENTATION Naïve: New: 1 Input: 2 Delete(3) 1 4 5 2 3 1 4 5 2 5’ Replace(3,5) 4 5 36 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  37. SCALABLE: REPRESENTATION Naïve: New: 1 Input: New fitness, crossover, and 2 Delete(3) mutation operators to work with 1 a variable-length genome. 4 5 2 3 1 4 5 2 5’ Replace(3,5) 4 5 37 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  38. SCALABLE: PARALLELISM Fitness: • Subsample test cases. • Evaluate in parallel. Random runs: • Multiple simultaneous runs on different seeds. 38 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  39. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC, SCALABLE, COMPETITIVE BUG REPAIR. 39 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  40. AUTOMATED PROGRAM REPAIR GENPROG: AUTOMATIC, SCALABLE, COMPETITIVE BUG REPAIR. 40 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  41. How many bugs can GenProg fix? COMPETITIVE How much does it cost? 41 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  42. SETUP Goal: systematically test GenProg on a general, indicative bug set. General approach: • Avoid overfitting: fix the algorithm. • Systematically create a generalizable benchmark set. • Try to repair every bug in the benchmark set, establish grounded cost measurements. 42 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  43. SETUP Goal: systematically evaluate GenProg on a general, indicative bug set. General approach: • Avoid overfitting: fix the algorithm. • Systematically create a generalizable benchmark set. • Try to repair every bug in the benchmark set, establish grounded cost measurements. 43 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  44. CHALLENGE: INDICATIVE BUG SET 44 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  45. SYSTEMATIC BENCHMARK SELECTION Goal: a large set of important, reproducible bugs in non-trivial programs. Approach: use historical data to approximate discovery and repair of bugs in the wild. 45 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

  46. SYSTEMATIC BENCHMARK SELECTION Consider top programs from SourceForge, Google Code, Fedora SRPM, etc: • Find pairs of viable versions where test case behavior changes. • Take all tests from most recent version. • Go back in time through the source control. Corresponds to a human-written repair for the bug tested by the failing test case(s). 46 http://genprog.cs.virginia.edu Claire Le Goues, ICSE 2012

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Design of Repair Operators for Automated Program Repair Shin Hwei Tan National University of

Design of Repair Operators for Automated Program Repair Shin Hwei Tan National University of

Design of Repair Operators for Automated Program Repair Shin Hwei Tan National University of Singapore What is automated program repair? BUG! Given a failing Test T , buggy program P 1.Fault localization Where to fix? 2. Patch Generation

517 views • 22 slides
Do Automated Program Repair Techniques Repair Hard and Important Bugs? Manish Motwani Sandhya

Do Automated Program Repair Techniques Repair Hard and Important Bugs? Manish Motwani Sandhya

Do Automated Program Repair Techniques Repair Hard and Important Bugs? Manish Motwani Sandhya Sankarnarayanan Ren e Just Yuriy Brun University of Massachusetts Amherst Automatic Program Repair: An Active Research Area patched program

525 views • 37 slides
Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me...

Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me...

Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me... Research Interests: Postdoc Carnegie Mellon University, USA Automated Program Repair w/ Claire Le Goues Fault Localisation GI for

676 views • 39 slides
Graph-based, Self-Supervised Program Repair from Diagnostic Feedback ICML 2020 Michihiro

Graph-based, Self-Supervised Program Repair from Diagnostic Feedback ICML 2020 Michihiro

Graph-based, Self-Supervised Program Repair from Diagnostic Feedback ICML 2020 Michihiro Yasunaga, Percy Liang Stanford University Why program repair? Programmers spend 75% of time fixing source code errors Automatic program repair can

1.69k views • 130 slides
Advances in Automated Program Repair and a Call to Arms Westley Weimer University of Virginia

Advances in Automated Program Repair and a Call to Arms Westley Weimer University of Virginia

Advances in Automated Program Repair and a Call to Arms Westley Weimer University of Virginia Andreas Zeller, SSBSE Keynote 2011 Westley Weimer 2 For The Next Hour Automated Program Repair Historical Context Mistakes

1.16k views • 87 slides
Measuring and improving quality of automated program repair

Measuring and improving quality of automated program repair

Measuring and improving quality of automated program repair Yuriy Brun, UMass Amherst Ted Smith

651 views • 22 slides
Leveraging Program Equivalence for Adaptive Program Repair: Models and First Results Westley

Leveraging Program Equivalence for Adaptive Program Repair: Models and First Results Westley

Leveraging Program Equivalence for Adaptive Program Repair: Models and First Results Westley Weimer, UVA Zachary P . Fry, UVA Stephanie Forrest, UNM Automated Program Repair Given a program, a notion of correct behavior, and evidence

679 views • 30 slides
Enhancing Automated Program Repair With Deductive Verification Xuan Bach D. Le 1 , Quang-Loc Le 2

Enhancing Automated Program Repair With Deductive Verification Xuan Bach D. Le 1 , Quang-Loc Le 2

Enhancing Automated Program Repair With Deductive Verification Xuan Bach D. Le 1 , Quang-Loc Le 2 , David Lo 1 , Claire Le Goues 3 1 Singapore Management University 2 Singapore University of Technology and Design 3 Carnegie Mellon University 1

368 views • 20 slides
SFH Program 504 Home Repair Program: This program provides loans to very low income homeowners

SFH Program 504 Home Repair Program: This program provides loans to very low income homeowners

Presented by: Jeanie Barbrow USDA Rural Development 504 Repair Loans and Grants SFH Program 504 Home Repair Program: This program provides loans to very low income homeowners to repair, improve, or modernize their homes or provide grants to

357 views • 32 slides
Automated Fixing of Programs with Contracts Yi Wei, Yu Pei, Carlo A. Furia, Lucas S. Silva,

Automated Fixing of Programs with Contracts Yi Wei, Yu Pei, Carlo A. Furia, Lucas S. Silva,

Automated Fixing of Programs with Contracts Yi Wei, Yu Pei, Carlo A. Furia, Lucas S. Silva, Stefan Buchholz, Bertrand Meyer, Andreas Zeller Presented by Christine Zeller Motivation Programming is not just about writing code Find errors

207 views • 19 slides
Toward Automated Info-Flow Integrity Verification (or, Fixing your security policy) Umesh

Toward Automated Info-Flow Integrity Verification (or, Fixing your security policy) Umesh

Toward Automated Info-Flow Integrity Verification (or, Fixing your security policy) Umesh Shankar (UC Berkeley) Trent Jaeger (Penn State / IBM) Reiner Sailer (IBM) The goal, with an example Integrity property: Trusted processes dont

317 views • 20 slides
Automated Concurrency-Bug Fixing Guoliang Jin, Wei Zhang, Dongdong Deng, Den Libit, Shan Lu (OSDI

Automated Concurrency-Bug Fixing Guoliang Jin, Wei Zhang, Dongdong Deng, Den Libit, Shan Lu (OSDI

Automated Concurrency-Bug Fixing Guoliang Jin, Wei Zhang, Dongdong Deng, Den Libit, Shan Lu (OSDI 2012) Presented by: Kevin Burns October 2, 2013 Outline Paper Presentation Background/Motivation CFix Overview Bug Understanding Fix Strategy

347 views • 24 slides
Neighborhood Sidewalk Repair Program Botanic Gardens August 11, 2018 Neighborhood Sidewalk

Neighborhood Sidewalk Repair Program Botanic Gardens August 11, 2018 Neighborhood Sidewalk

Neighborhood Sidewalk Repair Program Botanic Gardens August 11, 2018 Neighborhood Sidewalk Repair Program Overview Program is addressing sidewalks, citywide, that are damaged, uneven, or sloping excessively, Authorizing lower-cost repair

429 views • 14 slides
A Revisit of the Integration of Metamorphic Testing and Test Suite Based Automated Program

A Revisit of the Integration of Metamorphic Testing and Test Suite Based Automated Program

A Revisit of the Integration of Metamorphic Testing and Test Suite Based Automated Program Repair Mingyue Jiang 1,2, Tsong Yueh Chen 2, Fei-Ching Kuo 2, Zuohua Ding 1, Eun-Hye Choi 3, Osamu Mizuno 4 1 Zhejiang Sci-Tech University,

465 views • 19 slides
Home Repair Program North Philadelphia & West Philadelphia Repair Program Who is Habitat

Home Repair Program North Philadelphia & West Philadelphia Repair Program Who is Habitat

Home Repair Program North Philadelphia & West Philadelphia Repair Program Who is Habitat for Humanity Philadelphia? Housing nonprofit organization that works in partnership with eligible Working in Philadelphia Philadelphia

368 views • 15 slides
Does Automated Refactoring Obviate Systematic Editing? Na Meng*

Does Automated Refactoring Obviate Systematic Editing? Na Meng*

Does Automated Refactoring Obviate Systematic Editing? Na Meng* Lisa Hua* Miryung Kim + Kathryn S. McKinley The University of Texas at

340 views • 19 slides
Randomization methods Tamuno Alfred, PhD Biostatistician DataCamp Designing and Analyzing

Randomization methods Tamuno Alfred, PhD Biostatistician DataCamp Designing and Analyzing

DataCamp Designing and Analyzing Clinical Trials in R DESIGNING AND ANALYZING CLINICAL TRIALS IN R Randomization methods Tamuno Alfred, PhD Biostatistician DataCamp Designing and Analyzing Clinical Trials in R DataCamp Designing and

687 views • 50 slides
trial Jim Bolognese www.cytel.com Email: bolognese@cytel.com OUTLINE Overview of Adaptive

trial Jim Bolognese www.cytel.com Email: bolognese@cytel.com OUTLINE Overview of Adaptive

Shaping the Future of Drug Development Adaptive Clinical Trials Overview Focus: Ph2a PoC+Dose-finding trial Jim Bolognese www.cytel.com Email: bolognese@cytel.com OUTLINE Overview of Adaptive Design Example Adaptive Designs with

1.24k views • 62 slides
in Rare Patients " Some day risk will be zero My, oh my Some day pills will be magic And

in Rare Patients " Some day risk will be zero My, oh my Some day pills will be magic And

6/22/2013 Ninewells Hospital & Medical School Some day drugs will be perfect If we try Some day drugs will be perfect " Assessing Therapies And no one will ever die in Rare Patients " Some day risk will be zero My, oh my Some

355 views • 23 slides
Impact of Plate Shape and Size on Individual Food Waste in a University Dining Hall BRENNA

Impact of Plate Shape and Size on Individual Food Waste in a University Dining Hall BRENNA

Impact of Plate Shape and Size on Individual Food Waste in a University Dining Hall BRENNA ELLISON AG & CONSUMER ECONOMICS, UNIVERSITY OF ILLINOIS Research Team Rachel Richardson, MS (thesis research topic) Melissa Pflugh Prescott,

496 views • 16 slides
Specifications A Controlled Experiment REFSQ18, Utrecht, The Netherlands Jonas Paul

Specifications A Controlled Experiment REFSQ18, Utrecht, The Netherlands Jonas Paul

Using Tools to Assist Identification of Non-Requirements in Requirements Specifications A Controlled Experiment REFSQ18, Utrecht, The Netherlands Jonas Paul Winkler, Andreas Vogelsang DCAITI, Technische Universitt Berlin March 20, 2018

278 views • 17 slides
Adaptive Operator Selection via Online Learning and Fitness Landscape Metrics Pietro Consoli

Adaptive Operator Selection via Online Learning and Fitness Landscape Metrics Pietro Consoli

Adaptive Operator Selection via Online Learning and Fitness Landscape Metrics Pietro Consoli Leandro L. Minku Xin Yao CERCIA, School of Computer Science University of Birmingham, United Kingdom www.cs.bham.ac.uk/~pac265

698 views • 24 slides
Metastability for Interacting Particle Systems Introduction Frank den Hollander, Leiden

Metastability for Interacting Particle Systems Introduction Frank den Hollander, Leiden

Metastability for Interacting Particle Systems Introduction Frank den Hollander, Leiden University, The Netherlands Elena Pulvirenti, Bonn University, Germany Online Open Probability School , June 22, 2020, supported by PIMS, CRM, SMS,

515 views • 26 slides
The QCD crossover from Lattice QCD July 25, 2018 Patrick Steinbrecher HotQCD collaboration The

The QCD crossover from Lattice QCD July 25, 2018 Patrick Steinbrecher HotQCD collaboration The

The QCD crossover from Lattice QCD July 25, 2018 Patrick Steinbrecher HotQCD collaboration The QCD phase diagram July 25, 2018 Patrick Steinbrecher Slide 1 Quantum Chromodynamics from first principles #configurations m s /m l =27, N

270 views • 26 slides

More recommend