cyberdyne
play

CYBERDYNE: Automatic bug-finding at scale Peter Goodman - PowerPoint PPT Presentation

Feb 13, 2024 •167 likes •808 views

CYBERDYNE: Automatic bug-finding at scale Peter Goodman COUNTERMEASURE 2016 Cyberdyne (ex)terminates bugs Finds bug in binaries Combines different techniques Coverage-guided fuzzing Symbolic execution Trail of Bits |

  1. CYBERDYNE: Automatic bug-finding at scale Peter Goodman COUNTERMEASURE 2016

  3. Cyberdyne (ex)terminates bugs  Finds bug in binaries  Combines different techniques  Coverage-guided fuzzing  Symbolic execution Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 2

  4. Get to know the mind of the machine  Part 1: high level architecture  How to coordinate bug-finding tools  Part 2: low level tools  How do the bug-finding tools work? Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 3

  5. History: Cyber Grand Challenge (1) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 4

  6. History: Cyber Grand Challenge (2)  Capture - the - flag (CTF) competition  Goal: find and exploit bugs in binaries  Goal: patch binaries  Competitors were programs  “Cyber Reasoning Systems” (CRS) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 5

  7. History: Cyber Grand Challenge (3)  Shaped the design of Cyberdyne  Distributed system  Runs on any number of nodes  Automated system  No human intervention required Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 6

  8. Part 1 Skeleton of a bug-finding system Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 7

  9. Ideally, a bug-finding system should …  Find bugs  Simple, right?  Work on real programs  Be easy to scale Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 8

  10. When I grow up … Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 9

  11. First kill: simple fuzzing (1) Byte flips Bit flips Splice Slice Mutation Seed Inputs Mutated Inputs Engine Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 10

  12. First kill: simple fuzzing (1) Radamsa, zzuf, etc. Mutation Seed Inputs Mutated Inputs Engine Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 11

  13. First kill: simple fuzzing (2)  Mutate inputs 12  11 1 10 2 3 9 8 4 7 5 6 Execute inputs  … Terminator Profit?  pag@sloth:~/ cyberdyne start pag@sloth:~/ cyberdyne analyze – program foo – binaries bar pag@sloth:~/ cyberdyne seed – program foo – inputs ./inputs/*  Find bugs! Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 12

  14. First kill: simple fuzzing (2)  Mutate inputs   Execute inputs  12 11 1 10 2 3 9 8 4 7 5 6 … Terminator Profit?  pag@sloth:~/ cyberdyne start pag@sloth:~/ cyberdyne analyze – program foo – binaries bar pag@sloth:~/ cyberdyne seed – program foo – inputs ./inputs/*  Find bugs! Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 13

  15. First kill: simple fuzzing (2)  Mutate inputs   Execute inputs  … Terminator Profit? 12  11 1 pag@sloth:~/ cyberdyne start 10 2 pag@sloth:~/ cyberdyne analyze – program 3 9 foo – binaries bar 8 4 pag@sloth:~/ cyberdyne seed – program foo – 7 5 6 inputs ./inputs/*  Find bugs! Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 14

  16. First kill: simple fuzzing (2)  Mutate inputs   Execute inputs  … Terminator Profit? 12  11 1 pag@sloth:~/ cyberdyne start 10 2 pag@sloth:~/ cyberdyne analyze – program 3 9 foo – binaries bar 8 4 pag@sloth:~/ cyberdyne seed – program foo – 7 5 6 inputs ./inputs/*  Find bugs!  Right???? Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 15

  17. First kill: simple fuzzing (2)  Mutate inputs   Execute inputs  … Terminator Risk of loss!  pag@sloth:~/ cyberdyne start pag@sloth:~/ cyberdyne analyze – program foo – binaries bar pag@sloth:~/ cyberdyne seed – program foo – inputs ./inputs/*  No bugs found  Lost cycles, time Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 16

  18. Misfire: Check your targets  Searching for bugs takes time  Need accountability  Is it worth it to keep searching?  Is progress being made?  How do we measure progress? Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 17

  19. Reload: Track bug-finding progress  Idea: has something new happened?  Track when new code is executed  Code coverage: Instrument program to detect when new code is executed  Inputs that cover new code signal progress Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 18

  20. Need more ammo  Eventually hit a “coverage ceiling”  Decreasing marginal returns  Need heavier guns  Coverage-guided fuzzing: re-seed with inputs that got new coverage (next)  Symbolic execution (later) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 19

  21. Coverage-guided mutational fuzzing (1) Terminator cyberdyne start cyberdyne analyze –pr… cyberdyne launch nukes Step p 3 Ste tep p 1 Ste tep p 2 Ste tep p 4 Gets new Mutate Execute Re-seed Crashes! inputs mutations Coverage? mutator Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 20

  22. Coverage-guided mutational fuzzing (1) AFL Terminator cyberdyne start cyberdyne analyze –pr… cyberdyne launch nukes Step p 3 Ste tep p 1 Ste tep p 2 Ste tep p 4 Gets new Mutate Execute Re-seed Crashes! inputs mutations Coverage? mutator Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 21

  23. Coverage-guided mutational fuzzing (2)  Trivially parallelizable  Run mutation engines concurrently  Scaling fuzzing in Cyberdyne  Fuzzer service internalizes mutation, execution, code coverage  Runs many fuzzers, one mutator each Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 22

  24. Look under the skin of Cyberdyne (1) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 23

  25. Look under the skin of Cyberdyne (2) Terminator cyberdyne start cyberdyne analyze –pr… cyberdyne launch nukes Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 24

  26. Look under the skin of Cyberdyne (3) Fuz uzze zer r (with GRR) Terminator cyberdyne start cyberdyne analyze –pr… Mutates and  cyberdyne launch nukes executes inputs Easy to scale  Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 25

  27. Look under the skin of Cyberdyne (4) Py PySy SymE mEmu mu Terminator cyberdyne start cyberdyne analyze –pr… Coverage-guided  cyberdyne launch nukes binary symbolic executor Harder to scale  Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 26

  28. Look under the skin of Cyberdyne (5) KLEE KL EE (with McSema) Terminator cyberdyne start cyberdyne analyze –pr… LLVM bitcode  cyberdyne launch nukes symbolic executor Hard to use  Hard to scale  Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 27

  29. Look under the skin of Cyberdyne (6) Or Orac acle le Terminator cyberdyne start cyberdyne analyze –pr…  Gatekeeper cyberdyne launch nukes for minset  Detects crashes  Easy to scale Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 28

  30. Look under the skin of Cyberdyne (7) Mins Mi nset Terminator cyberdyne start cyberdyne analyze –pr… Finds inputs that get  cyberdyne launch nukes new code coverage One input at a time  Bottleneck?  Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 29

  31. Part 2 The servos and the gears Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 30

  32. How it works: Minset (1)  What is it?  Minimum set of inputs that produce maximum code coverage  Why use it?  Identify “interesting” inputs  Good candidates for exploration Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 31

  33. How it works: Minset (2) 4 3 2 1 1 4 3 2 3 1 2 4 Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 09.17.2016 | trailofbits.com 32

  34. How it works: Minset (3) 4 3 2 1 1 4 3 2 3 1 2 4 Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 09.17.2016 | trailofbits.com 33

  35. How it works: Minset (4) 4 3 2 1 1 4 3 2 3 1 2 4 𝐷𝑝𝑤(𝐽 3 ) ⊆ 𝐷𝑝𝑤(𝐽 1 ) ∪ 𝐷𝑝𝑤(𝐽 2 ) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 09.17.2016 | trailofbits.com 34

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

At-sea validation of AIRS radiances Peter Minnett Meteorology and Physical Oceanography

At-sea validation of AIRS radiances Peter Minnett Meteorology and Physical Oceanography

At-sea validation of AIRS radiances Peter Minnett Meteorology and Physical Oceanography Rosenstiel School of Marine and Atmospheric Science University of Miami AIRS Validation Meeting Pasadena, 8 November, 2001 The need for validation The

817 views • 37 slides
Radiative forcing of a small-scale wildfire smoke plume at the surface, atmosphere, and TOA from

Radiative forcing of a small-scale wildfire smoke plume at the surface, atmosphere, and TOA from

Radiative forcing of a small-scale wildfire smoke plume at the surface, atmosphere, and TOA from surface and satellite observations John A. Augustine 1 , Robert Stone 1,2 , David Rutan 3 and Anand Inamdar 4 1 Earth System Research Laboratory,

221 views • 18 slides
Using Webly Supervised Data Christopher Thomas and Adriana Kovashka Published in NeurIPS 2019 1

Using Webly Supervised Data Christopher Thomas and Adriana Kovashka Published in NeurIPS 2019 1

Predicting the Politics of an Image Using Webly Supervised Data Christopher Thomas and Adriana Kovashka Published in NeurIPS 2019 1 Ou Outli line Problem introduction Related research Dataset Our method Quantitative

721 views • 37 slides
How to write a good research paper Bill Freeman MIT CSAIL and Google June 22, 2018 Two informal

How to write a good research paper Bill Freeman MIT CSAIL and Google June 22, 2018 Two informal

How to write a good research paper Bill Freeman MIT CSAIL and Google June 22, 2018 Two informal manuscripts about doing research research advice: http://people.csail.mit.edu/billf/publications/How_To_Do_Research.pdf advice to graduate

687 views • 49 slides
Shower & Hadronisation Uncertainties for Precision Top Physics Peter Skands (Monash U) Scale

Shower & Hadronisation Uncertainties for Precision Top Physics Peter Skands (Monash U) Scale

Shower & Hadronisation Uncertainties for Precision Top Physics Peter Skands (Monash U) Scale Variations : How big and how correlated? 7-point variations, with (conservative) soft compensation terms Provided automatically as vector of

393 views • 28 slides
Insightful Automatic Performance Modeling Alexandru Calotoiu 1 , Torsten Hoefler 2 , Martin Schulz

Insightful Automatic Performance Modeling Alexandru Calotoiu 1 , Torsten Hoefler 2 , Martin Schulz

VIRTUAL INSTITUTE HIGH PRODUCTIVITY SUPERCOMPUTING Insightful Automatic Performance Modeling Alexandru Calotoiu 1 , Torsten Hoefler 2 , Martin Schulz 3 , Sergei Shudler 1 and Felix Wolf 1 1 TU Darmstadt , 2 ETH Zrich , 3 Lawrence Livermore

872 views • 63 slides
The Loewner framework for model reduction of large-scale systems: An Overview and Sensitivity

The Loewner framework for model reduction of large-scale systems: An Overview and Sensitivity

The Loewner framework for model reduction of large-scale systems: An Overview and Sensitivity considerations Thanos Antoulas Rice University, Houston, and Max-Planck Institute, Magdeburg e-mail: aca@rice.edu URL: www.ece.rice.edu/ aca

759 views • 49 slides
A Kernel Density Based Approach for Large Scale Image Retrieval and Its Application to Tattoo

A Kernel Density Based Approach for Large Scale Image Retrieval and Its Application to Tattoo

A Kernel Density Based Approach for Large Scale Image Retrieval and Its Application to Tattoo Identification Wei Tong CMU Outline Why tattoo retrieval Used in forensic to identify Victims, criminals Content-based image

802 views • 68 slides
Assimilation of Geostationary Satellite Land Surface Skin Temperature Observations into the GEOS-5

Assimilation of Geostationary Satellite Land Surface Skin Temperature Observations into the GEOS-5

Assimilation of Geostationary Satellite Land Surface Skin Temperature Observations into the GEOS-5 Global Atmospheric Modeling and Assimilation System Clara Draper 1 , 2 , Rolf Reichle 2 , Gabrielle De Lannoy 1 , 2 , and Qing Liu 3 1. GESTAR,

580 views • 19 slides
JET SUBSTRUCTURE AT THE LHC & BEYOND Simone Marzani Universit di Genova & INFN

JET SUBSTRUCTURE AT THE LHC & BEYOND Simone Marzani Universit di Genova & INFN

JET SUBSTRUCTURE AT THE LHC & BEYOND Simone Marzani Universit di Genova & INFN Sezione di Genova Interpreting the LHC Run 2 data and Beyond ICTP Trieste 27 th - 31 st May 2019 1 OUTLINE Jet substructure: where we are

474 views • 29 slides
CSCI 5417 Information Retrieval Systems Jim Martin Lecture 13 10/6/2011 Text classification

CSCI 5417 Information Retrieval Systems Jim Martin Lecture 13 10/6/2011 Text classification

CSCI 5417 Information Retrieval Systems Jim Martin Lecture 13 10/6/2011 Text classification First Nave Bayes Simple, fast, low training and testing cost Then K Nearest Neighbor classification Simple, can easily

730 views • 27 slides
Scaling Marty Weiner Yashh Nelapati Orodruin, Mordor The Shire Friday, November 9, 12

Scaling Marty Weiner Yashh Nelapati Orodruin, Mordor The Shire Friday, November 9, 12

Scaling Marty Weiner Yashh Nelapati Orodruin, Mordor The Shire Friday, November 9, 12 Pinterest is . . . An online pinboard to organize and share what inspires you. Scaling Pinterest Friday, November 9, 12 Friday, November 9, 12 Friday,

1.08k views • 55 slides
UP UP AND OUT: SCALING SOFTWARE WITH AKKA Jonas Bonr CTO Typesafe @jboner Scaling software

UP UP AND OUT: SCALING SOFTWARE WITH AKKA Jonas Bonr CTO Typesafe @jboner Scaling software

UP UP AND OUT: SCALING SOFTWARE WITH AKKA Jonas Bonr CTO Typesafe @jboner Scaling software with Jonas Bonr CTO Typesafe @jboner Scaling Scaling software with software with Scaling Scaling software with software with Akka

1.98k views • 174 slides
Performance Scaling How is my parallel code performing and scaling? Performance metrics

Performance Scaling How is my parallel code performing and scaling? Performance metrics

Performance Scaling How is my parallel code performing and scaling? Performance metrics Measure the execution time T how do we quantify performance improvements? Speed up typically S(N,P) < P Parallel efficiency

377 views • 17 slides
Designing your SaaS Database for Scale with Postgres Lukas

Designing your SaaS Database for Scale with Postgres Lukas

Designing your SaaS Database for Scale with Postgres Lukas Fi9l Ozgun Erdogan What is Citus? Citus extends PostgreSQL (not a fork) to provide

706 views • 39 slides
disordered field theories Ofer Aharony Weizmann Institute of Science CRM-PCTS workshop, October

disordered field theories Ofer Aharony Weizmann Institute of Science CRM-PCTS workshop, October

Renormalization group flows in disordered field theories Ofer Aharony Weizmann Institute of Science CRM-PCTS workshop, October 4, 2018 Based on the papers 1803.08529,1803.08534 with Vladimir Narovlansky Disorder In QFT we like to assume

483 views • 34 slides
Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh | http://reza-zadeh.com Machine Learning

Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh | http://reza-zadeh.com Machine Learning

Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh | http://reza-zadeh.com Machine Learning Pipeline Learning Replicate Algorithm model Data Trained Serve Model Model Repeat entire pipeline Scaling Machine Learning Datasets and

890 views • 21 slides
Classifier Inspired Scaling for Training Set Selection Walter Bennette DISTRIBUTION A: Approved

Classifier Inspired Scaling for Training Set Selection Walter Bennette DISTRIBUTION A: Approved

Classifier Inspired Scaling for Training Set Selection Walter Bennette DISTRIBUTION A: Approved for public release: distribution unlimited: 16 May 2016. Case #88ABW-2016- 2511 Outline Instance-based classification Training set

609 views • 46 slides
Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS

Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS

Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS Workshop Large-scale graph mining Many applications Friend suggestions Recommendation systems Security Advertising Benefits Big data available Rich

822 views • 71 slides
Scaling container policy management with kernel features Joe Stringer Cilium.io Linux Plumbers

Scaling container policy management with kernel features Joe Stringer Cilium.io Linux Plumbers

Scaling container policy management with kernel features Joe Stringer Cilium.io Linux Plumbers 2019, Lisbon, Portugal Joe Stringer Scaling container policy with eBPF Sep 11, 2019 1 / 29 Overview 1 Background 2 Deploying fast datapaths fast

831 views • 53 slides
Scaling Methodology Scaling Methodology Dan Smith Director HW Engineering dsmith@nvidia.com

Scaling Methodology Scaling Methodology Dan Smith Director HW Engineering dsmith@nvidia.com

EDP- -2002 2002 EDP Scaling Methodology Scaling Methodology Dan Smith Director HW Engineering dsmith@nvidia.com Introduction NVIDIA NVIDIAs growth Methodology Group NVIDIAs methodology beginnings How the methodology has changed

135 views • 13 slides
Scaling the Practical Education Experience Joel Sommers Andrew Moore Colgate University

Scaling the Practical Education Experience Joel Sommers Andrew Moore Colgate University

Scaling the Practical Education Experience Joel Sommers Andrew Moore Colgate University University of Cambridge jsommers@colgate.edu andrew.moore@cl.cam.ac.uk Motivation Many tools, environments, and approaches developed for practical,

349 views • 7 slides
Platforms and Algorithms for Big Data Analytics Chandan K. Reddy Department of Computer Science

Platforms and Algorithms for Big Data Analytics Chandan K. Reddy Department of Computer Science

Platforms and Algorithms for Big Data Analytics Chandan K. Reddy Department of Computer Science Wayne State University http://www.cs.wayne.edu/~reddy/ http://dmkd.cs.wayne.edu/TUTORIAL/Bigdata/ What is Big Data? A collection of large and

1.28k views • 89 slides
Finite-size scaling For a system of length L, the correlation length Express divergent quantities

Finite-size scaling For a system of length L, the correlation length Express divergent quantities

Finite-size scaling For a system of length L, the correlation length Express divergent quantities in terms of correlation length, e.g., The largest value is obtained by substituting At what T does the maximum occur? The peak position of a

427 views • 14 slides

More recommend