THE IMPACT OF HEARTBLEED Performed regular vulnerability scans - - PowerPoint PPT Presentation

ΘΕΜΑ:BUGS

Αδάμος Κουμή Πανεπιστήμιο Κύπρου -Τμήμα Πληροφορικής

ΕΠΛ 682: Προχωρημένα Θέματα Ασυάλειας

THE MATTER OF HEARTBLEED UNDERSTANDING THE REPRODUCIBILITY

OF CROWD-REPORTED SECURITY

VULNERABILITIES

MEMORY ERROR VULNERABILITY

 Security vulnerability allows attackers to

manipulate in-memory content to crash a program,

• r obtain unauthorized access to a system.

 Memory error vulnerabilities such as ―Stack

Overflows‖, ―Heap Overflows‖, and ―Use After Free‖ have been ranked among the most dangerous software errors.

3

THE MATTER OF HEARTBLEED

*Z. Durumeric1, J. Kasten1,D. Adrian1, J. A. Halderman1,M. Bailey1,2 ,*F. Li3, N. Weaver3,4, J. Amann4, J. Beekman3, M. Payer3,5, V. Paxson3,4

1 University of Michigan ,2 University of Illinois,

Urbana Champaign3 EECS, University of California, Berkeley,4 International Computer Science Institute,5 Purdue University

THE MATTER OF HEARTBLEED

 On April 7 2014, OpenSSL project publicly disclosed the

Heartbleed vulnerability.

 Βug στην υλοποίηση του TLS Heartbeat Extension.  Vulnerability επέτρεπε στους επιτιθεμένους να διαβάσουν

προστατευόμενη μνήμη από τους εξυπηρετητές(servers) αλλά και τους πελάτες(clients).

5

BACKGROUND

 OpenSSL: open-source cryptographic library that

implements the SSL and TLS protocols

 The Heartbeat Extension:  Either end-point of a TLS connection detects whether its

peer is still present.

 Motivated by the need for session management in

Datagram TLS (DTLS).

 Not require for Standard implementations of TLS(use

tcp for session management )

6

HEARTBEAT EXTENSION

 Peers indicate support for the extension during

the initial TLS handshake.

 Following negotiation, either end-point can send

a HeartbeatRequest message to verify connectivity.

7

NORMAL HEARTBEAT

Heartbeat Request 01 2 hi e7f0n2...... Type Length Payload Random padding Heartbeat Response 02 2 hi dc0n2...... Type Length Payload Random padding

8

HEARTBLEED VULNERABILITY

 OpenSSL Heartbeat Extension Vulnerability,

allowed either end-point to read data following the payload message in its peer’s memory.

 How?  Specifying a payload length larger than the amount of

data in the message.

 Bug: The peer trusts the attacker-specified length of

an attacker-controlled message.

9

HEARTBLEED VULNERABILITY

Heartbeat Request 01 64kb hi e7f0n2...... Type Length Payload Random padding Heartbeat Response 02 64kb hi,username, private cryptographic Keys…………….. dc0n2...... Type Length Payload Random padding

10

Attacker

HEARTBLEED TIMELINE

 21 /03 Neel Mehta of Google discovers Heartbleed  21/03 Google patches OpenSSL on their servers  01/04 Google notifies the OpenSSL core team  02/04 Codenomicon independently discovers Heartbleed  03 /04 Codenomicon informs NCSC-FI National Cyber Security

Centre Finland

 06/04 OpenSSL notifies several Linux distributions  07/04 NCSC-FI notifies OpenSSL core team  07/04 OpenSSL releases version 1.0.1g and a security advisory  08/04 Al-Bassam scans the Alexa Top 10,000  09/04 University of Michigan begins scanning

11

SOLUTIONS

 Patch: Discards the HeartbeatRequest, if the

payload length field exceeds the length of the payload.

 Recompile OpenSSL, with the handshake

removed from the code by using compile time

• ption -DOPENSSL_NO_HEARTBEATS.

12

THE IMPACT OF HEARTBLEED

 Performed regular vulnerability scans against:  Alexa Top 1 Million domains 

1% samples of the public, non-reserved IPv4 address space.

 Every 8 hours. Between April 9 - June 4  Scanning Methodology  Modifying Zmap to send Heartbeat requests

 with no payload  no padding,  zero length

 TLS, DTLS these requests should be rejected.  Vulnerable versions of OpenSSL send a response containing only

padding.

13

SCANNING METHODOLOGY

Heartbeat Request 01 Type Length (no data) (No padding) Heartbeat Response 02 dc0n2...... Type Length (no data) Random padding

14

ALEXA TOP 100

• All of the Alexa Top 100 websites were patched within 48

hours of disclosure.

• At least 44 of the Alexa Top 100 websites were vulnerable.



Combining press releases, Mashable’s report, and Al-Bassam’s scan

15

ESTIMATING INITIAL IMPACT

 Upper bound

60% of HTTPS sites support the Heartbeat extension 91% of these were powered by known vulnerable web servers

at most about 55% of the HTTPS sites in the Alexa Top 1 Million were initially vulnerable

16

ESTIMATING INITIAL IMPACT

 Estimate -> 24–55% of HTTPS servers in the Alexa Top 1 Million were

initially vulnerable

72.7% used known vulnerable web servers

At least about 24% of the HTTPS sites in the Alexa Top 1 Million were initially vulnerable

32.6% sites supported TLS 1.1 or 1.2. TLS 1.1 and 1.2—features introduced in OpenSSL 1.0.1 with the Heartbeat Extension.

 Lower bound

17

VULNERABLE DEVICES AND PRODUCTS

 Heartbleed affected embedded systems.  Communication Servers: Zimbra collaboration iPECS VoIP systems, and

Polycom and Cisco video conference products.

 Software Control Panels: Puppet Enterprise Dashboard, IBM System X

Integrated Management Modules control panel, VMWare servers, Parallels control panels for Plesk .

 Network Attached Storage: QNAP, D-Link, ReadyNAS, LaCie, Synology,

and Western Digital NAS devices.

 Firewall and VPN Devices: Cisco, SonicWALL, WatchGuard, OpenVPN  Printers: Dell, Lexmark, Brother, HP printers.  Miscellaneous: Hikvision and SWANN security cameras, AcquiSuite

power monitors, SpeedLine Solutions ( Pizza POS System‖)

18

OTHER IMPACTS

 Mail Servers: 

Can use TLS for transport security via usage of a StartTLS directive within a plaintext session.

 Scanned a random 1% sample of IPv4 address space

for vulnerable SMTP servers.

 45% providing SMTP+TLS supported the Heartbeat

Extension.

 7.5% were vulnerable to Heartbleed.

19

OTHER IMPACTS

 Tor relays and bridges use OpenSSL to provide TLS-

enabled inter-relay communication.

 April 10 scan (3 days after announcement of the vulnerability)  Found that 97% of relays supported Heartbeat.  48% of the relays remained vulnerable at that time.  The vulnerability allowed an attacker to  extract both short-term onion and long-term identity keys.  intercept traffic and impersonate a relay.  Tor client 

Vulnerability allowing entry guards to read sensitive information from a client’s memory, such as recently visited websites.

20

OTHER IMPACTS

 Bitcoin Clients/ Exchanges  Bitcoin software from May 2012 to April 2014, used a

vulnerable OpenSSL version.

 After Heartbleed’s disclosure, a new Bitcoin version was

released linking to the newly patched OpenSSL version.

 Heartbleed allowed attackers to:  compromise wallets  retrieve private keys  12 customers had a total of 28 BTC (⇡ $6,500) stolen from

BTCJam after account credentials were compromised.

21

OTHER IMPACTS

 Android  Heartbleed only affected Android version 4.1.1.  Google estimated that 33.5% of all Android devices

currently running Android 4.1.

 A vulnerable device would have been

susceptible to having memory read by a malicious server.

22

OTHER IMPACTS

 Wireless Networks  Extended Authentication Protocol  framework for wireless network Authentication use

TLS

 Heartbleed allowed attackers to retrieve

network keys and user credentials from wireless clients and access points.

23

PATCHING BEHAVIOR

Alexa Top 1 Million sites patched within the first week, the patch rate quickly dropped after two weeks.

24

CERTIFICATE REPLACEMENT

 Heartbleed allowed attackers to extract private

cryptographic keys.

 Security community recommended that:  Administrators should generate new cryptographic keys  Revoke compromised certificates  To track which sites replaced certificates and cryptographic

keys they combined data from

 Heartbleed scans,  Michigan’s daily scans of the HTTPS ecosystem ,  ICSI’s Certificate Notary service

25

CERTIFICATE REPLACEMENT

 Less than 40% of Alexa Top 1 Million sites replaced

certificates in the week following disclosure.

 Only 10% of the sites that were vulnerable, 48 hours

after disclosure replaced their certificates within the next month.

 Of those that did, 14% re-used the same private key,

gaining no actual protection by the replacement.

 Only 19% of the vulnerable sites that did replace

their certificates, revoked the original certificate in the same time frame.

26

ATTACK SCENE

 They analyzed who was scanning for the Heartbleed

vulnerability by examining network traffic collected from passive taps at

 Lawrence Berkeley National Laboratory (LBNL),  International Computer Science Institute (ICSI)  National Energy Research Scientific Computing Center

(NERSC),

 honeypot operated on Amazon EC2.  To detect Heartbleed scanning, they extended the Bro’s

SSL/TLS analyzer to recognize Heartbeat messages

27

ATTACK SCENE

 Pre-Disclosure Activity:  Found no evidence of any exploit attempt up through April 7,

2014

 Post-disclosure Activity:  5,948 attempts to exploit the vulnerability from 692 distinct

hosts.



These connections targeted a total 217 hosts.



7 attackers successfully completed 103 exploit attempts against 12 distinct hosts.

 Hosts performing widespread scans exclusively targeted port

443 (HTTPS).

 Small number of exploit against ports 993 (IMAPS), 995

(POP3S), 465 (SMTPS).

28

THE IMPACT OF LARGE-SCALE VULNERABILITY NOTIFICATION

 Three weeks after the initial disclosure a large

number of hosts remained vulnerable.

 Οpportunity to study the impact of large-scale

vulnerability notification.

 They began notifying the system operators

responsible for unpatched systems via email.

 They randomly split the abuse contacts into two

groups, which they notified in two stages.

29

THE IMPACT OF LARGE-SCALE VULNERABILITY NOTIFICATION

30

PATCH RATES FOR DIFFERENT RESPONSE TYPES

31

CONCLUSION

 On April 7, 2014, the OpenSSL project publicly disclosed

the Heartbleed vulnerability,

 A bug in OpenSSL implementation of the TLS Heartbeat

Extension.

 The vulnerability allowed attackers to remotely dump

protected memory including data passed over the secure channel and private cryptographic keys from both clients and servers.

 Heartbleed had the potential to affect any service that used

OpenSSL to facilitate TLS connections.

 Including web,mail, messaging, and database servers,

andoid devices, embedded systems, Tor ,Bitcoin.

32

UNDERSTANDING THE REPRODUCIBILITY

OF CROWD REPORTED SECURITY

VULNERABILITIES

Dongliang Mu, Nanjing University; Alejandro Cuevas, The Pennsylvania State University; Limin Yang and Hang Hu, Virginia Tech; Xinyu Xing, The Pennsylvania State University; Bing Mao, Nanjing University; Gang Wang, Virginia Tech

SECURITY VULNERABILITIES IN SOFTWARE

SYSTEMS

 Τα Security vulnerabilities λογισμικού, αποτελούν

σοβαρή απειλή στις μέρες μας για:

 Χρήστες,  Οργανισμούς,  Έθνη.  Vulnerability exploited by WannaCry ransomware

cryptoworm:



shutdown more than 300,000 servers

 Vulnerability in Equifax’s Apache servers:  Εξέθεσε τους αριθμούς κοινωνικής ασφάλισης του μισού

πληθυσμού των Ηνωμένων Πολιτειών.

34

VULNERABILITY IDENTIFICATION

 Εντοπισμός Security vulnerabilities στις μέρες μας

γίνεται όλο και πιο δύσκολος.

 Software vendors ―the power of the crowd‖ .  Anyone on the Internet can identify and report a

vulnerability:



white hat hackers,

 security analysts  regular software users

35

ΑΝΑΦΟΡΑ VULNERABILITY

 Request a CVE ID from CVE Numbering

Authorities (i.e., MITRE Corporation, software companies).

 After the vulnerability can be publicly released.  CVE ID & vulnerability information added to the

CVE list.

36

CVE WEBSITE

 The CVE ( Common Vulnerabilities and Exposures ) website.  Διατηρεί μια λίστα όλων των γνωστών vulnerabilities που έχουν

αποκτήσει CVE ID.

 Κάθε CVE ID έχει την δική του σελίδα: 

Περιγραφή Vulnerability



List of external references



Τεχνικές λεπτομέρειες περιέχονται στις external references(technical reports, blog/forum posts, oR a PoC.).

 PoC (Proof Of Concept exploit) 

Απόδειξη πως το Vulnerabilitiy υπάρχει.



Δείχνει πώς κάποιος μπορεί να ενεργοποίηση/εκμεταλλευτεί Vulnerabilitiy στο λογισμικό.



Συχνά δεν είναι διαθέσιμο.

37

EXTERNAL REFERENCES

 ExploitDB  Συλλέγει και αρχειοθέτη PoC files  Redhat Bugzilla , OpenWall 

Δέχονται αναφορές Vulnerability από χρήστες

 Προωθούν την συζήτηση μεταξύ των χρηστών  SecurityTracker,SecurityFocus  Ολοκληρωμένες και δομημένες πληροφορίες για γνωστά

vulnerabilities.

38

NATIONAL VULNERABILITY DATABASE (NVD)

 CVE list παρέχεται στο National Vulnerability

Database (NVD)

 Αναλυτές διεξάγουν περαιτέρω έρευνες  Προσθέτουν επιπλέον πληροφορίες για να βοηθήσουν

του οργανισμούς να παράξουν ξανά το Vulnerability.

 The Common Vulnerability Scoring System (CVSS)  Βαθμολογεί τα Vulnerability αναλόγως με την

επικινδυνότητα και επίπτωση τους.

39

VULNERABILITY REPRODUCTION

 Όταν αναφερθεί vulnerability υπάρχει η επιτακτική

ανάγκη αναπαραγωγής του.

 Developers & Εταιρίες παράγωγης λογισμικού με

σκοπό:

 Να βρεθούν τα αίτια.  Να δημιουργηθούν security patches.  Security Analysts με σκοπό:  Να εκτιμήσουν του κινδύνους για τους πελάτες τους.  Να μειώσουν τις επιπτώσεις στους πελάτες τους.  Ερευνητές  Για αξιολόγηση των τεχνικών ανίχνευσης και μετριασμού

Vulnerabilities.

40

VULNERABILITY REPRODUCTION

 Υπαρχή χάσμα μεταξύ της αναφοράς vulnerability και

της επιδιόρθωσης του(patching).

 Vulnerabilities που δεν μπορούν να αναπαραχθούν

συνήθως δεν διορθώνονται.

 Χρήστης του Facebook ανάφερε ότι υπάρχει κάποιο vulnerability

το οποίο, επιτρέπει σε attackers να κάνουν post στα timeline των χρηστών.

 Αρχικά αγνοήθηκε από τους μηχανικούς λογισμικού του Facebook

 ―lack of enough details to reproduce the vulnerability‖

 Διορθώθηκε όταν το Facebook CEO’s timeline έγινε hack.

41

THIS PAPER

 They develop a series of experiments to assess the

usability of the information provided by the reporters.

 Manually reproducing vulnerabilities:  How reproducible are public security vulnerability

reports?

 What makes vulnerability reproduction difficult?  How to improve the efficiency of vulnerability

reproduction?

42

VULNERABILITY REPORT DATASET

 Large collection of reported vulnerabilities from the past 17 years.  Two datasets:  Primary dataset of 291 vulnerabilities with CVE IDs.  Complementary of 77 vulnerabilities without CVE IDs (From

ExploitDB) .

 Focus on memory error vulnerabilities:  high severity and significant real-world impact.  Average CVSS Score 7.6 > Overall Average CVSS Score 6.2  Linux Software:  requires the source code of the Vulnerable software  Linux-based vulnerabilities have a high impact. (servers, data center

nodes, supercomputers, Android devices)

43

THE ANALYST TEAM

 Η ομάδα αποτελείται από 5 security analysts:  In-depth knowledge of memory error vulnerabilities.  First-hand experience analyzing vulnerabilities, writing

exploits, and developing patches.

 The analysts discovered and reported over 20 new

vulnerabilities to CVE website.

 Οι αναλυτές εργάζονται μαζί για να αναπαράξουν όσο

περισσότερα vulnerabilities γίνεται.

 Εάν ένας αναλυτής δεν μπόρεσει να αναπαράξει ένα

vulnerability τότε, το επιχειρεί κάποιος άλλος αργότερα.

44

REPRODUCTION WORKFLOW

45

Report Gathering Environment setup Software Preparation stage Vulnerability reproduction

Security analyst συλλέγει τις αναφορές

και τις πήγες που συσχετίζονται με το συγκεκριμένο Vulnerability.

46

Report Gathering Environment setup Software Preparation stage Vulnerability reproduction

 Security analyst εξάγει τις ακόλουθες πληροφορίες από την αναφορά: 

Vulnerable Software Version

 Εάν δεν αναφέρεται η προσπάθεια αναπαραγωγής θεωρείται αποτυχημένη.



Operating System



Installation & Configuration



Proof-of-Concept (PoC) File



Trigger Method for PoC



Vulnerability Verification



Χρήση Default Settings για της πληροφορίες που λείπουν.

 Εγκατάσταση λειτουργικού συστήματος

 Εάν δεν αναφέρεται η έκδοση του λειτουργικού συστήματος :



Επιλέγεται Linux λειτουργικό σύστημα που κυκλοφόρησε στο ίδιο έτος που αναφέρθηκε το vulnerability 47

Report Gathering Environment setup Software Preparation stage Vulnerability reproduction

Εγκατάσταση και ρύθμιση του λογισμικού

 Ο αναλυτής κάνει Compile και εγκαθιστά το λογισμικό,

ακολουθώντας τα compilation και configuration options που παρέχονται στο report.

 Εάν δεν παρέχονται, ακολουθούνται οι προκαθορισμένες

οδηγίες εγκατάστασης του λογισμικού.

48

Report Gathering Environment setup Software Preparation stage Vulnerability reproduction

 Γίνεται προσπάθεια αναπαραγωγής του Vulnerability με

χρήση:

 PoC File

 Απαραίτητο  Εάν δεν υπάρχει η προσπάθεια θεωρείται αποτυχημένη.

 Trigger Method for PoC

 Εάν δεν υπάρχει ακολουθούνται κάποιες προκαθορισμένες ενέργειες

που εξαρτώνται από το είδος του λογισμικού.

 Η αναπαραγωγή είναι επιτυχής εάν παρατηρήσουμε τον

απροσδόκητο τερματισμό του προγράμματος( εξαιτίας Memory Vulnerability).

49

REPRODUCTION EXPERIMENT: CONTROLLED INFORMATION SOURCE

 Για κάθε Vulnerability που έχει CVE ID έγινε

αξιολόγηση και σύγκριση της ποιότητας τον πληροφοριών που παρέχονται από κάθε external references.

 Έγινε προσπάθεια αναπαραγωγής του Vulnerability

μόνο από τις πληροφορίες που παρέχονται από κάθε ένα external reference.

50

REPRODUCTION EXPERIMENT: CONTROLLED INFORMATION SOURCE

CVE Single- source CVE Combined-top5 CVE Combined-all Reproduction Failure

 Αναπαραγωγή του Vulnerability με την χρήση τον

πληροφοριών ενός από top 5 referenced websites στο CVE (SecurityFocus,Redhat Bugzilla, ExploitDB, OpenWall,SecurityTracker.)

 Δεν χρησιμοποιηθήκαν εξωτερικοί σύνδεσμοι

(external links).

Failed Failed

Success Success Success

Reproduction Success

Failed

51

REPRODUCTION EXPERIMENT: CONTROLLED INFORMATION SOURCE

CVE Single- source CVE Combined-top5 CVE Combined-all Reproduction Failure

 Αναπαραγωγή του Vulnerability συνδυάζοντας

πληροφορίες και από τα 5 top referenced websites στο CVE.

 Δεν χρησιμοποιηθήκαν εξωτερικοί σύνδεσμοι (external

links).

Failed Failed Succes s Succes s Succes s

Reproduction Success

Failed

52

REPRODUCTION EXPERIMENT: CONTROLLED INFORMATION SOURCE

CVE Single- source CVE Combined-top5 CVE Combined-all Reproduction Failure

 Αναπαραγωγή του Vulnerability συνδυάζοντας

πληροφορίες από όλες τις διαθέσιμες πηγές.

Failed Failed Succes s Success Succes s

Reproduction Success

Failed

53

RESULTS

 Successfully reproduced 202 out of 368

vulnerabilities (54.9%).

 Most time-consuming part  set up the environment  compile the vulnerable software with correct options  Result indicates poor usability and

reproducibility in crowd sourced vulnerability reports.

54

REPRODUCIBILITY

 single-source

setting returns a low success rate

 CVE Combined-

top5 improved the success rate (43.9%).

 CVE Combined-

all success rate is improved to 62.5%

55

MISSING INFORMATION

 Even after combining all the information sources 

at least 95.1% of the 368 vulnerabilities reports, still missed

• ne required information field.

 Missing Information: 

87% missed details on software installation options and configurations.

 22.8 % missed OS.  11.7% missed PoC files. 

26.4% methods to trigger the vulnerability.

 Combining different information sources helps retrieve

missing pieces (PoC files, trigger methods, OS information).

56

MISSING INFORMATION VS. REPRODUCIBILITY

 Successful cases do not necessarily have complete

information.

 Missing information of the succeeded cases can be

resolved by the ―common-sense‖ knowledge (i.e., default settings).

 Reproduction is prone to failure if one of the following

is missing :

 Vulnerable software version.  PoC files. 

trigger method.

57

ADDITIONAL FACTORS

 Vulnerability Type: Stack Overflow are most difficult to reproduce

(reproduction rate 40% )

 Vulnerability Severity: highly severe vulnerabilities (CVSS score

>8) are more difficult to reproduce.

 Project Size: 

Small projects (less than1,000 lines of code ) vulnerabilities are not necessarily easier to reproduce.

 Poor software documentation.  Lack of comprehensive reports



Middle categories (1,000–1,000,000 lines of code) high reproduction rate

 Comprehensive vulnerability reports  Good bug reporting guidelines



Large projects (more than 1,000,000 lines of code)

 difficult to reproduce reported vulnerabilities

58

TRY TO REPRODUCE FAILED CASES WITH MANUAL

TROUBLESHOOTING

 For a given ―failed‖ case  Understand the exact underlying causes for the reproduction

failure.

 Use ad-hoc techniques as demanded by each case to reproduce the

Vulnerability

 Ad-hoc techniques  Debugging the software and PoC files  Inspecting and modifying the source code,  Testing the cases in multiple operating systems and versions  Searching related hints on the web

59

TRY TO REPRODUCE FAILED CASES WITH MANUAL

TROUBLESHOOTING

 Successfully reproduced another: 

94 CVE vulnerabilities

 57 non-CVE vulnerabilities,  Increasing the overall success rate from 54.9% to 95.9%.  The failed cases take substantially longer to

troubleshoot combined with the previous experiments.

60

OBSERVATIONS AND LESSONS

 Priority of Information  Given a failed case, the key question is which piece of

information is problematic.

 Manual troubleshooting

 Trigger method  Software installation options  PoC  Software configuration  Operating system

 Correlation of Different Vulnerabilities. It is helpful

to recover missing information from other similar vulnerabilities reports.

61

Q1:WHAT INFORMATION IS NECESSARY TO

THE REPRODUCTION?

Q2:WHAT INFORMATION IS OFTEN MISSING

IN EXISTING REPORTS?

62

Q3 :WHAT TECHNIQUES DO YOU USUALLY USE TO

REPRODUCE THE VULNERABILITY IF CERTAIN INFORMATION IS MISSING?

63

SUGGESTIONS TO IMPROVE THE

REPRODUCIBILITY OF THE REPORTED VULNERABILITIES

 Standardizing Vulnerability Reports.  Minimal set of required information fields (PoC, PoC

trigger method, Software Version, Software compilation

• ption).

 Automated Tools to Assist Vulnerability Reporters.  Develop automated tools which can help collecting

information and generating standardized reports.

64

SUGGESTIONS TO IMPROVE THE

REPRODUCIBILITY OF THE REPORTED VULNERABILITIES

 Vulnerability Reproduction Automation  Report gathering step

 Build automated tools to search, collect, and fuse all the available

information online to generate a ―reproducible‖ report.

 The environment setup step

 Reproducer prepare a configuration file to specify the environment

requirements

 Automated tools can generate a Dockerfile and a container for the

reproducer to directly verify the vulnerability.

 Reproduction step

 Automate the verification process.

65

CONCLUSION

 Reproducing a vulnerability is a prerequisite step

when diagnosing and eliminating a security threat.

 Depth analysis on the reproducibility of crowd-

reported security vulnerabilities.

 Result: Poor reproducibility of crowd-reported security

vulnerabilities, due to missing information (software

• version. PoC files. trigger method. ), in vulnerability

reports.

 It is helpful to recover missing information from other

similar vulnerabilities reports

 Need to automate vulnerability reproduction

66