International Symposium on Empirical Software Engineering and Measurements, Toronto, Canada – November 10th, 2017 Graphical vs. Tabular Notations for Risk Models: On the Role of Textual Labels and Complexity Katsiaryna (Kate) Labunets TU Delft, The Netherlands E: k.labunets@tudelft.nl Joint work with Fabio Massacci, University of Trento, Italy Alessandra Tedeschi, DeepBlue srl, Rome, Italy 1
Rationale • Risk recommendations should be “consumed” mostly by not-experts in security • What if the security representation is not easy to understand? – Stakeholder does not understand you – The security recommendations are not implemented • “Understand” != “Believe to have understood” 2
Example Risk Models A simple example of one attack path represented in graphical and tabular notation. CORAS diagram UML-style diagram Treatment Consequence Regularly inform Treatment customers of terms of Regularly inform Unwanted incident Asset customers of security use Vulnerability best practces Consequence Vulnerability Customer Unauthorized Severe severe Lack of compliance shares credentials account login with terms of use [unlikely] with next-of-kin Integrity of Lack of Customer Threat scenario Unwanted incident Threat Asset compliance account data Unauthorized Customer Integrity of Customer with terms account login shares credental account data Threat Threat scenario Likelihood of use with next-of-kin [Likelihood: unlikely] NIST table row entry Threat event Threat Vulnerability Impact Overall Level of Asset Security source likelihood impact control Customer shares Customer Lack of Unauthorized Unlikely Severe Integrity of Regularly inform credentials with compliance with account login account customers of next-of-kin terms of use data terms of use 3
Previous work • Published in EMSE journal: – Labunets, K., Massacci, F., Paci, F., Marczak, S. and de Oliveira, F.M., 2017. Model comprehension for security risk assessment: an empirical comparison of tabular vs. graphical representations. Empirical Software Engineering , pp.1-40 . • Included two studies: – [2014] 69 MSc and BSc students from Italy and Brazil – [2015] 83 professional post-master course and MSc students from Italy • Treatment – graphical and tabular risk modeling notations • Findings – Tabular is more effective that the graphical representation for simple comprehension tasks – Less difference for complex tasks, but still tabular is better 4
Research questions We address the following questions for participants with a significant work experience: RQ1 : Does the task complexity have an impact on the comprehensibility of the models? RQ2 : Does the availability of textual labels improve the participants effectiveness in extracting correct information about security risks? 5
Experiment Description [1/2] • Goal: – Compare tabular vs. graphical risk models w.r.t. comprehensibility • Treatments – Notations: NIST 800-30 (tabular); CORAS (graphical); UML-style (graphical) – Task: open questions with different level of complexity about information presented in the model • 7 questions (originally 12 but 5 questions were discarded due to an implementation error) • Between-subject design – one treatment per participant 6
Experiment Description [2/2] • Application scenario: – Online Banking scenario by Poste Italiane • Recruitment process: – Email invitation distributed via mailing lists by UNITN and DeepBlue – Offered a reward of 20 euro (via PayPal) – 572 attempts to start the experiment • Participants: – 61 professional (avg. 9 years of working experience) The number of participants reached each experimental phase 7
Demographics Age Education degree 11% 23% BSc 36% 24–30 yrs old 45% MSc 31-40 yrs old 36% MBA 41-62 yrs old PhD 41% 8% Experience in graph. modeling Working experience languages 3% 18% Did not report 2% Novices 16% 36% 23% Had 1-3 yrs Beginners Had 4-7 yrs Competent users 28% Had >7 yrs Proficient users 43% 31% Experts 8
Used Risk Models: NIST Threat Overall Level of Threat Event Vulnerabilities Impact Asset Security Controls Source Likelihoo Impact 1. Poor security 1. Regularly inform customers about Customer's browser infected by Trojan and this leads awareness Unauthorized transaction via Integrity of security best practices. Hacker Likely Severe to alteration of transaction data 2. Weak malware web application account data 2. Strengthen authentication of protection transaction in web application. Keylogger installed on computer and this leads to sniffing customer credentials. Which leads to Cyber Insufficient detection of Unauthorized transaction via Integrity of Severe Strengthen authentication of Likely unauthorized access to customer account via web criminal spyware web application account data transaction in web application. application. 1. Regularly inform customers about Spear-phishing attack on customers leads to sniffing Cyber Unauthorized transaction via Integrity of security best practices. customer credentials. Which leads to unauthorized criminal Poor security awareness Likely Severe web application account data 2. Strengthen authentication of access to customer account via web application. transaction in web application. Keylogger installed on customer's computer leads to Cyber Insufficient detection of Unauthorized access to customer User authenticity Certain Severe sniffing customer credentials criminal spyware account via web application Spear-phishing attack on customers leads to sniffing Cyber Unauthorized access to customer Severe Regularly inform customers about criminal Poor security awareness User authenticity Certain customer credentials account via web application security best practices. Keylogger installed on customer's computer leads to Cyber Insufficient detection of Unauthorized access to customer Confidentiality of Certain Severe sniffing customer credentials criminal spyware account via web application customer data Spear-phishing attack on customers leads to sniffing Cyber Unauthorized access to customer Confidentiality of Severe Regularly inform customers about criminal Poor security awareness Certain customer credentials account via web application customer data security best practices. Fake banking app offered on application store and this Cyber Lack of mechanisms for Unauthorized access to customer User authenticity Likely Critical Conduct regular searches for fake apps. leads to sniffing customer credentials criminal authentication of app account via fake app Fake banking app offered on application store and this Cyber Lack of mechanisms for Unauthorized access to customer Confidentiality of Likely Severe Conduct regular searches for fake apps. leads to sniffing customer credentials criminal authentication of app account via fake app customer data Fake banking app offered on application store leads to Cyber Lack of mechanisms for Unauthorized transaction via Integrity of sniffing customer credentials. Which leads to Unlikely Minor Conduct regular searches for fake apps. criminal authentication of app Poste App account data unauthorized access to customer account via fake app. Fake banking app offered on application store leads to Cyber Lack of mechanisms for Unauthorized transaction via Integrity of Unlikely Minor Conduct regular searches for fake apps. alteration of transaction data criminal authentication of app Poste App account data Smartphone infected by malware and this leads to Weak malware protection Unauthorized transaction via Integrity of Minor Regularly inform customers about Hacker Unlikely alteration of transaction data Poste App account data security best practices. 1. Use of web application Online banking service goes Availability of Minor 1. Monitor network traffic. Denial-of-service attack Hacker Certain 2. Insufficient resilience down service 2. Increase bandwidth. System Online banking service goes Availability of Minor Strengthen verification and validation 9 Web-application goes down Immature technology Certain failure down service procedures.
Used Risk Models: CORAS 10
Used Risk Models: UML-style 11
Comprehension Questions We ask to identify a risk element of a specific type that is related to another element of a different type. “Which threats can exploit the vulnerability ‘Poor security awareness’? Please specify all threats:” At least one question per element type: Graphical element types: Tabular element types: 1. Threat 1. Threat source 2. Vulnerability 2. Vulnerability 3. Threat scenario 3. Threat event 4. Unwanted incident 4. Impact 5. Likelihood 5. Overall likelihood 6. Consequence 6. Level of impact 7. Asset 7. Asset 8. Treatment 8. Security control 12
Recommend
More recommend
