Security Metrics, Security Investment Models and Intro to R Tyler - PDF document

Notes Security Metrics, Security Investment Models and Intro to R Tyler Moore CSE 7338 Computer Science & Engineering Department, SMU, Dallas, TX Lecture 2 Notes Outline Managing security investment 1 Security metrics 2 R 3 Gordon-Loeb model 4 Baseline investment models 5 Measuring the security level 6 2 / 70 Managing security investment Overview Notes Motivation It can be important to frame information security decisions using the language of business ⇒ Security investment decisions must balance expected costs and benefits To model rational decisions, we start by simplifying our assumptions of attacker behavior X Strategic adversary Attacker exogenously given, follows a probability of attack known to the defender In this sense, we treat security like a safety problem When is the simplified attacker model appropriate? + Indiscriminate attackers (e.g., phishing, scanning) - Targeted attackers (e.g., spear-phishing, adaptive attacks) 4 / 70 Managing security investment Overview Notes Security cost and benefits cost of benefit of security security $ $ expected prevented direct / indirect losses variable / fixed onetime / recurring sunk / recoverable 5 / 70

Managing security investment Overview Notes Cost of security Definition (Cost of security, security level) The cost of security c is the amount spent to reach a security level s . No security investment ( c = 0) implies s = 0, and for any c > 0, s increases monotonically in c . Definition (Effective security investment) If security investment is effective , the security level can be approximated by the cost of security, i.e., s ≈ c . When does the effective security investment definition apply? When not? 6 / 70 Managing security investment Measuring security benefits Notes Security benefit : reduction of losses incurred in the absence of security In other words: take a small fixed loss now to reduce the chances of a large but uncertain future loss We already have the tools to deal with uncertainty about outcomes: expected utility! 7 / 70 Managing security investment Measuring security benefits Notes Expected utility (discrete) � E [ U ( a )] = U ( o ) · P ( o | a ) o ∈O P ( o | a ) 0.9 0.1 o o 1 : no attack o 2 : attack 8 / 70 Managing security investment Measuring security benefits Notes Expected utility (continuous) � v E [ U ( a )] = U ( x ) · P ( x | a ) dx u P ( o | a ) o u v 9 / 70

Managing security investment Measuring security benefits Notes Loss distribution function Definition (Loss distribution function) Let L s : R + → [0 , 1] be the family of probability distribution functions describing the monetary losses incurred from insecurity for a given security level s . L 0 is the loss distribution function in the absence of security investment Benefit of security: L s − L 0 We use expected utility to compare outcomes for the loss functions 10 / 70 Managing security investment Measuring security benefits Notes Comparing loss functions (discrete) � E [ U ( L )] = U ( o ) · L ( o ) o ∈O L ( loss ) L s 0.9 0.8 L 0 0.2 0.1 loss $0 $2,000 11 / 70 Security metrics Security-benefit metrics Notes Annual loss expectancy Definition (ALE) The annual loss expectancy ALE s is the expected loss per period due to information security failures given security level s , � ∞ ALE s = E ( L s ) = x · L s ( x ) dx . 0 Note that annual suggests a multi-period view. Even when this isn’t the case, the ALE term is used 13 / 70 Security metrics Security-benefit metrics Notes Annual loss expectancy visualized � ∞ � ∞ ALE s = E ( L s ) = x · L s ( x ) dx ALE 0 = E ( L 0 ) = x · L 0 ( x ) dx 0 0 L ( loss ) L s L 0 loss 14 / 70

Security metrics Security-benefit metrics Notes Metrics for security benefits Definition (EBIS) The expected benefit of information security EBIS s is the difference between the loss expectancy without security and the loss expectancy given security level s , EBIS s = ALE 0 − ALE s � ∞ = E ( L 0 ) − E ( L s ) = x · ( L 0 ( x ) − L s ( x )) dx . 0 15 / 70 Security metrics Security-benefit metrics Notes Metrics for security benefits Definition (ENBIS) The expected net benefit of information security investment ENBIS s is given by the expected benefit of information security minus the cost of the investment to reach security level s . ENBIS s = EBIS s − c = ALE 0 − ALE s − c , or, assuming effective security investment, ENBIS s = EBIS s − s . Straightforward investment rule: only invest if ENBIS s > 0 16 / 70 Security metrics Security-benefit metrics Notes Let’s calculate the metrics for discrete loss functions L ( loss ) L s 0.9 ALE 0 = $0 · 0 . 8 + $2000 · 0 . 2 = $400 0.8 L 0 ALE s = $0 · 0 . 9 + $2000 · 0 . 1 = $200 EBIS s = ALE 0 − ALE s = $400 − $200 = $200 ENBIS s = ALE 0 − ALE s − c = $200 − c 0.2 0.1 loss $0 $2,000 17 / 70 Security metrics Security-benefit metrics Notes Bernoulli loss assumption OK, so continuous loss distribution functions are nice, but they can be difficult to analyze Not to mention it can be hard to justify assumptions about how the loss distribution might be shaped Simplified scenario Two loss outcomes: { 0 , λ } λ > 0: fixed loss, occurs with p s = L s ( λ ) With probability 1 − p s = L s (0), suffers no loss 18 / 70

Security metrics Security-benefit metrics Notes Metrics under Bernoulli loss assumption � � ALE s = p s · λ + (1 − p s ) · 0 = p s · λ � �� � E ( L s ) � � � � � � EBIS s = p 0 · λ + (1 − p 0 ) · 0 − p s · λ + (1 − p s ) · 0 = p 0 − p s · λ � �� � � �� � E ( L 0 ) E ( L s ) � � � � ENBIS s = p 0 · λ + (1 − p 0 ) · 0 − p s · λ + (1 − p s ) · 0 − s � �� � � �� � E ( L 0 ) E ( L s ) � � = p 0 − p s · λ − s 19 / 70 Security metrics Security-benefit metrics Notes Recall the antivirus example Suffering a hack costs $2000, AV costs $75 Without AV, 10% chance of being hacked With AV, 1% chance of being hacked no hack o 1 hack o 2 Action U ( o 1 ) P ( o 1 | action ) U ( o 2 ) P ( o 2 | action ) E [ U ( action )] p s E ( L s ) − s s s λ buy AV - $75 0.99 - $75 - $2000 - $95 .01 don’t buy AV 0 0.9 - $2000 - $200 0.1 E ( L 0 ) p 0 λ 20 / 70 Security metrics Security-benefit metrics Notes Metrics under Bernoulli loss assumption ALE s = p s · λ � � EBIS s = p 0 − p s · λ � � ENBIS s = p 0 − p s · λ − s 21 / 70 Security metrics Security-benefit metrics Notes Metrics under Bernoulli loss assumption & λ = 1 Things get simplified even more if we scale the loss to 1 ( λ = 1) ALE s = p s , EBIS s = p 0 − p s , and ENBIS s = p 0 − p s − s 22 / 70

Security metrics High-level investment metrics Notes Return on security investment (ROSI) cost of benefit of security security $ $ ROSI 1) = benefit of security − cost of security cost of security 1) R eturn O n S ecurity I nvestment 23 / 70 Security metrics High-level investment metrics Notes Return on security investment (ROSI) Definition (ROSI) The return on information security investment ROSI s is the ratio of the expected net benefit over the cost of security, ROSI s = ENBIS s = ALE 0 − ALE s − c c c 24 / 70 Security metrics High-level investment metrics Notes NPV: evaluating security investments over time Definition (NPV) The net present value NPV s aggregates the expected net benefit of information security over multiple future periods into a monetary equivalent at present, ∞ ALE 0 , t − ALE s , t − c t � NPV s = − c 0 + , (1 + r ) t t =1 where c 0 is the one-off cost of security at t = 0, c t are recurring costs of security in period t (if any), ALE s , t is loss expectancy for period t and security level s , and r is the discount rate. 25 / 70 Security metrics High-level investment metrics Notes Internal rate of return Definition (IRR) The internal rate of return IRR s is the discount rate r ∗ at which a decision maker using NPV as a sole criterion is indifferent between making the security investment or not, i.e., NPV s = 0. 26 / 70

Security metrics High-level investment metrics Notes Example: countering data breaches 27 / 70 Security metrics High-level investment metrics Notes Comparing two security investments to combat data loss Security investment option 1. Data loss prevention 2. User training Variable Est. Remark Est. Remark c 0 Initial investment 15 K License and 6 K Training deployment material c t Recurring cost per year 1 K Maintenance, 3 K Fee and lost opportunity cost work time of false positives ALE 0 w/o security investment 5 K 20 K legal settlement, probability 25% ALE s with security investment 2 K 1 K False negatives Residual risk (lapses etc.) 28 / 70 Security metrics High-level investment metrics Notes Exercise: compute ENBIS s for both options Which approach (DLP or training) appears to be the better investment over 10 years using expected-net benefit calculations over 10 years? ENBIS s = t max · ( ALE 0 − ALE s − c t ) − c 0 DLP: ENBIS s (1) = ? User training: ENBIS s (2) = ? These calculations favor DLP, but what about the net-present value? 29 / 70 Security metrics High-level investment metrics Notes Net-present value t max ALE 0 , t − ALE s , t − c t � NPV s = − c 0 + (1 + r ) t t =1 Let’s calculate NPV assuming r = 5% , t max = 10 10 5 K − 2 K − 1 K � NPV s (1) = − 15 K + = $443 (1 . 05) t t =1 10 5 K − 1 K − 3 K � NPV s (2) = − 6 K + = $1 , 722 (1 . 05) t t =1 Using NPV, we find training to be better value than DLP! 30 / 70