Improving Internet Security Through Information Disclosure: A Field Quasi-Experiment BCSI 2013 Qian Tang (School of Information Systems, SMU) Leigh Linden (Economics Department, UT Austin) John S. Quarterman (Quarterman Creations) Andrew Whinston (McCombs School of Business, UT Austin)
Reality Check 97% (of data beaches) were avoidable, without the need for organizations to resort to difficult or expensive countermeasures … A partner’s lax security practices and poor governance — often outside the victim’s control or expertise — are frequent catalysts in security incidents. (Verizon 2012 Data Breach Investigation Report, p.3, 23) 2
Challenges in Internet Security • Cost concern – Information security costs are difficult to justify when things are going well. • No transparency – No knowledge or internal policies to recognize or deal with security threats – Not to reveal info to protect company reputation – No legislative enforcement • Negative externalities – Someone else’s problem 3
Information Disclosure • Disclosure makes information transparent. • Disclosure can internalize negative externalities of insecurity as reputational and financial loss. • Current security info disclosure regulations have mainly been guidelines and suggestions rather than legislation. • Problems of voluntary reporting 4
What We Propose • Mandatory information disclosure – steer management away from any temptation to suppress unfavorable information – produce standardized data across different industries and companies • Using existing data on security vulnerabilities – minimize corporate reporting costs • Straightforward information presentation – deliver comparable and understandable information 5
Literature Review • Economics of information security – Anderson 2001, Anderson and Moore 2006, Schneier 2002, Kanich et al. 2008, Stone-Gross et al. 2011, Caballero et al. 2011, Rao and Reiley 2012 • Corporate information disclosure – Davis et al. 2008, Balakrishnan et al. 2008, Loughran and McDonald 2011 – Villiers 2012, Verrecchia 1983, Field et al. 2005, Rogers et al. 2010, Jorgensen and Kirschenheiter 2003 • Security information disclosure – Arora et al. 2004, Gal-Or and Ghose 2005, Gordon et al. 2003, Romanosky et al. 2011, Campbell et al. 2003 – Lenard and Rubin 2005, Cate 2009, Kannan et al. 2007 6
Research Hypotheses • Awareness effect – Provide the company with its security information • Reputational effect – Put the company subject to public scrutiny • Overall effect of security info disclosure – H1: Disclosure can improve overall internet security. – H2: The more security issues a company has, the more likely it will improve under disclosure. 7
Research Hypotheses • Social comparison effect – Companies compare themselves against other companies for self-evaluation and tend to behave in consistency with other companies (peer influence). – H3: Companies are more likely to improve their security when the worst security companies are disclosed to have less problems. 8
Experimental Setting • Outgoing spam – A severe security issue worldwide – Common symptom of security problems FireEye, Spamhaus, ISPs took down Grum botnet on 19 July 2012 9
Experimental Setting • Outgoing spam – A severe security issue worldwide – Common symptom of security problems – Data available from blocklists – Measurability – Extreme negative externalities Activity Spam Pollution Property crime Externality ratio 100:1 1:10 7:30 Sources : Stone-Gross et al. 2011, Kanich et al. 2008, Caballero, et al. 2011, Rao and Reiley 2012 • Key terms: Botnet, AS/ASN, Blocklist 10
Data collection and processing • IP level spam data: blocklists – Text files from CBL (Composite Blocking List) and PSBL (Passive Spam Block List) everyday – Each text file consists of millions of lines like this: “1.0.17.248,AS2519,1.0.16.0/23,JP,vectant.ne.jp,,,1349617 960,spamsalot,39” IP address, ASN, Netblock, Country code, Domain,,, Time, Botnet, Volume • Mapping data: Team Cymru and CBL – BGP routing data – Aggregate from IP address to netblock to AS and to organization 11
SpamRankings.net 12
Quasi-Experimental Design • Treatment: info disclosed on SpamRankings.net every month • Clustered assignment: spam rankings for each country • Period: Jan 2011-Jan 2012, with treatment started for US in May, CA in June, BE and TR in July • Matched pair design 13
Traffic to SpamRankings.net Distribution of Internet traffic to SpamRankings.net Within North America and Europe: • US: 29.2% • Canada: 11% • Belgium: 0.1% • Turkey: 1.4% 14
Quasi-experimental data • Monthly outbound spam data on the top 250 spamming ASNs from Jan 2011 to Jan 2012 for the eight selected countries. 15
Statistical models • Basic model 𝑍 𝑗𝑑𝑞𝑢 = 𝜄 0 + 𝜄 1 𝐸 𝑑𝑞 + 𝜁 𝑗𝑑𝑞𝑢 (1) 𝑍 𝑗𝑑𝑞𝑢 is the outcome of interest for AS 𝑗 in country 𝑑 pair 𝑞 at time 𝑢 𝐸 𝑑𝑞𝑢 is treatment indicator for whether information is available for country 𝑑 pair 𝑞 at time 𝑢 • With covariates 𝑍 𝑗𝑑𝑞𝑢 = 𝜄 0 + 𝜄 1 𝐸 𝑑𝑞 + 𝜄 2 𝑌 𝑗𝑑𝑞 + 𝜕 𝑞 + 𝜗 𝑗𝑑𝑞𝑢 (2) 𝑌 𝑗𝑑𝑞 is a vector of pre-treatment AS characteristics 𝜕 𝑞 is country pair fixed effects • With interaction effects – including interaction by country pair 𝑍 𝑗𝑑𝑞𝑢 = 𝜄 0 + 𝜄 1 𝐸 𝑑𝑞 + 𝜄 2 𝑌 𝑗𝑑𝑞 + 𝜄 3 𝑌 𝑗𝑑𝑞 ∗ 𝐸 𝑑𝑞 + 𝜕 𝑞 + 𝜁 𝑗𝑑𝑞𝑢 (3) 16
Standard errors • Within country correlations ( ρ =0.004**) – Cluster-robust standard errors: cluster by countries to correct for heteroskedasticity and within-cluster error correlation (Bertrand et al. 2004) • Small number of clusters (J=8) – Bootstrap-t procedure: correct for over-rejection problem with few clusters (Cameron et al. 2008) 17
Manipulation Check: Country Level • Trend in total outgoing spam volumes for the treated group and the control group 18
Manipulation Check: Company Level Covariates Selection bias does exist. ASes in the treatment group tend to generate less spam and have more IP addresses. The treated group have better-security and large- sized companies. 19
Overall Treatment Effect of Disclosure +Covariates +Covariates Treatment effect is significantly negative and robust to log transformation and asymptotic refinement. H1 is supported. 20
Treatment Effect by AS Characteristics Treatment effect is stronger on ASes with more outgoing spam. H2 is supported. 21
Treatment Effect by Country Pair Treatment effect is significantly negative for all country pairs except for Belgium-Netherlands pair. 22
Robustness Check: Treatment effect over time Treatment effect is consistent over time. 23
Robustness check: Serial Correlation +Covariates Collapse the time series into “pre” - and “post” - periods can take into account the effective sample size. (Bertrand et al. 2004) 24
Social Comparison Effect The more spam observed from other ASes, the less likely an AS would reduce its own spam. H3 is supported. ct ct 25
Conclusions • Mandatory info disclosure on info security with straightforward info presentation can lead to improvement in info security. – We found an average of 15.9% reduction in spam volume of the treated group, compared to control group, due to spam info disclosure. • Awareness, reputational, and social comparison effects can be all leveraged to incentivize organizations. • Limitations – Country pair – Covariates – Awareness effect vs. reputational effect 26
Future Research: Field Experimentation • Use US organizations to design randomized field experiment – Collect covariate data • Industry • Geographic location • Baseline spam volume – Stratify by industry • Randomize organizations to treatment and control groups within each industry section • Check the balance in other covariates between the treated and the control • Rerandomize if the balance is not acceptable 27
Next version SpamRankings.net 28
Next version SpamRankings.net 29
Thank you Questions? qiantang@smu.edu.sg
Model development • Level-1: ASN (Individual) 𝑗 ASN 𝑑 Country 2 ) 𝑍 𝑗𝑑𝑞 = 𝜌 0𝑑𝑞 + 𝑓 𝑗𝑑𝑞 , 𝑓 𝑗𝑑𝑞 ~𝑂(0, 𝜏 𝑑 𝑞 Country pair • Level-2: Country (Cluster) 2 ) 𝜌 0𝑑𝑞 = 𝛾 00𝑞 + 𝛾 01𝑞 𝐸 𝑑𝑞 + 𝑠 0𝑑𝑞 , 𝑠 0𝑑𝑞 ~𝑂(0, 𝜐 𝑞 • Level-3: Country pair (assuming fixed pair effects) 𝛾 00𝑞 = 𝛿 000 + 𝑣 00𝑞 𝛾 01𝑞 = 𝛿 010 + 𝑣 01𝑞 Average treatment effect • Full model 𝑍 𝑗𝑑𝑞 = 𝛿 000 + 𝑣 00𝑞 + (𝛿 010 + 𝑣 01𝑞 )𝐸 𝑑𝑞 + 𝜁 𝑗𝑑𝑞 , 𝜁 𝑗𝑑𝑞 = 𝑠 0𝑑𝑞 + 𝑓 𝑗𝑑𝑞 Errors correlated Country pair Treatment-by-country-pair fixed effects within country interaction 31
Recommend
More recommend
