Compliance Cautions Investigating Security Issues Associated with - - PowerPoint PPT Presentation

1

Compliance Cautions

Investigating Security Issues Associated with U.S. Digital-Security Standards

Rock Stevens, Kevin Halliday, Michelle Mazurek / / University of Maryland Josiah Dykstra, James Chapman, Alexander F armer / / Independent Researchers Wendy Knox Everette / / Leviathan Security Group Garrett Bladow / / Dragos, Inc.

LASER Workshop 2020

Compliance Standards

2

◂ What are they? ◂ Why use them? ◂ How are they enforced? ◂ What’s the problem?

3

First empirical evaluation of compliance standards for security issues that exist within perfect compliance

Even if you had perfect compliance, what else could go wrong?

4

Standards we examined

5

1 A u d i t S i x r e s e a r c h e r s c

• n

d u c t i n g l i n e

• b

y

• l

i n e a u d i t

• f

t h r e e s t a n d a r d s 2 E x p e r t e v a l u a t i

• n

s F

• u

r c

• m

p l i a n c e e x p e r t s e v a l u a t e r e s e a r c h e r s ’ fi n d i n g s 3 D i s c l

• s

u r e V a r i

• u

s a p p r

• a

c h e s t

• r

e s p

• n

s i b l y d i s c l

• s

e fi n d i n g s t

• r

g a n i z a t i

• n

s

Study Methods

Real-world experience Exploitation in the wild Unanimous agreement

6

1 A u d i t S i x r e s e a r c h e r s c

• n

d u c t i n g l i n e

• b

y

• l

i n e a u d i t

• f

t h r e e s t a n d a r d s

Probability

7

Unlikely Seldom Occasional Likely Frequent Catastrophic M H H E E Critical L M H H E Moderate L L M M H Negligible L L L L M E - Extremely High H - High M - Moderate L - Low Severity

In total, 148 issues ranging from low to extremely high risk

8

1 A u d i t S i x r e s e a r c h e r s c

• n

d u c t i n g l i n e

• b

y

• l

i n e a u d i t

• f

t h r e e s t a n d a r d s

Data vulnerability

9

1 A u d i t S i x r e s e a r c h e r s c

• n

d u c t i n g l i n e

• b

y

• l

i n e a u d i t

• f

t h r e e s t a n d a r d s

Data vulnerability

10

1 A u d i t S i x r e s e a r c h e r s c

• n

d u c t i n g l i n e

• b

y

• l

i n e a u d i t

• f

t h r e e s t a n d a r d s

Under-defined process

11

1 A u d i t S i x r e s e a r c h e r s c

• n

d u c t i n g l i n e

• b

y

• l

i n e a u d i t

• f

t h r e e s t a n d a r d s

12

13

LASER Talking Points, Pt 1

• Cold calling the experts!

○ Friends/past contacts ○ Industry experts met a previous conferences

• Interrater reliability
• Codebook development

14

LASER Talking Points

• Interrater reliability

○ How to get results when people are in 4 different time zones, and everyone has a full-time job?? ○ Calculated with Krippendorff's alpha

• Codebook development

15

LASER Talking Points

16

LASER Talking Points

• Codebook development

○ Determine root cause ○ Do it iteratively until you have agreement ○ Define terms upfront!!!

CISOs and authors Challenge assumptions Validate findings Provide context

17

2 E x p e r t e v a l u a t i

• n

s F

• u

r c

• m

p l i a n c e e x p e r t s e v a l u a t e r e s e a r c h e r s ’ fi n d i n g s

◂ Confirmed real-world misuse of compliance standards ◂ “White box” pentest

18

2 E x p e r t e v a l u a t i

• n

s F

• u

r c

• m

p l i a n c e e x p e r t s e v a l u a t e r e s e a r c h e r s ’ fi n d i n g s

LASER Talking Points, Pt 2

19

3 D i s c l

• s

u r e V a r i

• u

s a p p r

• a

c h e s t

• r

e s p

• n

s i b l y d i s c l

• s

e fi n d i n g s t

• r

g a n i z a t i

• n

s

Enforcers Creators Aggregators

20

Centralized repository

CVEs Federal reporting RFCs

3 D i s c l

• s

u r e V a r i

• u

s a p p r

• a

c h e s t

• r

e s p

• n

s i b l y d i s c l

• s

e fi n d i n g s t

• r

g a n i z a t i

• n

s

Direct reporting

NDAs RFCs closed “Not my Job” Cease communications!

21

What did you try that did not succeed before getting to the results you presented? 8 months to finish first part, 8 months of NDA negotiations, and Several follow-on interviews to clarify survey data

22

Compliance Cautions: Investigating Security Issues Associated with U.S. Digital-Security Standards > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw

• Wrap up discussion
• Next steps?
• Plans for post-workshop paper?

Wrap-up