Compliance Cautions Investigating Security Issues Associated with - PowerPoint PPT Presentation

LASER Workshop 2020 Compliance Cautions Investigating Security Issues Associated with U.S. Digital-Security Standards Rock Stevens , Kevin Halliday, Michelle Mazurek / / University of Maryland Josiah Dykstra, James Chapman, Alexander F armer / / Independent Researchers Wendy Knox Everette / / Leviathan Security Group Garrett Bladow / / Dragos, Inc. 1

Compliance Standards ◂ What are they? ◂ Why use them? ◂ How are they enforced? ◂ What’s the problem? 2

Even if you had perfect compliance, what else could go wrong? First empirical evaluation of compliance standards for security issues that exist within perfect compliance 3

Standards we examined 4

Study Methods e t a u l o a t v g e s n g s i t n t c s r o i u d n e e t d n o p e s n fi i x r e t h o e a s e h t c u g e s c f l n c a o s o e a n i r r o l v d c e t u a r i e n s h d s p i l fi i c t u p o p d r t r a m ’ l a e i a s c y d s p l e e r s s o b n u e n x u s i c D i o A h E e i o s l s c i r n r - i t y u r r d a o x a a b o 3 r z 1 2 i p e V - a S F i e s n s d n e a e n i r r g l a r t o s 5

g n i Real-world experience t c u e d e n r h o t c f Exploitation in the wild s o r e t i h d c u t r a i a d e e u n Unanimous agreement s A e i l s r - y d x b r 1 i - a S e d n n i l a t s 6

Probability Unlikely Seldom Occasional Likely Frequent Catastrophic M H H E E Severity Critical L M H H E Moderate L L M M H Negligible L L L L M E - Extremely High H - High M - Moderate L - Low 7

g n i t c In total, 148 issues ranging u e d e n r h o t c from low to extremely high f s o r e t i h d c u risk t r a i a d e e u n s A e i l s r - y d x b r 1 i - a S e d n n i l a t s 8

Data vulnerability g n i t c u e d e n r h o t c f s o r e t i h d c u t r a i a d e e u n s A e i l s r - y d x b r 1 i - a S e d n n i l a t s 9

Data vulnerability g n i t c u e d e n r h o t c f s o r e t i h d c u t r a i a d e e u n s A e i l s r - y d x b r 1 i - a S e d n n i l a t s 10

Under-defined process g n i t c u e d e n r h o t c f s o r e t i h d c u t r a i a d e e u n s A e i l s r - y d x b r 1 i - a S e d n n i l a t s 11

12

LASER Talking Points, Pt 1 ● Cold calling the experts! ○ Friends/past contacts ○ Industry experts met a previous conferences ● Interrater reliability ● Codebook development 13

LASER Talking Points ● Interrater reliability ○ How to get results when people are in 4 different time zones, and everyone has a full-time job?? ○ Calculated with Krippendorff's alpha ● Codebook development 14

LASER Talking Points 15

LASER Talking Points ● Codebook development ○ Determine root cause ○ Do it iteratively until you have agreement ○ Define terms upfront!!! 16

e t a u CISOs and authors l a v e s t s r n e o p Challenge assumptions i x t e a s u g e l n c a n i v d a e n Validate findings i l fi t p r m ’ e s p r o e x c h E c r u r Provide context a o 2 e F s e r 17

LASER Talking Points, Pt 2 ◂ Confirmed e t a u l a v e real-world misuse of s t s r n e o p i x t e compliance a s u g e l n c a n i v d a e n i l fi t p standards r m ’ e s p r o e x c h E c r u r ◂ “White box” pentest a o 2 e F s e r 18

o Enforcers t s g n o i d t n s fi Creators e e h s c a o e r o l c u r s s p i Aggregators o p d l a c y s l s s b n u i D i o o s i n i t r a o a 3 z p V i s n e a r g r o 19

“Not my Job” CVEs o NDAs t RFCs closed s g n o i d t n s Centralized fi e Direct e h RFCs s c repository reporting a o e r o l c u r s s p i o p d l a c y s l s s b n u i D i o o s i n i t r a o a 3 z p V i Cease communications! s n e a r g r Federal o reporting 20

What did you try that did not succeed before getting to the results you presented? 8 months to finish first part, 8 months of NDA negotiations, and Several follow-on interviews to clarify survey data 21

Wrap-up • Wrap up discussion • Next steps? • Plans for post-workshop paper? > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw Compliance Cautions: Investigating Security Issues Associated with U.S. Digital-Security Standards 22