UNDERSTANDING SECURITY MISTAKES DEVELOPERS MAKE Qualitative - - PowerPoint PPT Presentation

▶
understanding security mistakes developers make
SMART_READER_LITE
LIVE PREVIEW

UNDERSTANDING SECURITY MISTAKES DEVELOPERS MAKE Qualitative - - PowerPoint PPT Presentation

Jan 14, 2023 144 likes •1.64k views

UNDERSTANDING SECURITY MISTAKES DEVELOPERS MAKE Qualitative Analysis From Build It, Break It, Fix It Daniel Votipka , Kelsey Fulton, James Parker, Matthew Hou, Michelle Mazurek, and Mike Hicks University of Maryland, College Park 1 SOLVED

slide-1
SLIDE 1

UNDERSTANDING SECURITY MISTAKES DEVELOPERS MAKE

Daniel Votipka, Kelsey Fulton, James Parker, Matthew Hou, Michelle Mazurek, and Mike Hicks

Qualitative Analysis From Build It, Break It, Fix It

University of Maryland, College Park

1

slide-2
SLIDE 2

“SOLVED” VULNERABILITIES ARE STILL A VERY REAL PROBLEM

2

slide-3
SLIDE 3

“SOLVED” VULNERABILITIES ARE STILL A VERY REAL PROBLEM

2

slide-4
SLIDE 4

“SOLVED” VULNERABILITIES ARE STILL A VERY REAL PROBLEM

2

slide-5
SLIDE 5

“SOLVED” VULNERABILITIES ARE STILL A VERY REAL PROBLEM

2

slide-6
SLIDE 6

“SOLVED” VULNERABILITIES ARE STILL A VERY REAL PROBLEM

2

slide-7
SLIDE 7

3

slide-8
SLIDE 8

“rolling its own crypto rather than relying on tried and tested solutions…. The devices were sending hardcoded encryption keys

  • ver the network, and were using a fixed

initialization vector… Moreover, the devices didn’t include any message signing”

3

slide-9
SLIDE 9

“rolling its own crypto rather than relying on tried and tested solutions…. The devices were sending hardcoded encryption keys

  • ver the network, and were using a fixed

initialization vector… Moreover, the devices didn’t include any message signing”

3

slide-10
SLIDE 10

“rolling its own crypto rather than relying on tried and tested solutions…. The devices were sending hardcoded encryption keys

  • ver the network, and were using a fixed

initialization vector… Moreover, the devices didn’t include any message signing”

3

slide-11
SLIDE 11

“rolling its own crypto rather than relying on tried and tested solutions…. The devices were sending hardcoded encryption keys

  • ver the network, and were using a fixed

initialization vector… Moreover, the devices didn’t include any message signing”

3

slide-12
SLIDE 12

“rolling its own crypto rather than relying on tried and tested solutions…. The devices were sending hardcoded encryption keys

  • ver the network, and were using a fixed

initialization vector… Moreover, the devices didn’t include any message signing”

3

slide-13
SLIDE 13

“SBI's Mumbai-based data center had a server without password protection”

4

slide-14
SLIDE 14

5

slide-15
SLIDE 15

Why do developers continue to make stupid and lazy mistakes?

6

slide-16
SLIDE 16

Why do developers continue to make stupid and lazy mistakes?

7

slide-17
SLIDE 17

Why do developers continue to make stupid and lazy mistakes? How can we make secure programming easier?

8

slide-18
SLIDE 18

POSSIBLE SOLUTIONS

9

slide-19
SLIDE 19

POSSIBLE SOLUTIONS

More/Better Education

9

slide-20
SLIDE 20

POSSIBLE SOLUTIONS

More/Better Education Better APIs

9

slide-21
SLIDE 21

POSSIBLE SOLUTIONS

More/Better Education Better APIs Better documentation

9

slide-22
SLIDE 22

POSSIBLE SOLUTIONS

More/Better Education Better APIs Better documentation Automation

9

slide-23
SLIDE 23

POSSIBLE SOLUTIONS

More/Better Education Better APIs Better documentation Automation Etc

9

slide-24
SLIDE 24

POSSIBLE SOLUTIONS

More/Better Education Better APIs Better documentation Automation Etc

How can we improve the effectiveness of these solutions?

9

slide-25
SLIDE 25

IN ORDER TO IMPROVE THESE SOLUTIONS, WE NEED TO UNDERSTAND THE TYPES, CAUSES, AND PERVASIVENESS OF VULNERABILITIES.

10

slide-26
SLIDE 26

HOW CAN WE MEASURE THIS?

Field studies Field surveys Lab studies

11

slide-27
SLIDE 27

FIELD STUDIES

Immerse ourselves in the “real world” to observe and collect data

12

slide-28
SLIDE 28

FIELD STUDIES

Immerse ourselves in the “real world” to observe and collect data Pros:

12

slide-29
SLIDE 29

FIELD STUDIES

Immerse ourselves in the “real world” to observe and collect data Pros: We can see what happens in the real world

12

slide-30
SLIDE 30

FIELD STUDIES

Immerse ourselves in the “real world” to observe and collect data Pros: We can see what happens in the real world Cons:

12

slide-31
SLIDE 31

FIELD STUDIES

Immerse ourselves in the “real world” to observe and collect data Pros: We can see what happens in the real world Cons: Hard to get access to

12

slide-32
SLIDE 32

FIELD STUDIES

Immerse ourselves in the “real world” to observe and collect data Pros: We can see what happens in the real world Cons: Hard to get access to Hard to generalize site specific data

12

slide-33
SLIDE 33

FIELD SURVEYS

CVEs, GitHub, etc

13

slide-34
SLIDE 34

FIELD SURVEYS

CVEs, GitHub, etc Pros:

13

slide-35
SLIDE 35

FIELD SURVEYS

CVEs, GitHub, etc Pros: Large datasets publicly available

13

slide-36
SLIDE 36

FIELD SURVEYS

CVEs, GitHub, etc Pros: Large datasets publicly available Data is already categorized

13

slide-37
SLIDE 37

FIELD SURVEYS

CVEs, GitHub, etc Pros: Large datasets publicly available Data is already categorized Cons:

13

slide-38
SLIDE 38

FIELD SURVEYS

CVEs, GitHub, etc Pros: Large datasets publicly available Data is already categorized Cons: Hard to understand why

13

slide-39
SLIDE 39

FIELD SURVEYS

CVEs, GitHub, etc Pros: Large datasets publicly available Data is already categorized Cons: Hard to understand why Hard to compare possibly unrelated data

13

slide-40
SLIDE 40

LAB STUDIES

Have people participate in a controlled experiment

14

slide-41
SLIDE 41

LAB STUDIES

Have people participate in a controlled experiment Pros:

14

slide-42
SLIDE 42

LAB STUDIES

Have people participate in a controlled experiment Pros: A lot of control over conditions

14

slide-43
SLIDE 43

LAB STUDIES

Have people participate in a controlled experiment Pros: A lot of control over conditions Cons:

14

slide-44
SLIDE 44

LAB STUDIES

Have people participate in a controlled experiment Pros: A lot of control over conditions Cons: Ecological validity

14

slide-45
SLIDE 45

LAB STUDIES

Have people participate in a controlled experiment Pros: A lot of control over conditions Cons: Ecological validity Potentially simple problems

14

slide-46
SLIDE 46

BUILD IT, BREAK IT, FIX IT

Secure programming contest

Ruef et al. , CCS 2016

15

slide-47
SLIDE 47

BUILD IT, BREAK IT, FIX IT

Secure programming contest Build-It Phase

Ruef et al. , CCS 2016

15

slide-48
SLIDE 48

BUILD IT, BREAK IT, FIX IT

Secure programming contest Build-It Phase 2 weeks

Ruef et al. , CCS 2016

15

slide-49
SLIDE 49

BUILD IT, BREAK IT, FIX IT

Secure programming contest Build-It Phase 2 weeks Develop to spec with open choices

Ruef et al. , CCS 2016

15

slide-50
SLIDE 50

BUILD IT, BREAK IT, FIX IT

Secure programming contest Build-It Phase 2 weeks Develop to spec with open choices Incentivized:

Ruef et al. , CCS 2016

15

slide-51
SLIDE 51

BUILD IT, BREAK IT, FIX IT

Secure programming contest Build-It Phase 2 weeks Develop to spec with open choices Incentivized: Make it performant

Ruef et al. , CCS 2016

15

slide-52
SLIDE 52

BUILD IT, BREAK IT, FIX IT

Secure programming contest Build-It Phase 2 weeks Develop to spec with open choices Incentivized: Make it performant Make it secure

Ruef et al. , CCS 2016

15

slide-53
SLIDE 53

BUILD IT, BREAK IT, FIX IT

Break-It Phase

Ruef et al. , CCS 2016

16

slide-54
SLIDE 54

BUILD IT, BREAK IT, FIX IT

Break-It Phase Get other teams’ source code

Ruef et al. , CCS 2016

16

slide-55
SLIDE 55

BUILD IT, BREAK IT, FIX IT

Break-It Phase Get other teams’ source code Attack breadth of submissions

Ruef et al. , CCS 2016

16

slide-56
SLIDE 56

BUILD IT, BREAK IT, FIX IT

Break-It Phase Get other teams’ source code Attack breadth of submissions Find unique vulnerabilities

Ruef et al. , CCS 2016

16

slide-57
SLIDE 57

BUILD IT, BREAK IT, FIX IT

Break-It Phase Get other teams’ source code Attack breadth of submissions Find unique vulnerabilities Prioritize security bugs over correctness

Ruef et al. , CCS 2016

16

slide-58
SLIDE 58

BUILD IT, BREAK IT, FIX IT

Break-It Phase Get other teams’ source code Attack breadth of submissions Find unique vulnerabilities Prioritize security bugs over correctness Fix-It Phase

Ruef et al. , CCS 2016

16

slide-59
SLIDE 59

BUILD IT, BREAK IT, FIX IT

Break-It Phase Get other teams’ source code Attack breadth of submissions Find unique vulnerabilities Prioritize security bugs over correctness Fix-It Phase Make fixes and get points back

Ruef et al. , CCS 2016

16

slide-60
SLIDE 60

SECURE LOG PROBLEM

log:

Event Log Time User Action Where 17

slide-61
SLIDE 61

SECURE LOG PROBLEM

./logappend –T 0800 –K XDFLKJSLJDLJFLKJLSDF –E Bob -A –R Gallery log

log:

Event Log Time User Action Where 17

slide-62
SLIDE 62

SECURE LOG PROBLEM

./logappend –T 0800 –K XDFLKJSLJDLJFLKJLSDF –E Bob -A –R Gallery log

log:

Event Log Time User Action Where 8:00 AM Bob Enter Gallery Event Log Time User Action Where 17

slide-63
SLIDE 63

SECURE LOG PROBLEM

./logappend –T 0800 –K XDFLKJSLJDLJFLKJLSDF –E Bob -A –R Gallery log ./logappend –T 0801 –K XDFLKJSLJDLJFLKJLSDF –E Alice -A –R Office log

log:

Event Log Time User Action Where 8:00 AM Bob Enter Gallery Event Log Time User Action Where 17

slide-64
SLIDE 64

SECURE LOG PROBLEM

./logappend –T 0800 –K XDFLKJSLJDLJFLKJLSDF –E Bob -A –R Gallery log ./logappend –T 0801 –K XDFLKJSLJDLJFLKJLSDF –E Alice -A –R Office log

log:

Event Log Time User Action Where 8:00 AM Bob Enter Gallery 8:01 AM Alice Enter Office Event Log Time User Action Where 8:00 AM Bob Enter Gallery Event Log Time User Action Where 17

slide-65
SLIDE 65

SECURE LOG PROBLEM

./logappend –T 0800 –K XDFLKJSLJDLJFLKJLSDF –E Bob -A –R Gallery log ./logappend –T 0801 –K XDFLKJSLJDLJFLKJLSDF –E Alice -A –R Office log

log:

./logread –K XDFLKJSLJDLJFLKJLSDF –R –E Alice log

Event Log Time User Action Where 8:00 AM Bob Enter Gallery 8:01 AM Alice Enter Office Event Log Time User Action Where 8:00 AM Bob Enter Gallery Event Log Time User Action Where 17

slide-66
SLIDE 66

SECURE LOG PROBLEM

./logappend –T 0800 –K XDFLKJSLJDLJFLKJLSDF –E Bob -A –R Gallery log ./logappend –T 0801 –K XDFLKJSLJDLJFLKJLSDF –E Alice -A –R Office log

log:

./logread –K XDFLKJSLJDLJFLKJLSDF –R –E Alice log Office

Event Log Time User Action Where 8:00 AM Bob Enter Gallery 8:01 AM Alice Enter Office Event Log Time User Action Where 8:00 AM Bob Enter Gallery Event Log Time User Action Where 17

slide-67
SLIDE 67

SECURE LOG PROBLEM

./logappend –T 0800 –K XDFLKJSLJDLJFLKJLSDF –E Bob -A –R Gallery log ./logappend –T 0801 –K XDFLKJSLJDLJFLKJLSDF –E Alice -A –R Office log

log:

./logread –K XDFLKJSLJDLJFLKJLSDF –R –E Alice log Office

Event Log Time User Action Where 8:00 AM Bob Enter Gallery 8:01 AM Alice Enter Office Event Log Time User Action Where 8:00 AM Bob Enter Gallery Event Log Time User Action Where 17

slide-68
SLIDE 68

SECURE LOG PROBLEM

./logappend –T 0800 –K XDFLKJSLJDLJFLKJLSDF –E Bob -A –R Gallery log ./logappend –T 0801 –K XDFLKJSLJDLJFLKJLSDF –E Alice -A –R Office log

log:

./logread –K XDFLKJSLJDLJFLKJLSDF –R –E Alice log Office

Event Log Time User Action Where 8:00 AM Bob Enter Gallery 8:01 AM Alice Enter Office

X

Event Log Time User Action Where 8:00 AM Bob Enter Gallery Event Log Time User Action Where 17

slide-69
SLIDE 69

SECURE COMMUNICATIONS PROBLEM

18

slide-70
SLIDE 70

SECURE COMMUNICATIONS PROBLEM

./bank –s auth

18

slide-71
SLIDE 71

SECURE COMMUNICATIONS PROBLEM

auth: XDFLKJSLJDLJFLKJLSDF

./bank –s auth

18

slide-72
SLIDE 72

SECURE COMMUNICATIONS PROBLEM

auth: XDFLKJSLJDLJFLKJLSDF

./bank –s auth

18

slide-73
SLIDE 73

SECURE COMMUNICATIONS PROBLEM

auth: XDFLKJSLJDLJFLKJLSDF

./bank –s auth ./atm –s auth –c card –a bob –n 1000

18

slide-74
SLIDE 74

SECURE COMMUNICATIONS PROBLEM

auth: XDFLKJSLJDLJFLKJLSDF

./bank –s auth

card: DFLLKSDF

./atm –s auth –c card –a bob –n 1000

bob balance:

1000

18

slide-75
SLIDE 75

SECURE COMMUNICATIONS PROBLEM

auth: XDFLKJSLJDLJFLKJLSDF

./bank –s auth

card: DFLLKSDF

./atm –s auth –c card –a bob –n 1000 ./atm –s auth –c card –a bob –d 50

bob balance:

1000 1050

18

slide-76
SLIDE 76

SECURE COMMUNICATIONS PROBLEM

auth: XDFLKJSLJDLJFLKJLSDF

./bank –s auth

card: DFLLKSDF

./atm –s auth –c card –a bob –n 1000 ./atm –s auth –c card –a bob –d 50 ./atm –s auth –c card –a bob –w 600

bob balance:

1000 1050 450

18

slide-77
SLIDE 77

SECURE COMMUNICATIONS PROBLEM

auth: XDFLKJSLJDLJFLKJLSDF

./bank –s auth

card: DFLLKSDF

./atm –s auth –c card –a bob –n 1000 ./atm –s auth –c card –a bob –d 50 ./atm –s auth –c card –a bob –w 600

bob balance:

1000 1050 450

18

slide-78
SLIDE 78

SECURE COMMUNICATIONS PROBLEM

auth: XDFLKJSLJDLJFLKJLSDF

./bank –s auth

card: DFLLKSDF

./atm –s auth –c card –a bob –n 1000 ./atm –s auth –c card –a bob –d 50 ./atm –s auth –c card –a bob –w 600

bob balance:

1000 1050 450

18

slide-79
SLIDE 79

MULTIUSER DATABASE PROBLEM

19

slide-80
SLIDE 80

MULTIUSER DATABASE PROBLEM

as principal admin password "admin" do create principal alice "alices_password" set msg = "Hi Alice. Good luck in Build it, Break it, Fix it!" set delegation msg admin read -> alice return "success" ***

19

slide-81
SLIDE 81

MULTIUSER DATABASE PROBLEM

as principal admin password "admin" do create principal alice "alices_password" set msg = "Hi Alice. Good luck in Build it, Break it, Fix it!" set delegation msg admin read -> alice return "success" *** as principal alice password ”alices_password" do return msg ***

19

slide-82
SLIDE 82

MULTIUSER DATABASE PROBLEM

as principal admin password "admin" do create principal alice "alices_password" set msg = "Hi Alice. Good luck in Build it, Break it, Fix it!" set delegation msg admin read -> alice return "success" *** as principal alice password ”alices_password" do return msg *** as principal bob password ”bobs_password" do return msg ***

19

slide-83
SLIDE 83

RESEARCH QUESTIONS

20

slide-84
SLIDE 84

RESEARCH QUESTIONS

What types of vulnerabilities do developers introduce?

20

slide-85
SLIDE 85

RESEARCH QUESTIONS

What types of vulnerabilities do developers introduce? How severe are the vulnerabilities? If exploited, what is the effect

  • n the system?

20

slide-86
SLIDE 86

RESEARCH QUESTIONS

What types of vulnerabilities do developers introduce? How severe are the vulnerabilities? If exploited, what is the effect

  • n the system?

How exploitable are the vulnerabilities? What level of insight is required and how much work is necessary?

20

slide-87
SLIDE 87

What types of vulnerabilities do developers introduce? How severe are the vulnerabilities? If exploited, what is the effect

  • n the system?

How exploitable are the vulnerabilities? What level of insight is required and how much work is necessary?

RESEARCH QUESTIONS

21

slide-88
SLIDE 88

ANALYSIS APPROACH

22

slide-89
SLIDE 89

ANALYSIS APPROACH

Examine projects and associated exploits in detail

22

slide-90
SLIDE 90

ANALYSIS APPROACH

Examine projects and associated exploits in detail Iterative open coding

22

slide-91
SLIDE 91

ANALYSIS APPROACH

Examine projects and associated exploits in detail Iterative open coding Two independent researchers with high reliability

22

slide-92
SLIDE 92

ANALYSIS APPROACH

Examine projects and associated exploits in detail Iterative open coding Two independent researchers with high reliability 76 projects with 866 submitted exploits

22

slide-93
SLIDE 93

ANALYSIS APPROACH

Examine projects and associated exploits in detail Iterative open coding Two independent researchers with high reliability 76 projects with 866 submitted exploits Both qualitative and quantitative analysis performed

22

slide-94
SLIDE 94

RESULTS

23

slide-95
SLIDE 95

Mistake

Vulnerability classes

No implementation Misunderstanding

Intuitive Bad Choice Conceptual Error Unintuitive

24

slide-96
SLIDE 96

Vulnerability classes

No implementation

25

slide-97
SLIDE 97

Vulnerability classes

No implementation

Intuitive

  • Missed something “Intuitive”

26

slide-98
SLIDE 98

Vulnerability classes

No implementation

Intuitive

  • Missed something “Intuitive”
  • No encryption (log, ATM)

26

slide-99
SLIDE 99

Vulnerability classes

No implementation

Intuitive

  • Missed something “Intuitive”
  • No encryption (log, ATM)
  • No access control (MD)

26

slide-100
SLIDE 100

Vulnerability classes

No implementation

Intuitive Unintuitive

  • Missed something “Intuitive”
  • No encryption (log, ATM)
  • No access control (MD)
  • Missed something “Unintuitive”
  • No MAC (log)

27

slide-101
SLIDE 101

Vulnerability classes

No implementation

Intuitive Unintuitive

  • Missed something “Intuitive”
  • No encryption (log, ATM)
  • No access control (MD)
  • Missed something “Unintuitive”
  • No MAC (log)
  • Side-channel leakage (ATM,

MD)

27

slide-102
SLIDE 102

Vulnerability classes

No implementation

Intuitive Unintuitive

  • Missed something “Intuitive”
  • No encryption (log, ATM)
  • No access control (MD)
  • Missed something “Unintuitive”
  • No MAC (log)
  • Side-channel leakage (ATM,

MD)

  • No replay prevention (ATM)

27

slide-103
SLIDE 103

Vulnerability classes

Misunderstanding

28

slide-104
SLIDE 104

Vulnerability classes

Misunderstanding

Bad Choice

  • Made a “Bad Choice”

29

slide-105
SLIDE 105

Vulnerability classes

Misunderstanding

Bad Choice

  • Made a “Bad Choice”
  • Weak algorithms

(log, ATM)

29

slide-106
SLIDE 106

Vulnerability classes

Misunderstanding

Bad Choice

  • Made a “Bad Choice”
  • Weak algorithms

(log, ATM)

  • Homemade

encryption (log, ATM)

29

slide-107
SLIDE 107

Vulnerability classes

Misunderstanding

Bad Choice

  • Made a “Bad Choice”
  • Weak algorithms

(log, ATM)

  • Homemade

encryption (log, ATM)

  • strcpy (log, ATM,

MD)

29

slide-108
SLIDE 108

Vulnerability classes

Misunderstanding

Bad Choice Conceptual Error

  • Made a “Conceptual

Error”

30

slide-109
SLIDE 109

Vulnerability classes

Misunderstanding

Bad Choice Conceptual Error

  • Made a “Conceptual

Error”

  • Fixed value (log,

ATM, MD)

30

slide-110
SLIDE 110

31

slide-111
SLIDE 111

31

slide-112
SLIDE 112

31

slide-113
SLIDE 113

Vulnerability classes

Misunderstanding

Bad Choice Conceptual Error

  • Made a “Conceptual

Error”

  • Fixed value (log,

ATM, MD)

32

slide-114
SLIDE 114

Vulnerability classes

Misunderstanding

Bad Choice Conceptual Error

  • Made a “Conceptual

Error”

  • Fixed value (log,

ATM, MD)

  • Lacking sufficient

randomness (log, ATM)

32

slide-115
SLIDE 115

Vulnerability classes

Misunderstanding

Bad Choice Conceptual Error

  • Made a “Conceptual

Error”

  • Fixed value (log,

ATM, MD)

  • Lacking sufficient

randomness (log, ATM)

  • Disabling protections

in library (log)

32

slide-116
SLIDE 116

33

slide-117
SLIDE 117

33

slide-118
SLIDE 118

Mistake

Vulnerability classes

  • Made a “Mistake”

34

slide-119
SLIDE 119

Mistake

Vulnerability classes

  • Made a “Mistake”
  • Control flow mistake (ATM, MD)

34

slide-120
SLIDE 120

Mistake

Vulnerability classes

  • Made a “Mistake”
  • Control flow mistake (ATM, MD)
  • Skipped algorithmic step (ATM, MD)

34

slide-121
SLIDE 121

Mistake

Vulnerability classes

  • Made a “Mistake”
  • Control flow mistake (ATM, MD)
  • Skipped algorithmic step (ATM, MD)

34

slide-122
SLIDE 122

Mistake

Vulnerability classes

  • Made a “Mistake”
  • Control flow mistake (ATM, MD)
  • Skipped algorithmic step (ATM, MD)

34

slide-123
SLIDE 123

PREVALENCE

Percentage of projects that introduced a mistake, misunderstanding, and no implementation vulnerability grouped by problem:

20 % of projects 80 40 60 Secure log Secure communication Multiuser database Totals

Mistake Misund. No Impl.

35

slide-124
SLIDE 124

PREVALENCE

Percentage of projects that introduced a mistake, misunderstanding, and no implementation vulnerability grouped by problem:

20 % of projects 80 40 60 Secure log Secure communication Multiuser database Totals

Mistake Misund. No Impl.

35

slide-125
SLIDE 125

PREVALENCE

Percentage of projects that introduced a mistake, misunderstanding, and no implementation vulnerability grouped by problem:

20 % of projects 80 40 60 Secure log Secure communication Multiuser database Totals

Mistake Misund. No Impl.

35

slide-126
SLIDE 126

PREVALENCE

Intuitive Unituitive Bad choice Concept error

14 28 41 55

No Impl. Misund.

% of Projects that introduced each subclass

% of projects

36

slide-127
SLIDE 127

PREVALENCE

Intuitive Unituitive Bad choice Concept error

14 28 41 55

No Impl. Misund.

% of Projects that introduced each subclass

% of projects

36

slide-128
SLIDE 128

PREVALENCE

Intuitive Unituitive Bad choice Concept error

14 28 41 55

No Impl. Misund.

% of Projects that introduced each subclass

% of projects

36

slide-129
SLIDE 129

PREVALENCE

Intuitive Unituitive Bad choice Concept error

14 28 41 55

No Impl. Misund.

% of Projects that introduced each subclass

% of projects

36

slide-130
SLIDE 130

PREVALENCE

Intuitive Unituitive Bad choice Concept error

14 28 41 55

No Impl. Misund.

% of Projects that introduced each subclass

% of projects

36

slide-131
SLIDE 131

PREVALENCE

Intuitive Unituitive Bad choice Concept error

14 28 41 55

No Impl. Misund.

% of Projects that introduced each subclass

% of projects

36

slide-132
SLIDE 132

PREVALENCE

Intuitive Unituitive Bad choice Concept error

14 28 41 55

No Impl. Misund.

% of Projects that introduced each subclass

% of projects

36

slide-133
SLIDE 133

TRENDS WITHIN MISTAKES

37

slide-134
SLIDE 134

TRENDS WITHIN MISTAKES

Complexity breeds mistakes.

37

slide-135
SLIDE 135

TRENDS WITHIN MISTAKES

Complexity breeds mistakes. Most common in the multi-user database problem (most complex) and least common in log problem (least complex)

37

slide-136
SLIDE 136

TRENDS WITHIN MISTAKES

Complexity breeds mistakes. Most common in the multi-user database problem (most complex) and least common in log problem (least complex) Writing security checks once reduced mistakes

37

slide-137
SLIDE 137

TRENDS WITHIN MISTAKES

Complexity breeds mistakes. Most common in the multi-user database problem (most complex) and least common in log problem (least complex) Writing security checks once reduced mistakes

37

Almost all mistakes were found in the Break-It phase

slide-138
SLIDE 138

RECOMMENDATIONS

38

slide-139
SLIDE 139

RECOMMENDATIONS

Simplify API design Build in security primitives and focus on common use-cases

38

slide-140
SLIDE 140

RECOMMENDATIONS

Simplify API design Build in security primitives and focus on common use-cases Indicate security impact of non-default use in API Documentation Explain the negative effects of turning off certain things

38

slide-141
SLIDE 141

RECOMMENDATIONS

Simplify API design Build in security primitives and focus on common use-cases Indicate security impact of non-default use in API Documentation Explain the negative effects of turning off certain things Vulnerability Analysis Tools More emphasis on design-level conceptual issues

38

slide-142
SLIDE 142

SUMMARY

39

slide-143
SLIDE 143

SUMMARY

Developers struggle with security concepts

39

slide-144
SLIDE 144

SUMMARY

Developers struggle with security concepts Mostly knew they needed security and picked the right tools

39

slide-145
SLIDE 145

SUMMARY

Developers struggle with security concepts Mostly knew they needed security and picked the right tools Didn’t know all the security requirements (Unintuitive) or all the implementation details (Conceptual Error)

39

slide-146
SLIDE 146

SUMMARY

Developers struggle with security concepts Mostly knew they needed security and picked the right tools Didn’t know all the security requirements (Unintuitive) or all the implementation details (Conceptual Error) Mistakes happen, but can be reduced through code review and best practices

39

slide-147
SLIDE 147

SUMMARY

Developers struggle with security concepts Mostly knew they needed security and picked the right tools Didn’t know all the security requirements (Unintuitive) or all the implementation details (Conceptual Error) Mistakes happen, but can be reduced through code review and best practices Improve API design, documentation, and automation to handle conceptual nuances

39

slide-148
SLIDE 148

SUMMARY

Developers struggle with security concepts Mostly knew they needed security and picked the right tools Didn’t know all the security requirements (Unintuitive) or all the implementation details (Conceptual Error) Mistakes happen, but can be reduced through code review and best practices Improve API design, documentation, and automation to handle conceptual nuances

39

Questions dvotipka@cs.umd.edu sec-professionals.cs.umd.edu