Accuracies and Biases in Modeling Password Guessability Blase Ur, - - PowerPoint PPT Presentation

▶
accuracies and biases in modeling password
SMART_READER_LITE
LIVE PREVIEW

Accuracies and Biases in Modeling Password Guessability Blase Ur, - - PowerPoint PPT Presentation

Oct 05, 2023 205 likes •1.35k views

Measuring Real-World Accuracies and Biases in Modeling Password Guessability Blase Ur, Sean M. Segreti, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Saranga Komanduri, Darya Kurilova, Michelle L. Mazurek, William Melicher, Richard Shay

slide-1
SLIDE 1

1

Blase Ur, Sean M. Segreti, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Saranga Komanduri, Darya Kurilova, Michelle L. Mazurek, William Melicher, Richard Shay

Measuring Real-World Accuracies and Biases in Modeling Password Guessability

slide-2
SLIDE 2

2

How strong is a particular password?

slide-3
SLIDE 3

3

iloveyou

slide-4
SLIDE 4

4

iloveyou

slide-5
SLIDE 5

5

n(c$JZX!zKc^bIAX^N

slide-6
SLIDE 6

6

j@mesb0nd007!

slide-7
SLIDE 7

7

Why Measure Password Strength?

slide-8
SLIDE 8

8

Why Measure Password Strength?

  • Eliminate bad passwords

– Organizational password audits

slide-9
SLIDE 9

9

Why Measure Password Strength?

  • Eliminate bad passwords

– Organizational password audits

  • Help users make better passwords
slide-10
SLIDE 10

10

Why Measure Password Strength?

  • Eliminate bad passwords

– Organizational password audits

  • Help users make better passwords

– Determine if interventions are effective

slide-11
SLIDE 11

11

Why Measure Password Strength?

  • Eliminate bad passwords

– Organizational password audits

  • Help users make better passwords

– Determine if interventions are effective – Provide users feedback

slide-12
SLIDE 12

12

Password-Strength Metrics

slide-13
SLIDE 13

13

Password-Strength Metrics

  • Statistical approaches

– Traditionally: Shannon entropy – Recently: α-guesswork

slide-14
SLIDE 14

14

Password-Strength Metrics

  • Statistical approaches

– Traditionally: Shannon entropy – Recently: α-guesswork

  • Disadvantages for researchers

– No per-password estimates – Huge sample required

slide-15
SLIDE 15

15

Parameterized Guessability

  • How many guesses a particular cracking

algorithm with particular training data would take to guess a password

slide-16
SLIDE 16

16

j@mesb0nd007!

Guess # 366,163,847,194

slide-17
SLIDE 17

17

Guess # past cutoff

n(c$JZX!zKc^bIAX^N

slide-18
SLIDE 18

18

Guessability Plots

slide-19
SLIDE 19

19

Guessability Plots

slide-20
SLIDE 20

20

Guessability Plots

slide-21
SLIDE 21

21

Advantages of Guessability

  • Straightforward
  • Models an attacker
  • Per-password strength estimates
slide-22
SLIDE 22

22

Guessability in Practice

slide-23
SLIDE 23

23

Guessability in Practice

slide-24
SLIDE 24

24

Single Cracking Approach

slide-25
SLIDE 25

25

Default Configuration

slide-26
SLIDE 26

26

Questions About Guessability

slide-27
SLIDE 27

27

Questions About Guessability

1) How does guessability used in research compare to an attack by professionals?

slide-28
SLIDE 28

28

Questions About Guessability

1) How does guessability used in research compare to an attack by professionals? 2) Would substituting another cracking approach impact research results?

slide-29
SLIDE 29

29

Approach

slide-30
SLIDE 30

30

4 password sets 5 password-cracking approaches

Approach

password iloveyou teamo123 … passwordpassword 1234567812345678 !1@2#3$4%5^6&7*8 … Pa$$w0rd iLov3you! 1QaZ2W@x … pa$$word1234 12345678asDF !q1q!q1q!q1q …

slide-31
SLIDE 31

31

Four Password Sets

slide-32
SLIDE 32

32

Four Password Sets

  • Basic (3,062): 8+ characters

password

slide-33
SLIDE 33

33

Four Password Sets

  • Basic (3,062): 8+ characters

password

  • Complex (3,000): 8+ characters, 4 classes

Pa$$w0rd

slide-34
SLIDE 34

34

Four Password Sets

  • Basic (3,062): 8+ characters

password

  • Complex (3,000): 8+ characters, 4 classes

Pa$$w0rd

  • LongBasic (2,054): 16+ characters

passwordpassword

slide-35
SLIDE 35

35

Four Password Sets

  • Basic (3,062): 8+ characters

password

  • Complex (3,000): 8+ characters, 4 classes

Pa$$w0rd

  • LongBasic (2,054): 16+ characters

passwordpassword

  • LongComplex (990): 12+ characters, 3+ classes

pa$$word1234

slide-36
SLIDE 36

36

Five Cracking Approaches

  • John the Ripper
  • Hashcat
  • Markov models
  • Probabilistic Context-Free Grammar
  • Professionals
slide-37
SLIDE 37

37

  • Guesses variants of input wordlist

John the Ripper

slide-38
SLIDE 38

38

  • Guesses variants of input wordlist
  • Wordlist mode requires:

– Wordlist (passwords and dictionary entries) – Mangling rules

John the Ripper

slide-39
SLIDE 39

39

  • Guesses variants of input wordlist
  • Wordlist mode requires:

– Wordlist (passwords and dictionary entries) – Mangling rules

  • Speed: Fast

John the Ripper

slide-40
SLIDE 40

40

  • Guesses variants of input wordlist
  • Wordlist mode requires:

– Wordlist (passwords and dictionary entries) – Mangling rules

  • Speed: Fast

– 1013 guesses

John the Ripper

slide-41
SLIDE 41

41

  • Guesses variants of input wordlist
  • Wordlist mode requires:

– Wordlist (passwords and dictionary entries) – Mangling rules

  • Speed: Fast

– 1013 guesses

  • “JTR”

John the Ripper

slide-42
SLIDE 42

42

John the Ripper

wordlist rules guesses

slide-43
SLIDE 43

43

John the Ripper

usenix security wordlist rules guesses

slide-44
SLIDE 44

44

John the Ripper

usenix security [ ] [add 1 at end] [change e to 3] wordlist rules guesses

slide-45
SLIDE 45

45

usenix security [ ] [add 1 at end] [change e to 3] usenix security usenix1 security1 us3nix s3curity

John the Ripper

wordlist rules guesses

slide-46
SLIDE 46

46

usenix security [ ] [add 1 at end] [change e to 3] usenix security usenix1 security1 us3nix s3curity

John the Ripper

wordlist rules guesses

slide-47
SLIDE 47

47

usenix security [ ] [add 1 at end] [change e to 3] usenix security usenix1 security1 us3nix s3curity

John the Ripper

wordlist rules guesses

slide-48
SLIDE 48

48

Hashcat

  • Guesses variants of input wordlist
slide-49
SLIDE 49

49

Hashcat

  • Guesses variants of input wordlist
  • Wordlist mode requires:

– Wordlist (passwords and dictionary entries) – Mangling rules

slide-50
SLIDE 50

50

Hashcat

  • Guesses variants of input wordlist
  • Wordlist mode requires:

– Wordlist (passwords and dictionary entries) – Mangling rules

  • Speed: Fast
slide-51
SLIDE 51

51

Hashcat

  • Guesses variants of input wordlist
  • Wordlist mode requires:

– Wordlist (passwords and dictionary entries) – Mangling rules

  • Speed: Fast

– 1013 guesses

slide-52
SLIDE 52

52

Hashcat

wordlist rules guesses

slide-53
SLIDE 53

53

Hashcat

usenix security [ ] [add 1 at end] [change e to 3] wordlist rules guesses

slide-54
SLIDE 54

54

usenix security [ ] [add 1 at end] [change e to 3] usenix usenix1 us3nix security security1 s3curity

Hashcat

wordlist rules guesses

slide-55
SLIDE 55

55

usenix security [ ] [add 1 at end] [change e to 3] usenix usenix1 us3nix security security1 s3curity

Hashcat

wordlist rules guesses

slide-56
SLIDE 56

56

Markov Models

  • Predicts future characters from previous
slide-57
SLIDE 57

57

Markov Models

  • Predicts future characters from previous
  • Approach requires weighted data:

– Passwords – Dictionaries

slide-58
SLIDE 58

58

Markov Models

  • Predicts future characters from previous
  • Approach requires weighted data:

– Passwords – Dictionaries

  • Ma et al. IEEE S&P 2014
slide-59
SLIDE 59

59

Markov Models

  • Predicts future characters from previous
  • Approach requires weighted data:

– Passwords – Dictionaries

  • Ma et al. IEEE S&P 2014
  • Speed: Slow
slide-60
SLIDE 60

60

Markov Models

  • Predicts future characters from previous
  • Approach requires weighted data:

– Passwords – Dictionaries

  • Ma et al. IEEE S&P 2014
  • Speed: Slow

– 1010 guesses

slide-61
SLIDE 61

61

Probabilistic Context-Free Grammar

  • Generate password grammar

– Structures – Terminals

slide-62
SLIDE 62

62

Probabilistic Context-Free Grammar

  • Generate password grammar

– Structures – Terminals

  • Kelley et al. IEEE S&P 2012

– Based on Weir et al. IEEE S&P 2009

slide-63
SLIDE 63

63

Probabilistic Context-Free Grammar

  • Generate password grammar

– Structures – Terminals

  • Kelley et al. IEEE S&P 2012

– Based on Weir et al. IEEE S&P 2009

  • Speed: Slow Medium
slide-64
SLIDE 64

64

Probabilistic Context-Free Grammar

  • Generate password grammar

– Structures – Terminals

  • Kelley et al. IEEE S&P 2012

– Based on Weir et al. IEEE S&P 2009

  • Speed: Slow Medium

– 1014 guesses

slide-65
SLIDE 65

65

Probabilistic Context-Free Grammar

  • Generate password grammar

– Structures – Terminals

  • Kelley et al. IEEE S&P 2012

– Based on Weir et al. IEEE S&P 2009

  • Speed: Slow Medium

– 1014 guesses

  • “PCFG”
slide-66
SLIDE 66

66

Professionals (“Pros”)

slide-67
SLIDE 67

67

Professionals (“Pros”)

  • Contracted KoreLogic

– Password audits for Fortune 500 companies – Run DEF CON “Crack Me If You Can”

slide-68
SLIDE 68

68

Professionals (“Pros”)

  • Contracted KoreLogic

– Password audits for Fortune 500 companies – Run DEF CON “Crack Me If You Can”

  • Proprietary wordlists and configurations
slide-69
SLIDE 69

69

Professionals (“Pros”)

  • Contracted KoreLogic

– Password audits for Fortune 500 companies – Run DEF CON “Crack Me If You Can”

  • Proprietary wordlists and configurations

– 1014 guesses

slide-70
SLIDE 70

70

Professionals (“Pros”)

  • Contracted KoreLogic

– Password audits for Fortune 500 companies – Run DEF CON “Crack Me If You Can”

  • Proprietary wordlists and configurations

– 1014 guesses – Manually tuned, updated

slide-71
SLIDE 71

71

4 password sets 5 approaches

Approach

password iloveyou teamo123 … passwordpassword 1234567812345678 !1@2#3$4%5^6&7*8 … Pa$$w0rd iLov3you! 1QaZ2W@x … pa$$word1234 12345678asDF !q1q!q1q!q1q …

slide-72
SLIDE 72

72

Outline of Results

  • Importance of Configuration
  • Comparison of Approaches
  • Impact on Research Analyses
slide-73
SLIDE 73

73

Configuration Is Crucial

LongComplex

slide-74
SLIDE 74

74

Configuration Is Crucial

LongComplex

slide-75
SLIDE 75

75

Configuration Is Crucial

LongComplex

slide-76
SLIDE 76

76

Configuration Is Crucial

LongComplex

slide-77
SLIDE 77

77

Configuration Is Crucial

LongComplex

slide-78
SLIDE 78

78

Outline of Results

  • Importance of Configuration
  • Comparison of Approaches
  • Impact on Research Analyses
slide-79
SLIDE 79

79

Comparison for Basic Passwords

slide-80
SLIDE 80

80

Comparison for Basic Passwords

slide-81
SLIDE 81

81

Comparison for Basic Passwords

slide-82
SLIDE 82

82

Comparison for Basic Passwords

slide-83
SLIDE 83

83

Comparison for Basic Passwords

slide-84
SLIDE 84

84

Comparison for Basic Passwords

slide-85
SLIDE 85

85

Comparison for Complex Passwords

slide-86
SLIDE 86

86

Comparison for Complex Passwords

slide-87
SLIDE 87

87

Comparison for Complex Passwords

slide-88
SLIDE 88

88

Comparison for Complex Passwords

slide-89
SLIDE 89

89

Comparison for Complex Passwords

slide-90
SLIDE 90

90

Comparison for Complex Passwords

slide-91
SLIDE 91

91

Comparison for Complex Passwords

slide-92
SLIDE 92

92

Min_auto Conservative Proxy for Pros

slide-93
SLIDE 93

93

Outline of Results

  • Importance of Configuration
  • Comparison of Approaches
  • Impact on Research Analyses
slide-94
SLIDE 94

94

Impact on Research

  • Coarse-grained analyses
  • Fine-grained analyses
  • Analysis of one password
slide-95
SLIDE 95

95

Impact on Research

  • Coarse-grained analyses same results
  • Fine-grained analyses
  • Analysis of one password
slide-96
SLIDE 96

96

Impact on Research

  • Coarse-grained analyses same results
  • Fine-grained analyses different
  • Analysis of one password
slide-97
SLIDE 97

97

Impact on Research

  • Coarse-grained analyses same results
  • Fine-grained analyses different
  • Analysis of one password different
slide-98
SLIDE 98

98

Per-Password Highly Impacted

P@ssw0rd!

slide-99
SLIDE 99

99

Per-Password Highly Impacted

  • JTR guess # 801

P@ssw0rd!

slide-100
SLIDE 100

100

Per-Password Highly Impacted

  • JTR guess # 801
  • Not guessed in 1014 PCFG guesses

P@ssw0rd!

slide-101
SLIDE 101

101

Per-Password Highly Impacted

  • JTR guess # 801
  • Not guessed in 1014 PCFG guesses

P@ssw0rd!

slide-102
SLIDE 102

102

Per-Password Highly Impacted

12345678password

slide-103
SLIDE 103

103

Per-Password Highly Impacted

  • PCFG guess # 130,555

12345678password

slide-104
SLIDE 104

104

Per-Password Highly Impacted

  • PCFG guess # 130,555
  • Not guessed in 1010 JTR guesses

12345678password

slide-105
SLIDE 105

105

Conclusions

slide-106
SLIDE 106

106

Conclusions

  • Running a single approach is insufficient

– Especially out of the box

slide-107
SLIDE 107

107

Conclusions

  • Running a single approach is insufficient

– Especially out of the box

  • Min_auto conservative proxy for pros
slide-108
SLIDE 108

108

Conclusions

  • Running a single approach is insufficient

– Especially out of the box

  • Min_auto conservative proxy for pros
  • Coarse-grained analyses same results
  • Fine-grained analyses different
  • Analysis of one password different
slide-109
SLIDE 109

109

Password Guessability Service (PGS)

  • Guessability of plaintext passwords

https://pgs.ece.cmu.edu

slide-110
SLIDE 110

110

Password Guessability Service (PGS)

  • Guessability of plaintext passwords

https://pgs.ece.cmu.edu

asdfF123# P@ssw0rd! Qwertyuiop!1 …

slide-111
SLIDE 111

111

Password Guessability Service (PGS)

  • Guessability of plaintext passwords

https://pgs.ece.cmu.edu

asdfF123# P@ssw0rd! Qwertyuiop!1 … "Guess #", "Password" "127188816", "Qwertyuiop!1" "1853004462", "asdfF123#" "2251762491", "P@ssw0rd!" ...

slide-112
SLIDE 112

112

Password Guessability Service (PGS)

  • Guessability of plaintext passwords

https://pgs.ece.cmu.edu

"Guess #", "Password" "127188816", "Qwertyuiop!1" "1853004462", "asdfF123#" "2251762491", "P@ssw0rd!" ... asdfF123# P@ssw0rd! Qwertyuiop!1 …

slide-113
SLIDE 113

113

Blase Ur, Sean M. Segreti, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Saranga Komanduri, Darya Kurilova, Michelle L. Mazurek, William Melicher, Richard Shay

https://pgs.ece.cmu.edu

Measuring Real-World Accuracies and Biases in Modeling Password Guessability

slide-114
SLIDE 114

114

  • Per Thorsheim and Jeremi Gosney (PasswordsCon)
  • Hashcat / JTR developers
  • Matt Marx (@tehnlulz)
  • Jerry Ma, Weining Yang, Ninghui Li (Purdue)
  • KoreLogic (@CrackMeIfYouCan)
  • Dustin Heywood (@Evil_Mog)
  • Jonathan Bees
  • Michael Stroucken and Chuck Cranor (CMU)

114

Acknowledgments