Crying Wolf: An Empirical Study of SSL Warning Effectiveness Joshua - - PowerPoint PPT Presentation

▶
crying wolf an empirical study of ssl warning
SMART_READER_LITE
LIVE PREVIEW

Crying Wolf: An Empirical Study of SSL Warning Effectiveness Joshua - - PowerPoint PPT Presentation

Dec 07, 2023 215 likes •537 views

Crying Wolf: An Empirical Study of SSL Warning Effectiveness Joshua Sunshine Serge Egelman Hazim Almuhimedi Neha Atri Lorrie Faith Cranor C yLab U sable P rivacy and S ecurity Laboratory http://cups.cs.cmu.edu/ CyLab Usable Privacy and

slide-1
SLIDE 1

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 1

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/

Crying Wolf: An Empirical Study

  • f SSL Warning Effectiveness

Joshua Sunshine Serge Egelman Hazim Almuhimedi Neha Atri Lorrie Faith Cranor

slide-2
SLIDE 2

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 2

SSL Certificate Warnings

  • Browser’s warn about SSL Cert problems:

– Domain Mismatch – Unknown Certificate Authority – Expired

  • These warnings:

– May be user’s only protection – Commonly encountered when connecting to legitimate servers

slide-3
SLIDE 3

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 3

FF2 Warning

slide-4
SLIDE 4

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 4

FF2 Warning

Adapted from Jonathan Nightingale

slide-5
SLIDE 5

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 5

IE7 Warning

slide-6
SLIDE 6

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 6

FF3 Warning

slide-7
SLIDE 7

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 7

FF3 Warning

slide-8
SLIDE 8

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 8

FF3 Warning

slide-9
SLIDE 9

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 9

FF3 Warning

slide-10
SLIDE 10

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 10

Warning Design Strategies

  • Lessons from online survey:

– Context sensitivity – Prevent habituation – Avoid confusion with other, less serious, warnings

  • Warning science guidance:

– Avoid warnings when possible – Clearly explain risk – Provide straightforward instructions for avoiding the hazard

slide-11
SLIDE 11

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 11

Idea: Ask users a question

Multi-page warning

slide-12
SLIDE 12

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 12

Idea: Make risk obvious

Single-page warning

slide-13
SLIDE 13

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 13

Laboratory Study

  • 100 participants

– CMU students – Recruited by fliers, emails, and participant list

  • 5 Randomly-assigned conditions: FF2, FF3, IE7,

Single page custom warning and multi-page custom warning

  • Warning was triggered twice:

– Bank – Library catalog

slide-14
SLIDE 14

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 14

Laboratory Study

  • Users were instructed to find:

– Total area of Italy using Google – Account balance at bank website* – Price of Freakonomics at Amazon – Richistan call number with CMU library catalog*

*warning appeared

  • Alternate tasks provided

– Required calling or using a different site

  • Post-experiment survey on reactions
slide-15
SLIDE 15

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 15

Task Step 1

Use online banking (https://www.bank.com) to find your current account balance. Write down only the last two digits

  • f your account balance.

Alternate: Use automated phone banking (Phone: 1-888-555- 1212). Please use the campus phone in front of you and don’t forget to first dial ‘9.’ Please remember to “think aloud” as you complete this task.

slide-16
SLIDE 16

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 16

Task walkthrough

GO

slide-17
SLIDE 17

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 17

Task walkthrough

https://www.bank.com/

GO

slide-18
SLIDE 18

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 18

Task walkthrough

https://www.bank.com/

GO

slide-19
SLIDE 19

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 19

Task walkthrough

https://www.bank.com/

GO

slide-20
SLIDE 20

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 20

Task walkthrough

https://www.bank.com/

GO

BANK username: password:

sunshine

  • GO
slide-21
SLIDE 21

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 21

Task walkthrough alternate

https://www.bank.com/

GO

slide-22
SLIDE 22

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 22

Task walkthrough alternate

slide-23
SLIDE 23

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 23

Hypotheses

  • Participants would be likely to ignore the IE7

and FF2 warnings on both websites

  • Participants would be likely to obey the FF3

and our single-page warning on both websites

  • Participants who saw our multi-page warning

would obey on bank website, but continue to library website

slide-24
SLIDE 24

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 24

Bank Results

  • In risky situation, significantly fewer people heeded IE7 and

FF2 than other warnings

0% 20% 40% 60% 80% 100% FF2 FF3 IE7 1-page Multipage Ignored Warning

slide-25
SLIDE 25

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 25

Library Results

  • In low risk situation, almost all users overrode warnings except

in FF3 condition

0% 20% 40% 60% 80% 100% FF2 FF3 IE7 1-page Multipage Ignored Warning

slide-26
SLIDE 26

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 26

Library vs. Bank

  • In native warning conditions, no significant difference in

reactions at library and bank

  • In new warning conditions, users more likely to heed warnings

at bank than at library

0% 20% 40% 60% 80% 100% FF2 FF3 IE7 1-page Multipage Bank Library Ignored Warning

slide-27
SLIDE 27

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 27

Explain what to do

  • “Why did you choose to heed or ignore the

warning?”

  • Mentioned risk:

– FF2: 2 – FF3: 2 – IE7: 2 – Single-Page: 11

slide-28
SLIDE 28

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 28

Explain what to do

  • “What action(s) did you think the warning at

the bank wanted you to take?”

  • Wanted them not to proceed:

– FF2: 3 – FF3: 2 – IE7: 4 – Single-page: 10

slide-29
SLIDE 29

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 29

Making It Difficult

slide-30
SLIDE 30

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 30

Asking a Question

  • 15/20 participants answered correctly at bank

– 3 knowingly gave the wrong answer – 2 confused warning with server unavailable error

  • Critical Weakness: Finer grained origins attack

– attacker circumvents question by forcing connection to unintended website – See paper for details

  • Need a different context sensitive approach
slide-31
SLIDE 31

CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 31

Conclusion

  • We evaluated a wide class of warnings embodying

three solid strategies

  • Custom warnings conveyed risks and allowed users to

take risk into account when making a decision

  • Custom warnings were still not good enough
  • Need systems solutions that avoid warnings

altogether (e.g. Perspectives, ForceHTTPs)

– Need to evaluate false positive rate