Antri Kolani CS682 ADVANCED SECURITY TOPICS Two usability studies - PowerPoint PPT Presentation

Antri Kolani CS682 ADVANCED SECURITY TOPICS

Two usability studies held in 2011:  an Internet survey of 308 Android users, and  a laboratory study of 25 Android users  Study participants displayed low attention and comprehension rates 2

• Attention  Comprehension  Behavior 3

 Use of permissions from Android  Phone resources  Google’s role  Android Grayware 4

Final installation page and permission dialog 5

1. Attention switch and maintenance. 2. Comprehension and memory. 3. Attitudes and belief. 4. Motivation. 5. Behavior. 6

.  Decrease of the initial rate of participants  Completion rate  Advertisement for the survey 7

Screenshots of a quiz question and of permissions 8

 Craigslist ad  Requirement for participants to have android phone 9

1 . General Android usage questions 2. Installation of an application 3. Installation of a second application 4. Westin index questions. 5 . Participant’s recently used application 6. Details about past permission related behaviors 10

The last time you downloaded an Android application, what did you look at before deciding to download it?  17,5% of 308 respondents  40,5% of the 42 Privacy Fundamentalists  13,9% of the remaining 266 respondents 11

12

“ The last time you downloaded an Android application, what did you look at before deciding to download it?” 219 survey respondents saw review before installation. Of these, 193 respondents looked Market reviews 42 respondents looked other reviews on the Internet. 26 respondents looked both Internet and Market reviews. 13

14

Permission Comprehension Quiz 1. Free-Form Permission Descriptions 2. Specific Permission Comprehension 3. 15

16

To evaluate user understanding, graded participants’ freeform descriptions of permissions as follows:  Correct  Correct but overly broad  Incomplete  Incomplete and overly broad  Wrong  Unable to answer  Omitted 17

 SEND_SMS permission 18

 “Have you ever not installed an app because of permissions?”  Respondents were shown the following four choices: Yes, I didn’t like the permissions 1. Yes, there were too many permissions 2. No 3. I don’t know 4. 19

20

 Permission warnings  Current Android Permission system  Laboratory study participants  Reviews from users 21

1. Categories 2. Risks, Not Resources 3. Low-Risk Warnings 4. Absent Permissions 5. Optional Permissions 22

1. Timing 2. Reviews 3. Customization 23

 Effectiveness of Android permissions.  Android permissions fail to inform the majority of users  Minority of users demonstrated awareness and understanding of permissions 24

Alice in Warningland: A Large-Scale Field Study of Browser Security Warning Effectiveness 25

 Warnings to users  User clicks through a warning  User leaves the warning  Clickthrough rate  Telemetry mechanism 26

Focus on three types of browser security warnings:  Malware and Phishing  SSL warnings  Browser Release Channels 27

 Clickthrough Rate  Warning Mechanisms  Warning Design  Click Count 28

 Active warnings  Phishing websites  Egelman et al. study 29

 Critical step  Clickthrough Rate  Warning Design  Click Count  Dhamija study  Passive Indicators 30

 Mozilla and Google both follow rapid release cycles.  “stable” (Google Chrome) or “release” Mozilla Firefox)  Pre-release channels 31

 Measuring Clickthrough Rates  Ethics  Method Limitations 32

 Implemented metrics in both browsers  Bypassing warnings  Click through specific SSL errors .  Mozilla Firefox data set 33

 User shares usage data  Browser collects data  Browser periodically sends this pseudonymous data 34

 Private Data  Sampling Bias  Overrepresentation  Frames 35

 Clickthrough rates for malware warnings  Clickthrough rates for phishing warnings  Malware Rates by Date  Malware/Phishing Rates by Warning Type  Malware/Phishing Rates by Demographics  Malware/Phishing Rates by Browser 36

 Malware rates for Google Chrome  Clickthrough rates ranging  Mozilla Firefox malware warning clickthrough rate 37

 In Mozilla Firefox, higher clickthrough rate for phishing warnings than malware warnings 38

39

 Google Chrome and Mozilla Firefox stable users.  Mozilla Firefox’s warnings  Browsers have different demographics 40

 Clickthrough rates  SSL Rates by Demographic  SSL Rates by Browser  SSL Rates by Certificate Error Type  Additional SSL Metrics 41

 Nightly users  Firefox Linux users  Chrome Windows users 42

 Number of Clicks  Warning Appearance  Certificate Pinning  Remembering Exceptions  Demographics 43

 Google Chrome  Mozilla Firefox  Error Prevalence 44

 More Information  Add Exception Cancellation  Remember Exception 45

46

 Demographics  Number of Clicks  Warning Fatigue  More Information 47

 Clickthrough rates  Higher technical skill  Technically advanced users.  Studies of these users 48

 User behavior.  Simple Firefox warning. 49

 Common SSL errors 50

 Explanatory links such as “More Information” or “Learn More ” .  Designers of such links  Mozilla Firefox information about SSL errors  Google Chrome error details 51

 Google Chrome and Mozilla Firefox’s telemetry platforms  Browser security warnings can be successful  Clickthrough rates as high as 70.2% for Google Chrome SSL warnings 52