Measuring security and cybercrime Daniel R. Thomas Cambridge - - PowerPoint PPT Presentation

▶
measuring security and cybercrime
SMART_READER_LITE
LIVE PREVIEW

Measuring security and cybercrime Daniel R. Thomas Cambridge - - PowerPoint PPT Presentation

Nov 30, 2022 253 likes •676 views

Measuring security and cybercrime Daniel R. Thomas Cambridge Cybercrime Centre, Department of Computer Science and Technology, University of Cambridge, UK SecHuman 2018 GPG: 5017 A1EC 0B29 08E3 CF64 7CCD 5514 35D5 D749 33D9

slide-1
SLIDE 1

Measuring security and cybercrime

Daniel R. Thomas

Cambridge Cybercrime Centre, Department of Computer Science and Technology, University

  • f Cambridge, UK

SecHuman 2018

GPG: 5017 A1EC 0B29 08E3 CF64 7CCD 5514 35D5 D749 33D9 Firstname.Surname@cl.cam.ac.uk

slide-2
SLIDE 2

Format

  • 1. Group warm up (5 minutes)
  • 2. Short lecture (35 minutes).
  • 3. Experimental design and review (50 minutes)

3.1 Designing an experiment to measure security or cybercrime (30 minutes) 3.2 Plenary feedback (20 minutes)

2 of 39

slide-3
SLIDE 3

What is security and how to we measure it?

▶ Discuss in groups for 2 minutes ▶ Then we will listen to some of the ideas

3 of 39

slide-4
SLIDE 4

Measuring security and cybercrime is important

▶ Is security getting better or worse? ▶ Did this intervention work? ▶ Is there a difgerence in security between these products?

4 of 39

slide-5
SLIDE 5

Two examples of security measurement research

▶ Measuring security of Android ▶ Measuring DDoS attacks (cybercrime)

Drawing out the principles, insights, and mistakes as we go along.

5 of 39

slide-6
SLIDE 6

Security metrics for the Android ecosystem1

https://androidvulnerabilities.org/ Daniel R. Thomas Alastair R. Beresford Andrew Rice Daniel Wagner

1Daniel R. Thomas, Alastair R. Beresford, and Andrew Rice. 2015. Security

metrics for the Android ecosystem. In ACM CCS workshop on Security and Privacy in Smartphones and Mobile Devices (SPSM). ACM, Denver, Colorado, USA, (Oct. 2015), 87–98. isbn: 978-1-4503-3819-6.

6 of 39

slide-7
SLIDE 7

Smartphones contain many apps written by a spectrum

  • f developers

How “secure” is a smartphone?

7 of 39

slide-8
SLIDE 8

Root/kernel exploits are harmful

▶ Root exploits break permission model ▶ Cannot recover to a safe state ▶ In 2012 37% Android malware used root exploits ▶ We’re interested in critical vulnerabilities, exploitable by code

running on the device

8 of 39

slide-9
SLIDE 9

Hypothesis: devices vulnerable because they are not updated

▶ Anecdotal evidence was that updates rarely happen ▶ Android phones, sold on 1-2 year contracts

9 of 39

slide-10
SLIDE 10

No central database of Android vulnerabilities: so we built one

10 of 39

slide-11
SLIDE 11

Device Analyzer gathers statistics on mobile phone usage

▶ Deployed May ’11 ▶ 30 000

contributors

▶ 4 000 phone years ▶ 180 billion records ▶ 10TB of data ▶ 1089 7-day active

contributors (2015 numbers)

11 of 39

slide-12
SLIDE 12

Device Analyzer gathers wide variety of data

Including: system statistics

▶ OS version and build number ▶ Manufacturer and device model ▶ Network operators

12 of 39

slide-13
SLIDE 13

Is the ecosystem getting updated?

13 of 39

slide-14
SLIDE 14

Google data: device API levels

O c t 2 1 1 A p r 2 1 2 O c t 2 1 2 A p r 2 1 3 O c t 2 1 3 A p r 2 1 4 O c t 2 1 4 A p r 2 1 5 O c t 2 1 5 0.0 0.2 0.4 0.6 0.8 1.0

Proportion of devices

3 4 7 8 10 12 1314 15 16 17 18 19 21 22 23

14 of 39

slide-15
SLIDE 15

Are devices getting updated?

15 of 39

slide-16
SLIDE 16

LG devices by OS version

16 of 39

slide-17
SLIDE 17

Connecting the two data sets: assume OS version → vulnerability

▶ We have an OS version from Device Analyzer ▶ We have vulnerability data with OS versions ▶ Match on OS and Build Number and assign:

▶ Vulnerable ▶ Maybe invulnerable ▶ Invulnerable (not known vulnerable) 17 of 39

slide-18
SLIDE 18

Vulnerability varies over time

O c t 2 1 1 A p r 2 1 2 O c t 2 1 2 A p r 2 1 3 O c t 2 1 3 A p r 2 1 4 O c t 2 1 4 A p r 2 1 5 O c t 2 1 5 0.0 0.2 0.4 0.6 0.8 1.0 Proportion of devices vulnerable maybe invulnerable invulnerable zergRush APK duplicate file Fake ID Last AVO

19% 11% 70%

18 of 39

slide-19
SLIDE 19

The FUM metric measures the security of Android devices

FUM = 4f + 3u + 3 2 1 + em free from (known) vulnerabilities updated to the latest version mean unfjxed vulnerabilities

19 of 39

slide-20
SLIDE 20

A u g 2 1 1 F e b 2 1 2 A u g 2 1 2 F e b 2 1 3 A u g 2 1 3 F e b 2 1 4 A u g 2 1 4 F e b 2 1 5

4.4.4 KTU84Q

  • ther

2.3.4 GRJ22 2.3.6 GINGERBREAD 2.3.7 GRJ22 4.0.1 ITL41F 4.0.2 ICL53F 4.0.3 IML74K 4.0.4 ICL53F 4.0.4 IMM30B 4.0.4 IMM30D 4.0.4 IMM76D 4.0.4 IMM76I 4.0.4 IMM76K 4.1 JRN84D 4.1.1 JRO03C 4.1.1 JRO03L 4.1.1 JRO03O 4.1.1 JRO03R 4.1.1 JRO03U 4.1.2 JZO54K 4.2 JOP40C 4.2.1 JOP40D 4.2.1 JOP40G 4.2.2 JDQ39 4.2.2 JDQ39E 4.3 JLS36G 4.3 JSS15J 4.3 JSS15Q 4.3 JWR66V 4.3 JWR66Y 4.3 JWR67B 4.3.1 JLS36I 4.4.2 KOT49H 4.4.2 KVT49L 4.4.3 KTU84M 4.4.4 KTU84P

Galaxy Nexus

1.0 0.8 0.6 0.4 0.2 0.0

Proportion of devices

2.3.3 GRI40

20 of 39

slide-21
SLIDE 21

Lack of security updates

Aug 2011 Feb 2012 Aug 2012 Feb 2013 Aug 2013 Feb 2014 Aug 2014 Feb 2015 0.0 0.2 0.4 0.6 0.8 1.0 Proportion

2.3.3 GRI40 2.3.5 GRJ90

HTC Desire HD A9191

Aug 2011 Feb 2012 Aug 2012 Feb 2013 Aug 2013 Feb 2014 Aug 2014 Feb 2015 0.0 0.2 0.4 0.6 0.8 1.0 Proportion 4.2.2 JDQ39

Symphony W68

21 of 39

slide-22
SLIDE 22

Comparing manufacturers

Nexus devices LG Motorola Samsung Sony HTC Asus Alps Symphony Walton 1 2 3 4 5 6 7

FUM scores

m u f

FUM score

22 of 39

slide-23
SLIDE 23

Why is fjxing vulnerabilities hard: software ecosystem is complex

▶ Division of labour

▶ Open source software ▶ Core OS production ▶ Driver writer ▶ Device manufacturer ▶ Retailer ▶ Customer

▶ Apple and Google have difgerent models

▶ Hypothesis: Apple’s model is more secure 23 of 39

slide-24
SLIDE 24

Google to the rescue

▶ Play Store ▶ Verify apps ▶ Android Security Patch Level ▶ Later: Android Enterprise

Recommended

24 of 39

slide-25
SLIDE 25

What happened next?

▶ Plenty press coverage ▶ Contacts with Google, manufacturers, UK Home Offjce ▶ FTC cites work. ▶ Google uses graphs to pressure manufacturers to improve update

provision

▶ We move on: no further collection of vulnerability data, no

updated scores.

25 of 39

slide-26
SLIDE 26

1000 days of UDP amplifjcation DDoS attacks2

Daniel R. Thomas Richard Clayton Alastair R. Beresford

2Daniel R. Thomas, Richard Clayton, and Alastair R. Beresford. 2017. 1000 days

  • f UDP amplifjcation DDoS attacks. In APWG Symposium on Electronic Crime

Research (eCrime). IEEE, (Apr. 2017).

26 of 39

slide-27
SLIDE 27

UDP scanning

Reflector 8.8.8.8 Attacker 192.168.25.4

big.gov IN TXT src: 192.168.25.4 dst: 8.8.8.8

big.gov IN TXT " Extremely long response.............. ........................... ........................... .........................." src: 8.8.8.8 dst: 192.168.25.4

(1) (2)

27 of 39

slide-28
SLIDE 28

UDP refmection DDoS attacks

Reflector 8.8.8.8 Attacker 192.168.25.4 Victim 172.16.6.2

big.gov IN TXT src: 172.16.6.2 dst: 8.8.8.8

big.gov IN TXT " Extremely long response.............. ........................... ........................... .........................." src: 8.8.8.8 dst: 172.16.6.2

28 of 39

slide-29
SLIDE 29

We run lots of UDP honeypots

▶ Median 65 nodes since 2014 ▶ Hopscotch emulates abused protocols

QOTD, CHARGEN, DNS, NTP, SSDP, SQLMon, Portmap, mDNS, LDAP

▶ Snifger records all resulting UDP traffjc ▶ (try to) Only reply to black hat scanners

29 of 39

slide-30
SLIDE 30

Total attacks estimated using capture-recapture

A=160 B=200

Estimated population: 400 ± 62 80 80 120

30 of 39

slide-31
SLIDE 31

10 100 1000 10000 100000 2014-07 2014-10 2015-01 2015-04 2015-07 2015-10 2016-01 2016-04 2016-07 2016-10 2017-01 2017-04 2017-07 Estimated number of attacks per day (log) CHARGEN DNS NTP SSDP

31 of 39

slide-32
SLIDE 32

0.2 0.4 0.6 0.8 1 2014-07 2014-10 2015-01 2015-04 2015-07 2015-10 2016-01 2016-04 2016-07 2016-10 2017-01 2017-04 2017-07 10 20 30 40 50 60 70 80 90 Proportion of all attacks that we observe CHARGEN DNS NTP SSDP

32 of 39

slide-33
SLIDE 33

0.2 0.4 0.6 0.8 1 2014-07 2014-10 2015-01 2015-04 2015-07 2015-10 2016-01 2016-04 2016-07 2016-10 2017-01 2017-04 2017-07 10 20 30 40 50 60 70 80 90 Number of honeypots in operation # A+B # A

33 of 39

slide-34
SLIDE 34

0.2 0.4 0.6 0.8 1 2014-07 2014-10 2015-01 2015-04 2015-07 2015-10 2016-01 2016-04 2016-07 2016-10 2017-01 2017-04 2017-07 10 20 30 40 50 60 70 80 90 Proportion of all attacks that we observe Number of honeypots in operation # A+B # A CHARGEN DNS NTP SSDP

34 of 39

slide-35
SLIDE 35

This was ethical

▶ We reduce harm by absorbing attack traffjc ▶ We don’t reply to white hat scanners (no timewasting) ▶ We used leaked data for validation, this was necessary and did not

increase harm.

▶ Further discussion of the ethics of using leaked data for research

tomorrow.

35 of 39

slide-36
SLIDE 36

This is a solvable problem

▶ BCP38/SAVE ▶ Follow the money ▶ Enforce the law ▶ Warn customers it is illegal

36 of 39

slide-37
SLIDE 37

Experimental design [30 minutes]

How would you measure the relative security of difgerent: BO Banks BOT CPU vendors DO Residential ISPs DU Operating systems E Cycle lock manufacturers GE IoT manufacturers HER Offjces MH Elections OB Online payment providers RE Smartphones What data would you need to collect? How would you collect it? Would it be possible to cheat your measurement without actually improving security?

37 of 39

slide-38
SLIDE 38

Plenary discussion [20 minutes]

Feedback from each group on their experimental design.

38 of 39

slide-39
SLIDE 39

Thank you! Questions?

Daniel R. Thomas Daniel.Thomas@cl.cam.ac.uk @DanielRThomas24 https://www.cl.cam.ac.uk/~drt24/ 5017 A1EC 0B29 08E3 CF64 7CCD 5514 35D5 D749 33D9

Daniel Thomas is supported by the EPSRC [grant number EP/M020320/1].

39 of 39

slide-40
SLIDE 40

References I

[1] Daniel R. Thomas, Alastair R. Beresford, and Andrew Rice. 2015. Security metrics for the Android ecosystem. In ACM CCS workshop on Security and Privacy in Smartphones and Mobile Devices (SPSM). ACM, Denver, Colorado, USA, (Oct. 2015), 87–98. isbn: 978-1-4503-3819-6. [2] Daniel R. Thomas, Richard Clayton, and Alastair R. Beresford. 2017. 1000 days of UDP amplifjcation DDoS attacks. In APWG Symposium on Electronic Crime Research (eCrime). IEEE, (Apr. 2017).

1 of 1