Heat of the Moment: Characterizing the Efficacy of Thermal - - PowerPoint PPT Presentation

▶
heat of the moment characterizing the efficacy of thermal
SMART_READER_LITE
LIVE PREVIEW

Heat of the Moment: Characterizing the Efficacy of Thermal - - PowerPoint PPT Presentation

Dec 08, 2023 162 likes •1.2k views

Heat of the Moment: Characterizing the Efficacy of Thermal Camera-Based Attacks Keaton Mowery (UC San Diego) Sarah Meiklejohn (UC San Diego) Stefan Savage (UC San Diego) 1 Code-based access control 2 Code-based access control 2 Code-based

slide-1
SLIDE 1

Heat of the Moment: Characterizing the Efficacy of Thermal Camera-Based Attacks

Keaton Mowery (UC San Diego) Sarah Meiklejohn (UC San Diego) Stefan Savage (UC San Diego)

1

slide-2
SLIDE 2

Code-based access control

2

slide-3
SLIDE 3

Code-based access control

2

slide-4
SLIDE 4

Code-based access control

2

slide-5
SLIDE 5

Code-based access control

2

slide-6
SLIDE 6

Code-based access control

The problem: what if there is a camera watching you type in your code?

2

slide-7
SLIDE 7

Filming keypads

The solution: just shield the keypad!

3

slide-8
SLIDE 8

Filming keypads

The solution: just shield the keypad!

3

slide-9
SLIDE 9

Filming keypads

The solution: just shield the keypad!

3

slide-10
SLIDE 10

Filming keypads

The solution: just shield the keypad!

3

Another problem: this only protects the code while it is being typed, not after

slide-11
SLIDE 11

Filming keypads

The solution: just shield the keypad!

3

Another problem: this only protects the code while it is being typed, not after Turns out heat is transferred in the process of entering the code, heat residue is left after code entry

slide-12
SLIDE 12

Filming keypads

The solution: just shield the keypad!

3

Another problem: this only protects the code while it is being typed, not after Turns out heat is transferred in the process of entering the code, heat residue is left after code entry Our attack: this residue can then be recorded by a thermal camera

slide-13
SLIDE 13

Previous work

4

slide-14
SLIDE 14

Feasibility of this attack was demonstrated in 2005 by Michał Zalewski

Previous work

4

slide-15
SLIDE 15

Feasibility of this attack was demonstrated in 2005 by Michał Zalewski

Previous work

4

slide-16
SLIDE 16

Feasibility of this attack was demonstrated in 2005 by Michał Zalewski

Previous work

4

(images from lcamtuf.coredump.cx/tsafe)

slide-17
SLIDE 17

Feasibility of this attack was demonstrated in 2005 by Michał Zalewski He was able to retrieve thermal residue for between five and ten minutes after code was entered

Previous work

4

(images from lcamtuf.coredump.cx/tsafe)

slide-18
SLIDE 18

This work

5

slide-19
SLIDE 19

This work

5

We broaden the picture by considering different:

slide-20
SLIDE 20

This work

5

We broaden the picture by considering different:

  • Keypad materials (metal vs. plastic)
slide-21
SLIDE 21

This work

5

We broaden the picture by considering different:

  • Keypad materials (metal vs. plastic)
  • Keypad users (cold- vs. warm-blooded, etc.)
slide-22
SLIDE 22

This work

5

We broaden the picture by considering different:

  • Keypad materials (metal vs. plastic)
  • Keypad users (cold- vs. warm-blooded, etc.)
  • Review methods (automated vs. visual inspection)
slide-23
SLIDE 23

This work

5

We broaden the picture by considering different:

  • Keypad materials (metal vs. plastic)
  • Keypad users (cold- vs. warm-blooded, etc.)
  • Review methods (automated vs. visual inspection)
  • Degrees of success (exact code vs. partial information)
slide-24
SLIDE 24

This work

5

We broaden the picture by considering different:

  • Keypad materials (metal vs. plastic)
  • Keypad users (cold- vs. warm-blooded, etc.)
  • Review methods (automated vs. visual inspection)
  • Degrees of success (exact code vs. partial information)

Find that results vary substantially as we change above variables

slide-25
SLIDE 25

Outline

6

slide-26
SLIDE 26

Experiment design

Outline

6

slide-27
SLIDE 27

Experiment design

Outline

Camera data

6

slide-28
SLIDE 28

Experiment design

Outline

Camera data Analyzing the data

6

slide-29
SLIDE 29

Experiment design

Outline

Camera data Analyzing the data Conclusions

6

slide-30
SLIDE 30

Experiment design

Outline

Experiment design Camera data Analyzing the data Conclusions

6

slide-31
SLIDE 31

Our setup: equipment

7

slide-32
SLIDE 32

Our setup: equipment FLIR A320 IR camera

7

320 x 240 resolution $18,000 to purchase $2,000/month to rent Operates at 9Hz

slide-33
SLIDE 33

Our setup: equipment FLIR A320 IR camera Metal ATM keypad

7

320 x 240 resolution $18,000 to purchase $2,000/month to rent Operates at 9Hz

slide-34
SLIDE 34

Our setup: equipment FLIR A320 IR camera Metal ATM keypad Plastic ATM keypad

7

320 x 240 resolution $18,000 to purchase $2,000/month to rent Operates at 9Hz

slide-35
SLIDE 35

Our setup: getting things ready

8

slide-36
SLIDE 36

Set keypad in a vise and camera on a tripod across from it

Our setup: getting things ready

8

slide-37
SLIDE 37

Set keypad in a vise and camera on a tripod across from it Worked at two different distances: 14 and 28 inches

Our setup: getting things ready

8

slide-38
SLIDE 38

Set keypad in a vise and camera on a tripod across from it Worked at two different distances: 14 and 28 inches Used software to indicate ten regions of interest on the keypad (0-9)

Our setup: getting things ready

8

slide-39
SLIDE 39

Our setup: code entry

9

slide-40
SLIDE 40

Our setup: code entry

9

At each distance, had 21 people type in 27 different codes

slide-41
SLIDE 41

Our setup: code entry

9

At each distance, had 21 people type in 27 different codes

  • Wanted to allow for different body temperatures, key-pressing styles, etc.
  • 7 of these codes contained repeats (e.g., 6688 or 8728)
slide-42
SLIDE 42

Our setup: code entry

9

At each distance, had 21 people type in 27 different codes

  • Wanted to allow for different body temperatures, key-pressing styles, etc.
  • 7 of these codes contained repeats (e.g., 6688 or 8728)

Filmed the keypad for 3 seconds before code entry, then 100 seconds after, recorded 3 frames per second

slide-43
SLIDE 43

Outline

10

Experiment design Camera data Analyzing the data Conclusions

slide-44
SLIDE 44

Filming metal was a complete failure!

11

slide-45
SLIDE 45

Brushed metal acted as a thermal mirror, hard to even get any reading

Filming metal was a complete failure!

11

slide-46
SLIDE 46

Brushed metal acted as a thermal mirror, hard to even get any reading

Filming metal was a complete failure!

11

(images from “Identification and suppression of thermal reflections in infrared thermal imaging,” Henke et. al., InfraMation 2004.)

slide-47
SLIDE 47

Brushed metal acted as a thermal mirror, hard to even get any reading High conductivity of metal meant residue spread within seconds

Filming metal was a complete failure!

11

(images from “Identification and suppression of thermal reflections in infrared thermal imaging,” Henke et. al., InfraMation 2004.)

slide-48
SLIDE 48

Brushed metal acted as a thermal mirror, hard to even get any reading High conductivity of metal meant residue spread within seconds So the rest of our results are only for plastic keypads

Filming metal was a complete failure!

11

(images from “Identification and suppression of thermal reflections in infrared thermal imaging,” Henke et. al., InfraMation 2004.)

slide-49
SLIDE 49

An ideal run

12

slide-50
SLIDE 50

An ideal run

12

slide-51
SLIDE 51

Results can vary widely 1485 0368 1876 0482

13

Even in the first frame after entry, see very different pictures:

slide-52
SLIDE 52

Results can vary widely 1485 0368 1876 0482

13

Even in the first frame after entry, see very different pictures:

slide-53
SLIDE 53

Results can vary widely 1485 0368 1876 0482

13

Even in the first frame after entry, see very different pictures:

slide-54
SLIDE 54

Results can vary widely 1485 0368 1876 0482

13

Even in the first frame after entry, see very different pictures:

slide-55
SLIDE 55

Results can vary widely 1485 0368 1876 0482

13

Even in the first frame after entry, see very different pictures:

slide-56
SLIDE 56

Results can vary widely 1485 0368 1876 0482

14

See similar differences in how residue degrades over time:

slide-57
SLIDE 57

Results can vary widely 1485 0368 1876 0482

14

See similar differences in how residue degrades over time:

slide-58
SLIDE 58

Results can vary widely 1485 0368 1876 0482

14

See similar differences in how residue degrades over time:

slide-59
SLIDE 59

Outline

15

Experiment design Camera data Analyzing the data Conclusions

slide-60
SLIDE 60

Human review

16

slide-61
SLIDE 61

First approach: human visual inspection

Human review

16

slide-62
SLIDE 62

First approach: human visual inspection

  • Examine every 10th frame (in random order) to guess code entered

Human review

16

slide-63
SLIDE 63

First approach: human visual inspection

  • Examine every 10th frame (in random order) to guess code entered

Problem: this approach doesn’t scale very well! (looked at ~1800 images)

Human review

16

slide-64
SLIDE 64

First approach: human visual inspection

  • Examine every 10th frame (in random order) to guess code entered

Problem: this approach doesn’t scale very well! (looked at ~1800 images)

  • Second approach: automated review

Human review

16

slide-65
SLIDE 65

Automated review: what to do with all this footage?

17

slide-66
SLIDE 66

Automated review: what to do with all this footage?

17

slide-67
SLIDE 67

Automated review: what to do with all this footage?

17

calibration

slide-68
SLIDE 68

Automated review: what to do with all this footage?

17

calibration

slide-69
SLIDE 69

Automated review: what to do with all this footage?

17

calibration hand

slide-70
SLIDE 70

Automated review: what to do with all this footage?

17

calibration hand

slide-71
SLIDE 71

Automated review: what to do with all this footage?

17

calibration hand after entry

slide-72
SLIDE 72

Automated review: what to do with all this footage?

17

calibration hand after entry

slide-73
SLIDE 73

Automated review: what to do with all this footage?

17

calibration after entry

slide-74
SLIDE 74

Automated review: which buttons were pressed?

18

slide-75
SLIDE 75

Basic idea: for each region, determine if it is hot above a certain threshold

Automated review: which buttons were pressed?

18

slide-76
SLIDE 76

Basic idea: for each region, determine if it is hot above a certain threshold

Automated review: which buttons were pressed? calibration

18

slide-77
SLIDE 77

Basic idea: for each region, determine if it is hot above a certain threshold

Automated review: which buttons were pressed? calibration t0=71

18

slide-78
SLIDE 78

Basic idea: for each region, determine if it is hot above a certain threshold

Automated review: which buttons were pressed? calibration t0=71 after entry

18

slide-79
SLIDE 79

Basic idea: for each region, determine if it is hot above a certain threshold

Automated review: which buttons were pressed? calibration t0=71 average t=73.6 after entry

18

slide-80
SLIDE 80

Basic idea: for each region, determine if it is hot above a certain threshold Can repeat this process for each region, then sort in order of Δ = t - t0

Automated review: which buttons were pressed? calibration t0=71 average t=73.6 after entry

18

slide-81
SLIDE 81

Basic idea: for each region, determine if it is hot above a certain threshold Can repeat this process for each region, then sort in order of Δ = t - t0 Examined regions in isolation because we didn’t observe much heat spread

Automated review: which buttons were pressed? calibration t0=71 average t=73.6 after entry

18

slide-82
SLIDE 82

Basic idea: for each region, determine if it is hot above a certain threshold Can repeat this process for each region, then sort in order of Δ = t - t0 Examined regions in isolation because we didn’t observe much heat spread This is the mean method, also use max and binarize variants

Automated review: which buttons were pressed? calibration t0=71 average t=73.6 after entry

18

slide-83
SLIDE 83

How did we do?

19

slide-84
SLIDE 84

First goal: recover the exact code entered

How did we do?

19

slide-85
SLIDE 85

First goal: recover the exact code entered

How did we do? human review

19

slide-86
SLIDE 86

First goal: recover the exact code entered

How did we do? human review automated review

19

slide-87
SLIDE 87

First goal: recover the exact code entered Bad news: the picture doesn’t get much better if we allow for slight mistakes (transpositions, one wrong key, etc.)

How did we do? human review automated review

19

slide-88
SLIDE 88

How did we do?

20

slide-89
SLIDE 89

Second goal: recover the buttons pressed (not necessarily the correct order)

How did we do?

20

slide-90
SLIDE 90

Second goal: recover the buttons pressed (not necessarily the correct order)

How did we do? human review

20

slide-91
SLIDE 91

Second goal: recover the buttons pressed (not necessarily the correct order)

How did we do? human review automated review

20

slide-92
SLIDE 92

Second goal: recover the buttons pressed (not necessarily the correct order)

How did we do? human review automated review

recover ~30% after 1 minute

20

slide-93
SLIDE 93

Second goal: recover the buttons pressed (not necessarily the correct order)

How did we do? human review automated review

recover ~30% after 1 minute recover ~50% after 1 minute

20

slide-94
SLIDE 94

Second goal: recover the buttons pressed (not necessarily the correct order) Not only is automated review scalable, it’s also significantly more accurate

How did we do? human review automated review

recover ~30% after 1 minute recover ~50% after 1 minute

20

slide-95
SLIDE 95

Outline

21

Experiment design Camera data Analyzing the data Conclusions

slide-96
SLIDE 96

Conclusions and future work

22

slide-97
SLIDE 97

Conclusions and future work

Conducted study of the efficacy of thermal cameras in a variety of scenarios

22

slide-98
SLIDE 98

Conclusions and future work

Conducted study of the efficacy of thermal cameras in a variety of scenarios

  • Most effective: with plastic we recovered ~50% of codes a full minute after

22

slide-99
SLIDE 99

Conclusions and future work

Conducted study of the efficacy of thermal cameras in a variety of scenarios

  • Most effective: with plastic we recovered ~50% of codes a full minute after
  • Least effective: metal keypad doesn’t work at all right now

22

slide-100
SLIDE 100

Conclusions and future work

Conducted study of the efficacy of thermal cameras in a variety of scenarios

  • Most effective: with plastic we recovered ~50% of codes a full minute after
  • Least effective: metal keypad doesn’t work at all right now
  • Also saw that different body temperatures and pressing styles mattered

22

slide-101
SLIDE 101

Conclusions and future work

Conducted study of the efficacy of thermal cameras in a variety of scenarios

  • Most effective: with plastic we recovered ~50% of codes a full minute after
  • Least effective: metal keypad doesn’t work at all right now
  • Also saw that different body temperatures and pressing styles mattered

Future work and open problems:

  • Use a wider set of choices: different materials, temperatures, etc.
  • Analyzing footage rather than individual frames

22

slide-102
SLIDE 102

Conclusions and future work

Conducted study of the efficacy of thermal cameras in a variety of scenarios

  • Most effective: with plastic we recovered ~50% of codes a full minute after
  • Least effective: metal keypad doesn’t work at all right now
  • Also saw that different body temperatures and pressing styles mattered

Future work and open problems:

  • Use a wider set of choices: different materials, temperatures, etc.
  • Analyzing footage rather than individual frames

Thanks! Any questions?

22