Making and Measuring Progress in Adversarial Machine Learning - - PowerPoint PPT Presentation

▶
making and measuring progress in adversarial machine
SMART_READER_LITE
LIVE PREVIEW

Making and Measuring Progress in Adversarial Machine Learning - - PowerPoint PPT Presentation

Sep 15, 2023 316 likes •1.12k views

Making and Measuring Progress in Adversarial Machine Learning Nicholas Carlini Google Research Act I Background Why should we care about adversarial examples? Make ML Make ML robust better Act II An Apparent Problem Let's go

slide-1
SLIDE 1

Making and Measuring Progress in Adversarial Machine Learning

Nicholas Carlini

Google Research

slide-2
SLIDE 2
slide-3
SLIDE 3

Act I
 Background

slide-4
SLIDE 4
slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8

Why should we care about adversarial examples?

Make ML robust Make ML better

slide-9
SLIDE 9
slide-10
SLIDE 10

Act II
 An Apparent Problem

slide-11
SLIDE 11

Let's go back to ~5 years ago ...

slide-12
SLIDE 12

Generative Adversarial Nets

SotA, 2014

slide-13
SLIDE 13

Progressive Growing of GANs

SotA, 2017

slide-14
SLIDE 14
slide-15
SLIDE 15

SotA, 2013

Evasion Attacks against ML
 at Test Time

slide-16
SLIDE 16

Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness

SotA, 2019

slide-17
SLIDE 17

that is ... ... less impressive

slide-18
SLIDE 18

3 years: 6 years:

slide-19
SLIDE 19

Why?

slide-20
SLIDE 20
slide-21
SLIDE 21

Act III
 Measuring Progress

slide-22
SLIDE 22

Have we even made any progress?

slide-23
SLIDE 23

A Brief History of time defenses

  • Oakland'16 - broken
  • ICLR'17 - broken
  • CCS'17 - broken
  • ICLR'18 - broken (mostly)
  • CVPR'18 - broken
  • NeurIPS'18 - broken (some)
slide-24
SLIDE 24

Have we even made any progress?

slide-25
SLIDE 25

Is this a constant cat-and-mouse game?

slide-26
SLIDE 26

What does it mean to make progress?

slide-27
SLIDE 27

What does it mean to make progress? Learning something new.

slide-28
SLIDE 28

A Brief History of time defenses

  • Oakland'16 - gradient masking
  • ICLR'17 - attack objective functions
  • CCS'17 - transferability of examples
  • ICLR'18 - obfuscated gradients
slide-29
SLIDE 29

A Brief History of time defenses

  • Oakland'16 - gradient masking
  • ICLR'17 - attack objective functions
  • CCS'17 - transferability of examples
  • ICLR'18 - obfuscated gradients
  • 2019 - ???
slide-30
SLIDE 30

Measure by how much
 we learn; not by how
 much robustness we gain.

slide-31
SLIDE 31
slide-32
SLIDE 32

Act IV
 Making Progress
 (for defenses)

slide-33
SLIDE 33

While we have learned
 a lot, it's less than I would have hoped.

slide-34
SLIDE 34
slide-35
SLIDE 35

Cargo Cult Evaluations

slide-36
SLIDE 36

Going through the motions is

insufficient

to do proper security evaluations

slide-37
SLIDE 37

An all too common paper:

slide-38
SLIDE 38

An all too common paper:

slide-39
SLIDE 39

The two types of defenses:

Defenses that
 are broken by
 existing attacks Defenses that
 are broken by
 new attacks

slide-40
SLIDE 40

Exciting new directions

slide-41
SLIDE 41

Exciting new directions

slide-42
SLIDE 42

Exciting new directions

slide-43
SLIDE 43

Exciting new directions

slide-44
SLIDE 44
slide-45
SLIDE 45

Act IV ½
 Making Progress
 (for attacks)

slide-46
SLIDE 46

Advice for performing evaluations

slide-47
SLIDE 47
slide-48
SLIDE 48

Perform Adaptive Attacks

slide-49
SLIDE 49

An all too common paper:

slide-50
SLIDE 50

Ensure correct implementations

slide-51
SLIDE 51

An all too common paper:

slide-52
SLIDE 52

An all too common paper:

slide-53
SLIDE 53

Use meaningful threat models

slide-54
SLIDE 54

An all too common paper:

slide-55
SLIDE 55

An all too common paper:

slide-56
SLIDE 56

An all too common paper:

slide-57
SLIDE 57

Compute Worst-Case Robustness

slide-58
SLIDE 58

An all too common paper:

slide-59
SLIDE 59

An all too common paper:

slide-60
SLIDE 60

An all too common paper:

slide-61
SLIDE 61

Compare to Prior Work

slide-62
SLIDE 62

An all too common paper:

slide-63
SLIDE 63

Sanity-Check Conclusions

slide-64
SLIDE 64

An all too common paper:

slide-65
SLIDE 65

An all too common paper:

slide-66
SLIDE 66

Making errors in defense evaluations is okay. Making errors in
 attack evaluations is not.

slide-67
SLIDE 67

Breaking a defense is useful ... ... teaching a lesson is better

slide-68
SLIDE 68

Exciting new directions

slide-69
SLIDE 69

Exciting new directions

slide-70
SLIDE 70

Exciting new directions

slide-71
SLIDE 71

Exciting new directions

slide-72
SLIDE 72

Exciting new directions

slide-73
SLIDE 73

Exciting new directions

slide-74
SLIDE 74

Exciting new directions

slide-75
SLIDE 75
slide-76
SLIDE 76

Act VI
 Conclusions

slide-77
SLIDE 77

Research new topics Do good science Progress is learning

slide-78
SLIDE 78

Questions?

nicholas@carlini.com https://nicholas.carlini.com

slide-79
SLIDE 79

References

Biggio et al. Evasion Attacks on Machine Learning at Test Time. 
 https://arxiv.org/abs/1708.06131 Jaconbsen et al. Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness 
 https://arxiv.org/abs/1903.10484 Carlini et al. On Evaluating Adversarial Robustness. 
 https://arxiv.org/abs/1902.06705 Chou et al. SentiNet: Detecting Physical Attacks Against Deep Learning Systems. 
 https://arxiv.org/abs/1812.00292 Shumailov et al. Sitatapatra: Blocking the Transfer of Adversarial Samples. 
 https://arxiv.org/abs/1901.08121 Ilyas et al. Adversarial Examples Are Not Bugs, They Are Features. 
 https://arxiv.org/abs/1905.02175 Brendel et al. Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning
 https://arxiv.org/abs/1712.04248 Wong et al. Wasserstein Adversarial Examples via Projected Sinkhorn Iterations 
 https://arxiv.org/abs/1902.07906.