DATA ANALYTICS USING DEEP LEARNING GT 8803 // FALL 2019 // JOY - - PowerPoint PPT Presentation

DATA ANALYTICS USING DEEP LEARNING

GT 8803 // FALL 2019 // JOY ARULRAJ

L E C T U R E # 2 0 : A D V E R S A R I A L T R A I N I N G

GT 8803 // Fall 2019

administrivia

• Reminders

– Best project prize – Quiz cancelled – Guest lecture

2

GT 8803 // Fall 2019

CREDITS

• Slides based on a lecture by:

– Ian Goodfellow @ Google Brain

3

GT 8803 // Fall 2019

OVERVIEW

• What are adversarial examples?
• Why do they happen?
• How can they be used to compromise machine

learning systems?

• What are the defenses?
• How to use adversarial examples to improve

machine learning (even without adversary)?

4

GT 8803 // Fall 2019

ADVERSARIAL EXAMPLES

5

GT 8803 // Fall 2019 6 ...solving CAPTCHAS and reading addresses...

...recognizing

• bjects

and faces….

(Szegedy et al, 2014) (Goodfellow et al, 2013) (Taigmen et al, 2013) (Goodfellow et al, 2013)

and other tasks... Since 2013, deep neural networks have matched human performance at...

GT 8803 // Fall 2019

Adversarial Examples

7

Timeline: “Adversarial Classification” Dalvi et al 2004: fool spam filter “Evasion Attacks Against Machine Learning at Test Time” Biggio 2013: fool neural nets Szegedy et al 2013: fool ImageNet classifiers imperceptibly Goodfellow et al 2014: cheap, closed form attack

GT 8803 // Fall 2019

Turning Objects into “Airplanes”

8

GT 8803 // Fall 2019

Attacking a Linear Model

9

GT 8803 // Fall 2019

Not just for neural nets

• Linear models

– Logistic regression – Softmax regression – SVMs

• Decision trees
• Nearest neighbors

10

GT 8803 // Fall 2019

Adversarial Examples from Overfitting

11

x x x

O O O

x

O

GT 8803 // Fall 2019

Adversarial Examples from Overfitting

12

x x x

O O O O O

x

GT 8803 // Fall 2019

Modern deep nets are very piecewise linear

13

Rectified linear unit Carefully tuned sigmoid Maxout LSTM

GT 8803 // Fall 2019

Nearly Linear Responses in Practice

14

GT 8803 // Fall 2019

Small inter-class distances

15

Clean example Perturbation Corrupted example All three perturbations have L2 norm 3.96 This is actually small. We typically use 7! Perturbation changes the true class Random perturbation does not change the class Perturbation changes the input to “rubbish class”

GT 8803 // Fall 2019

The Fast Gradient Sign Method

16

GT 8803 // Fall 2019

Maps of Adversarial and Random Cross-Sections

17

(collaboration with David Warde-Farley and Nicolas Papernot)

GT 8803 // Fall 2019

18

Maps of Adversarial Cross-Sections

GT 8803 // Fall 2019

19

Maps of RANDOM Cross-Sections

(collaboration with David Warde-Farley and Nicolas Papernot)

GT 8803 // Fall 2019

Estimating the Subspace Dimensionality

20

GT 8803 // Fall 2019

Clever Hans

21

(“Clever Hans, Clever Algorithms,” Bob Sturm)

GT 8803 // Fall 2019

Wrong almost everywhere

22

GT 8803 // Fall 2019

Adversarial Examples for RL

23

(Huang et al., 2017)

GT 8803 // Fall 2019

High-Dimensional Linear Models

24

Weights Signs of weights Clean examples Adversarial examples

GT 8803 // Fall 2019

Linear Models of ImageNet

25

(Andrej Karpathy, “Breaking Linear Classifiers on ImageNet”)

GT 8803 // Fall 2019

RBFs behave more intuitively

26

GT 8803 // Fall 2019

Cross-model, cross-dataset generalization

27

GT 8803 // Fall 2019

Cross-technique transferability

28

(Papernot 2016)

GT 8803 // Fall 2019

Transferability Attack

29

Train your own model Target model with unknown weights, machine learning algorithm, training set; maybe non-differentiable Substitute model mimicking target model with known, differentiable function Adversarial examples Adversarial crafting against substitute Deploy adversarial examples against the target; transferability property results in them succeeding

GT 8803 // Fall 2019 30

(Papernot 2016)

GT 8803 // Fall 2019

Enhancing Transfer With Ensembles

31

(Liu et al, 2016)

GT 8803 // Fall 2019

Adversarial Examples in the Human Brain

32

(Pinna and Gregory, 2002)

These are concentric circles, not intertwined spirals.

GT 8803 // Fall 2019

Practical Attacks

• Fool real classifiers trained by remotely

hosted API (MetaMind, Amazon, Google)

• Fool malware detector networks
• Display adversarial examples in the physical

world and fool machine learning systems that perceive them through a camera

33

GT 8803 // Fall 2019

Adversarial Examples in the Physical World

34

GT 8803 // Fall 2019

Failed defenses

35

Weight decay Adding noise at test time Adding noise at train time Dropout Ensembles Multiple glimpses Generative pretraining Removing perturbation with an autoencoder Error correcting codes Confidence-reducing perturbation at test time Various non-linear units Double backprop

GT 8803 // Fall 2019

Generative Modeling is not SufficienT

36

GT 8803 // Fall 2019

Universal approximator theorem

37

Neural nets can represent either function: Maximum likelihood doesn’t cause them to learn the right function. But we can fix that...

GT 8803 // Fall 2019

ADVERSARIAL TRAINING

38

GT 8803 // Fall 2019

Training on Adversarial Examples

39

GT 8803 // Fall 2019

Adversarial Training of other Models

• Linear models: SVM / linear regression cannot learn a

step function, so adversarial training is less useful, very similar to weight decay

• k-NN: adversarial training is prone to overfitting.
• Takeway: neural nets can actually become more secure

than other models. Adversarially trained neural nets have the best empirical success rate on adversarial examples of any machine learning model.

40

GT 8803 // Fall 2019

Weaknesses Persist

41

GT 8803 // Fall 2019

Adversarial Training

42

Labeled as bird Decrease probability

• f bird class

Still has same label (bird)

GT 8803 // Fall 2019

Virtual Adversarial Training

43

Unlabeled; model guesses it’s probably a bird, maybe a plane Adversarial perturbation intended to change the guess New guess should match old guess (probably bird, maybe plane)

GT 8803 // Fall 2019

Text Classification with VAT

44

7.70 7.20 7.40 7.12 7.05 6.97 6.68

6.00 6.50 7.00 7.50 8.00

Earlier SOTA Our baseline Virtual Adversarial Both + bidirectional model

RCV1 Misclassification Rate

Zoomed in for legibility

GT 8803 // Fall 2019 45

Universal engineering machine (model-based optimization)

Training data Extrapolation Make new inventions by finding input that maximizes model’s predicted performance

GT 8803 // Fall 2019 46

cleverhans

Open-source library available at: https://github.com/openai/cleverhans Built on top of TensorFlow (Theano support anticipated) Standard implementation of attacks, for adversarial training and reproducible benchmarks

GT 8803 // Fall 2019

Conclusion

• Attacking is easy
• Defending is difficult
• Adversarial training provides regularization and

semi-supervised learning

• The out-of-domain input problem is a bottleneck for

model-based optimization generally

47