Towards Attack-Agnostic Defense for 2D and 3D Recognition Hao Su - - PowerPoint PPT Presentation

Towards Attack-Agnostic Defense for 2D and 3D Recognition

Hao Su

1

Workshop on Adversarial Machine Learning in Real-World Computer Vision Systems Long Beach, CA, USA

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Outline

• Background and Motivation
• Project-based Defense Mechanism
• For 2D Image Classification
• For 3D Shape Classification
• Summary

2

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Outline

• Background and Motivation
• Project-based Defense Mechanism
• For 2D Image Classification
• For 3D Shape Classification
• Summary

3

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

4

• Deep learning has made groundbreaking achievements on…
• However, deep learning faces robustness and security challenges

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Classic Adversarial Attacks

• FGSM: Fast Gradient Sign Method
• BIM: Basic Iterative Method
• DeepFool
• C&W
• …

Clean FGSM BIM DeepFool C&W

5

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

• The meta algorithm is to optimize the following score:

Defense Mechanisms

6 ack: Towards Deep Learning Models Resistant to Adversarial Attacks, Mądry et al.

minimize E(x,y)∼D[max

δ∈S L(θ, x + δ, y)]

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Adversarial Training

• Approximate the max of in the vicinity of training data by

gradient-based optimization

7

x

L

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Adversarial Training

• Approximate the max of in the vicinity of training data by

gradient-based optimization

• Can derive adversarial training methods such as FGSM, PGD, …

8

x

L

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Adversarial Training

• Approximate the max of in the vicinity of training data by

gradient-based optimization

• Can derive adversarial training methods such as FGSM, PGD, …
• Extensively and actively studied direction

9

x

L

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

• For each , we have to run the optimization, which is costly;
• For each , we have to create sufficient adversarial perturbations in , as

the model evolves;

Limitation of Adversarial Training

10

x

x

θ

S

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

• For each , we have to run the optimization, which is costly;
• For each , we have to create sufficient adversarial perturbations in , as

the model evolves.

Limitation of Adversarial Training

11

x

x

x

θ

S

Training is costly!

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

• Adversarially trained models tend to overfit the attacking method (even

parameters) used in training

Limitation of Adversarial Training (II)

\begin{tabular}{|l|l|l|}

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

• Adversarially trained models tend to overfit the attacking method (even

parameters) used in training

Limitation of Adversarial Training (II)

\begin{tabular}{|l|l|l|}

Logit Pairing Methods Can Fool Gradient-Based Attacks, by Mosbach et al.

Defense success rate when varying attacker PGD iteration

L∞ = 16.0

Note: adv. training with 10 iterations and

Iter 10 400 CIFAR-10 16.0 6.7 Tiny-ImageNet 25.5 16.3

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

• Adversarially trained models tend to overfit the attacking method (even

parameters) used in training

Limitation of Adversarial Training (II)

\begin{tabular}{|l|l|l|}

Logit Pairing Methods Can Fool Gradient-Based Attacks, by Mosbach et al.

Defense success rate when varying attacker PGD iteration

L∞ = 16.0

Note: adv. training with 10 iterations and

Iter 10 400 CIFAR-10 16.0 6.7 Tiny-ImageNet 25.5 16.3

Has to assume the attacking algorithm!

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Practically,

• We want a defense mechanism that is
• Fast to train and test
• Agnostic to attacking algorithms

15

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Outline

• Background and Motivation
• Project-based Defense Mechanism
• For 2D Image Classification
• For 3D Shape Classification
• Conclusion

16

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

• Observation: Higher dimensional data is more vulnerable to

adversarial attacks ( is “larger” and defense gets harder)

Other Strategies?

• Revisit our goal:

17

minimize E(x,y)∼D[max

δ∈S L(θ, x + δ, y)]

S

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Empirical Evidence I

18

Accuracy on CIFAR-10

32 × 32 64 × 64 Clean 95.33 95.53

• Attacking Undefended Model

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Empirical Evidence I

19

Accuracy on CIFAR-10

32 × 32 64 × 64 Clean 95.33 95.53 PGD (✏ = 2) 23.96 1.1 PGD (✏ = 4) 3.92

≥

• Attacking Undefended Model
• Lower-dimensional data seems more robust to attacks

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Empirical Evidence II

20

• Attacking Defended Model (adversarially trained by PGD with )

\begin{tabular}{c|cccc}

✏ = 1

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Empirical Evidence II

21

• Attacking Defended Model (adversarially trained by PGD with )

\begin{tabular}{c|cccc}

✏ = 1

Accuracy on CIFAR-10

PGD (✏ = 1) 32×32 83.7 64×64 80.5

• Lower-dimensional data seems to be easier to defend

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Empirical Evidence II

22

• Attacking Defended Model (adversarially trained by PGD with )

\begin{tabular}{c|cccc}

✏ = 1

Accuracy on CIFAR-10

PGD (✏ = 1) PGD (✏ = 2) PGD (✏ = 4) 32×32 83.7 77.3 47.8 64×64 80.5 66.4 35.9 Gap 3.2 10.9 11.9

• Lower-dimensional data seems to be easier to defend
• Defending in lower resolution seems to be more attack agnostic

Idea

Project adversarial examples to a low-dimensional data manifold that is easier to defend

E(x,y)∼D[max

δ∈S L(θ, Tψ(x + δ), y)] ≤ E(x,y)∼D[max δ∈S L(θ, x + δ, y)]

Idea

Project adversarial examples to a low-dimensional data manifold that is easier to defend

E(x,y)∼D[max

δ∈S L(θ, Tψ(x + δ), y)] ≤ E(x,y)∼D[max δ∈S L(θ, x + δ, y)]

How to design the family of ?

Tψ(·)

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Some Inspiration from Human Vision System

25

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Some Inspiration from Human Vision System

26

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Some Inspiration from Human Vision System

27

Gist first, details afterwards

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Misperception v.s. Ambiguity

(a) Abnormal. Typical reason: recognition obscured by details. (b) Ambiguous. Obfuscated labels.

bird -> people ship -> bird dog -> fish

bird or bicycle? 4 or 6? 0 or 6? unobvious

28

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Outline

• Background and Motivation
• Project-based Defense Mechanism
• For 2D Image Classification
• For 3D Shape Classification
• Conclusion

29

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Goal

• Build the space of natural data (2D images, 3D shapes, …),

which only focus on the gist

• Project the data to this low-dimensional space and wipe out

details

30

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Outline

• Background and Motivation
• Project-based Defense Mechanism
• For 2D Image Classification
• For 3D Shape Classification
• Conclusion

31

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Idea

Build a piece-wise linear image space for natural images

32

Tψ(·)

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Dictionary Learning/Sparse Coding

s.t.

33

• J. Mairal, F

. Bach, and J. Ponce. Sparse modeling for image and vision processing. Foundations and Trends in Computer Graphics and Vision, 8(2-3):85–283, 2014.

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Convolutional Dictionary Learning (CDL)

(pre-learned) dictionary filters: feature map

𝒈𝟐 𝒈𝟑 𝒈𝟒 𝒈𝒍

34

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

35

• Downsample images to 32×32 to train a DAE

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

36

• Split the latent space to 4 clusters for CIFAR-10 and

ImageNet-10, and 10 clusters for ImageNet.

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

37

• Filter number K = 64
• Filter size S = 8

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

38

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

39

Optimization based transformation

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Experiments — Denoising Results

ImageNet (Resolution: 224*224):

40

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Experiments — Denoising Results

ImageNet (Resolution: 224*224):

41

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

42

D3 Ours Clean

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Experiments — Effect of Coding Sparsity

43

Intrinsic tradeoff between image reconstruction quality and defensive robustness.

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Comparison with Other Transformation Methods

44

Defense Clean FGSM BIM

DeepFool

CW No Defense 0.930 0.582 0.180 0.176 0.094 MagNet[1] 0.904 0.615 0.431 0.654 0.485

PixelDefend[2]

0.853 0.681 0.773 0.741 0.758 STL 0.829 0.710 0.745 0.785 0.777 STL(Cluster) 0.836 0.711 0.753 0.796 0.791

On CIFAR-10 by VGG-16

• Transformation plug-and-play setting (classifier still trained on clean data)

T(·)

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Comparison with Other Transformation Methods

45

Defense Clean FGSM BIM

DeepFool

CW No Defense 0.930 0.582 0.180 0.176 0.094 MagNet[1] 0.904 0.615 0.431 0.654 0.485

PixelDefend[2]

0.853 0.681 0.773 0.741 0.758 STL 0.829 0.710 0.745 0.785 0.777 STL(Cluster) 0.836 0.711 0.753 0.796 0.791

}

Use a network to project

}

• Solve a sparse coding

Problem

• Not differentiable
• Transformation plug-and-play setting (classifier still trained on clean data)

T(·)

On CIFAR-10 by VGG-16

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Comparison with Other Transformation Methods

46

• If we retrain classifiers on transformed data:

Defense Clean FGSM BIM

DeepFool

CW No Defense 0.930 0.582 0.180 0.176 0.094 MagNet[1] 0.921 0.739 0.771 0.877 0.859

PixelDefend[2]

0.904 0.832 0.852 0.883 0.885 STL 0.900 0.852 0.875 0.884 0.888 STL(Cluster) 0.901 0.857 0.880 0.889 0.890

On CIFAR-10 by VGG-16

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

ImageNet, Top-1 acc, ResNet-50 (Resolution: 224*224):

Comparison with Other Transformation Methods

47

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Limitation: Weak Under White Box Attacks

48

Adversarial Defense by Stratified Convolutional Sparse Coding, Sun et al.

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Method

49

Optimization based transformation

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Outline

• Background and Motivation
• Project-based Defense Mechanism
• For 2D Image Classification
• For 3D Shape Classification
• Conclusion

50

Extending Adversarial Attacks AND Defenses To Deep 3D Point Cloud Classifiers, Liu et al, ICIP2019

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

3D Machine Learning on Different Representations

51

Primitive- based Point Cloud Mesh Volumetric Set Graph Sparse Array Hybrid Data Structure

Extending Adversarial Attacks AND Defenses To Deep 3D Point Cloud Classifiers, Liu et al, ICIP2019

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

3D Machine Learning on Different Representations

52

Primitive- based Point Cloud Mesh Volumetric Set Graph Sparse Array Hybrid Data Structure

Extending Adversarial Attacks AND Defenses To Deep 3D Point Cloud Classifiers, Liu et al, ICIP2019

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet — A Deep Network for Sets

53

(1,2,3) (1,1,1) (2,3,2) (2,3,4) simple symmetric function (e.g., max)

PointNet (vanilla)

h g γ

Observe:

f (x1,x2,…,xn) = γ ! g(h(x1),…,h(xn)) is symmetric if is symmetric

g

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, CVPR17, Qi et al

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet — A Deep Network for Sets

54

• Only skeleton points contributes to the classification score

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet — A Deep Network for Sets

55

• Only skeleton points contributes to the classification score
• Caused by the “max” operator

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet — A Deep Network for Sets

56

• Only skeleton points contributes to the classification score
• Caused by the “max” operator
• Builds a low-dimensional space!

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet can be Adversarially Attacked

57

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet can be Adversarially Attacked

58

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet can be Adversarially Attacked

59

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet can be Adversarially Attacked

60

Only salient points are moved!

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet can be Adversarially Attacked

61

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

PointNet can be Adversarially Attacked

62

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Evaluation

63

On 16 hand-picked object categories that humans can classify with 100% accuracy

Attack Method No Defense None 1.0 Fast Gradient L2 0.602 Iter Gradient L2 0.258 Iter Gradient L2 (clip norm) 0.548 Normalized Iter Gradient L2 0.355

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Defense by Adversarial Training (FGSM)

• Can be useful, but still lacks attack agnostic

64

Attack Method No Defense Adversarial Training None 1.0 0.995 Fast Gradient L2 0.602 0.927 Iter Gradient L2 0.258 0.629 Iter Gradient L2 (clip norm) 0.548 0.674 Normalized Iter Gradient L2 0.355 0.409

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Defense by Transformation

• An seemingly naive solution: “outlier removal”
• Outlier: high mean Euclidean distance of each point to its k-nearest

neighbors (k=10 in experiments)

• Outliers (over 1 standard deviation) are discarded

65

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Surprisingly, a Simple yet Good Defender

66

Attack Method No Defense Adversarial Training Removing Outliers None 1.0 0.995 0.974 Fast Gradient L2 0.602 0.927 0.954 Iter Gradient L2 0.258 0.629 0.838 Iter Gradient L2 (clip norm) 0.548 0.674 0.891 Normalized Iter Gradient L2 0.355 0.409 0.803

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Why?

67

• Because PointNet only relies on a small set of skeleton points!
• Removing outliers projects attacked shapes to the low-dim space of

skeletons

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Outline

• Background and Motivation
• Project-based Defense Mechanism
• For 2D Image Classification
• For 3D Shape Classification
• Summary

68

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

69

• Build a piece-wise linear image

space for natural images

• Projection by convolutional

sparse coding

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

70

• Build a piece-wise linear image

space for natural images

• Projection by convolutional

sparse coding

• Build a low-dimension shape

space by PointNet

• Projection by outlier removal

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

71

Conclusion and Future Work

• The projection methods in this talk are both very simple, yet they

achieve SOTA performance on attack agnostic defense

• The projection method itself has to be attack agnostic, thus we choose

not to use networks for this projection step

• Combining projection and adversarial learning may give better results

Attack-Agnostic Defense for 2D and 3D Recognition Hao Su UC San Diego

Acknowledgement

72

Bo Sun to be a Ph.D. at UT Austin Daniel Liu Torrey Pines High School Tiange Luo Peking University Ronald Yu UCSD Fangchen Liu UCSD Nian-hsuan Tsai National Tsinghua University

Bo Li, Assistant Professor at UIUC