Attacks Meet Interpretability: Attribute-steered Detection of - - PowerPoint PPT Presentation

Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples

Guanhong Tao, Shiqing Ma, Yingqi Liu, Xiangyu Zhang

Understanding Adversarial Samples

2

Model A.J. Buckley Legitimate input C&W2 attack Isla Fisher Pixel-wise Differences (×50 times)

Understanding Adversarial Samples

2

Model A.J. Buckley Legitimate input C&W2 attack Isla Fisher Human Pixel-wise Differences (×50 times)

Understanding Adversarial Samples

2

Model A.J. Buckley Legitimate input C&W2 attack Isla Fisher Human Pixel-wise Differences (×50 times)

Understanding Adversarial Samples

2

Model A.J. Buckley Legitimate input C&W2 attack Isla Fisher Human Pixel-wise Differences (×50 times)

Understanding Adversarial Samples

2

Model A.J. Buckley Legitimate input C&W2 attack Isla Fisher Human Pixel-wise Differences (×50 times)

Understanding Adversarial Samples

2

Model A.J. Buckley Legitimate input C&W2 attack Isla Fisher Human Pixel-wise Differences (×50 times)

• Idea: is the classification result of a model mainly based on human

perceptible attributes?

Understanding Adversarial Samples

2

Model A.J. Buckley Legitimate input C&W2 attack Isla Fisher Human Pixel-wise Differences (×50 times)

Architecture of AmI

3

Architecture of AmI

3

Input

Architecture of AmI

3

1

Landmark generation Input

Architecture of AmI

3

1

Landmark generation

2

✓ Left eye ✓ Right eye ✓ Nose ✓ Mouth ✓ …

Attribute annotation Input

Architecture of AmI

3

1

Landmark generation

2

✓ Left eye ✓ Right eye ✓ Nose ✓ Mouth ✓ …

Attribute annotation

3

Attribute witness extraction Input

Architecture of AmI

3

1

Landmark generation

2

✓ Left eye ✓ Right eye ✓ Nose ✓ Mouth ✓ …

Attribute annotation

3

Attribute witness extraction

4

Attribute-steered model Input

Architecture of AmI

3

1

Landmark generation

2

✓ Left eye ✓ Right eye ✓ Nose ✓ Mouth ✓ …

Attribute annotation

3

Attribute witness extraction

4

Attribute-steered model Original model Input

Architecture of AmI

3

1

Landmark generation

2

✓ Left eye ✓ Right eye ✓ Nose ✓ Mouth ✓ …

Attribute annotation

3

Attribute witness extraction

4

Attribute-steered model Original model

⊖

Input

5

Consistency

• bserver

Architecture of AmI

3

1

Landmark generation

2

✓ Left eye ✓ Right eye ✓ Nose ✓ Mouth ✓ …

Attribute annotation

3

Attribute witness extraction

4

Attribute-steered model Original model

⊖

Input

5

Consistency

• bserver

• Are there correspondences between attributes and neurons?
• If yes, how to extract corresponding neurons?

Challenges

4

• Are there correspondences between attributes and neurons?
• If yes, how to extract corresponding neurons?
• Propose: Bi-directional reasoning

Challenges

4

• Are there correspondences between attributes and neurons?
• If yes, how to extract corresponding neurons?
• Propose: Bi-directional reasoning
• Forward: attribute changes —> neuron activation changes

Challenges

4

• Are there correspondences between attributes and neurons?
• If yes, how to extract corresponding neurons?
• Propose: Bi-directional reasoning
• Forward: attribute changes —> neuron activation changes

Challenges

4

• Backward: neuron activation changes —> attribute changes

• Are there correspondences between attributes and neurons?
• If yes, how to extract corresponding neurons?
• Propose: Bi-directional reasoning
• Forward: attribute changes —> neuron activation changes

Challenges

4

• Backward: neuron activation changes —> attribute changes
• Backward: no attribute changes —> no neuron activation changes

Attribute Witness Extraction

5

Attribute Witness Extraction

5

Input

Attribute Witness Extraction

5

Input

Model

Attribute substitution

A

Model

⊖

C

Feature variants

Attribute Witness Extraction

5

Input

Model

Attribute substitution Attribute preservation

A

B

Model Model Model

⊖ ⊖

C

D

Feature variants Feature invariants

Attribute Witness Extraction

5

Input

Model

Attribute substitution Attribute preservation

A

B

Model Model Model

⊖ ⊖

C

D

Feature variants Feature invariants

E

Attribute witnesses

Experimental Results

6

Experimental Results

6

• Attribute witnesses

Experimental Results

6

• Attribute witnesses
• The number of witnesses extracted is smaller than 20, although there are 64-4096

neurons in each layer

Experimental Results

6

• Attribute witnesses
• The number of witnesses extracted is smaller than 20, although there are 64-4096

neurons in each layer

• Adversary detection

Experimental Results

6

• Attribute witnesses
• The number of witnesses extracted is smaller than 20, although there are 64-4096

neurons in each layer

• Adversary detection
• Achieve 94% detection accuracy for 7 different kinds of attacks with 9.91% false

positives on benign inputs

Experimental Results

6

• Attribute witnesses
• The number of witnesses extracted is smaller than 20, although there are 64-4096

neurons in each layer

• Adversary detection
• Achieve 94% detection accuracy for 7 different kinds of attacks with 9.91% false

positives on benign inputs

• A state-of-the-art technique Feature Squeezing (NDSS '18) can only achieve 55%

accuracy with 23.3% false positives for face recognition systems

Thank you! Please visit our poster #99

05:00-07:00 PM @ Room 210 & 230 AB