Adversarial Training and Provable Defenses: Bridging the Gap S 0 - PowerPoint PPT Presentation

Adversarial Training and Provable Defenses: Bridging the Gap

𝑀 ∞

𝑦 S 0 𝑦 𝑙 ∘ ℎ 𝜄 𝑙−1 ∘ ⋯ ∘ ℎ 𝜄 1 ℎ 𝜄 = ℎ 𝜄 1 2 3 ℎ 𝜄 ℎ 𝜄 ℎ 𝜄 Conv + ReLU Conv + ReLU Linear ′ = ℎ 𝜄 (𝑦′) 𝑦 ′ ∈ 𝑇 0 (𝑦) ′ ′ 𝑦 1 𝑦 3 𝑦 2

𝑑 𝑈 ℎ 𝜄 𝑦 ′ + 𝑒 < 0, ∀𝑦 ′ ∈ 𝑇 0 (𝑦) 1 2 3 ℎ 𝜄 ℎ 𝜄 ℎ 𝜄 Conv + ReLU Conv + ReLU Linear ′ = ℎ 𝜄 (𝑦′) 𝑦 ′ ∈ 𝑇 0 (𝑦) ′ ′ 𝑦 1 𝑦 3 𝑦 2

1 2 3 ℎ 𝜄 ℎ 𝜄 ℎ 𝜄 Conv + ReLU Conv + ReLU Check output condition: Linear ′ + 𝑒 < 0, ∀𝑦 3 ′ ∈ 𝐷 3 𝑦 𝑑 𝑈 𝑦 3 𝐷 0 𝑦 = 𝑇 0 (𝑦) 𝐷 1 𝑦 𝐷 2 𝑦 𝐷 3 𝑦 Guarantees: 𝑑 𝑈 ℎ 𝜄 𝑦 ′ + 𝑒 < 0, ∀𝑦 ′ ∈ 𝑇 0 (𝑦)

ℒ 𝑦 ′ ∈𝑇 0 (𝑦) ℒ(ℎ 𝜄 𝑦 ′ , 𝑧) min 𝜄 𝐹 𝑦,𝑧 ~𝐸 max lower upper

upper • • lower • • • •

1 2 3 ℎ 𝜄 ℎ 𝜄 ℎ 𝜄 ′ ′ ′ 𝑦 1 𝑦 2 𝑦 3 ′ 𝑦 2 ′ 𝑦 3 ′ 𝑦 1 𝐷 0 𝑦 = 𝑇 0 (𝑦) 𝐷 1 𝑦 𝐷 2 𝑦 𝐷 3 𝑦 ′ + 𝑒 < 0 → certification fails 𝑑 𝑈 𝑦 3

𝑇 0 (𝑦) 𝐷 1 𝑦 , 𝐷 2 𝑦 , 𝐷 3 (𝑦)

2 1 3 ℎ 𝜄 ℎ 𝜄 ℎ 𝜄 Conv + ReLU Conv + ReLU ′ ′ 𝑦 2 𝑦 1 Linear ′ , 𝑧) ℒ(𝑦 3 ′ , 𝑧) 𝛼 𝜄 ℒ(𝑦 3 ′ 𝑦 2 ′ 𝑦 3 ′ 𝑦 1 𝐷 0 𝑦 = 𝑇 0 (𝑦) 𝐷 1 𝑦 𝐷 2 𝑦 𝐷 3 𝑦

projection

𝐷 𝑚 𝑦 = 𝑏 𝑚 + 𝐵 𝑚 𝑓 𝑓 ∈ −1, 1 𝑛 𝑚 𝑏 𝑚 𝐵 𝑚 𝑀 ∞ 𝜗 𝑏 0 = 𝑦 𝐵 0 = 𝜗𝐽

Key idea 𝑦 ′ = 𝑏 𝑚 + 𝐵 𝑚 𝑓 ′ 𝑦 1 𝑓 1 ′ 𝑓 2 𝑦 2 ′ ≔ 2𝑓 1 − 𝑓 2 𝑦 1 ′ ≔ 𝑓 1 + 𝑓 2 𝑦 2

Method Accuracy (%) Certified Robustness (%)

Method Accuracy (%) Certified Robustness (%)