adversarial robustness via runtime masking and cleansing
play

Adversarial Robustness via Runtime Masking and Cleansing Yi-Hsuan - PowerPoint PPT Presentation

Aug 21, 2023 •754 likes •1.3k views

Adversarial Robustness via Runtime Masking and Cleansing Yi-Hsuan Wu Chia-Hung Yuan Shan-Hung Wu Department of Computer Science, National Tsing Hua University, Taiwan International Conference on Machine Learning, 2020 Y.H. Wu, C.H. Yuan,

  1. Adversarial Robustness via Runtime Masking and Cleansing Yi-Hsuan Wu Chia-Hung Yuan Shan-Hung Wu Department of Computer Science, National Tsing Hua University, Taiwan International Conference on Machine Learning, 2020 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 1 / 34

  2. Why many adversarial defenses are broken? Deep neural networks are shown to be vulnerable to adversarial attacks, which motivates robust learning techniques https://www.tensorflow.org/tutorials/generative/images/adversarial_example.png 1 Athalye, A., Carlini, N., and Wagner, D. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. ICML’ 2018 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 3 / 34

  3. Why many adversarial defenses are broken? Deep neural networks are shown to be vulnerable to adversarial attacks, which motivates robust learning techniques https://www.tensorflow.org/tutorials/generative/images/adversarial_example.png A plethora of defenses have been proposed, however, many of these have been shown to fail 1 1 Athalye, A., Carlini, N., and Wagner, D. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. ICML’ 2018 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 3 / 34

  4. Why many adversarial defenses are broken? Recent study 2 shows the sample complexity of robust learning can be significantly larger than standard training 2 Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., and Madry, A. Adversarially robust generalization requires more data. NeurIPS, 2018 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 4 / 34

  5. Why many adversarial defenses are broken? Recent study 2 shows the sample complexity of robust learning can be significantly larger than standard training A theoretically grounded way to increase the adversarial robustness is to acquire more data 2 Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., and Madry, A. Adversarially robust generalization requires more data. NeurIPS, 2018 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 4 / 34

  6. Why many adversarial defenses are broken? Recent study 2 shows the sample complexity of robust learning can be significantly larger than standard training A theoretically grounded way to increase the adversarial robustness is to acquire more data This partially explains why the adversarial training, a data augmentation technique, is empirically strong 2 Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., and Madry, A. Adversarially robust generalization requires more data. NeurIPS, 2018 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 4 / 34

  7. Outline Goal 1 Related Works 2 Runtime Masking and Cleansing (RMC) 3 Experiments 4 Train-Time Attacks Defense-Aware Attacks Implications & Conclusion 5 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 5 / 34

  8. WebNN 3 Use a web-scale image database as a manifold and project a test image onto the manifold Make more robust prediction by taking only the projected image as inputs 3 Dubey, A., Maaten, L. v. d., Yalniz, Z., Li, Y., and Mahajan, D. Defense against adversarial images using web-scale nearest-neighbor search. CVPR, 2019 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 6 / 34

  9. Drawback: 50 Billion Images May be Too Large Web-scale database may not be available in other domains Performance drops when using smaller datasets Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 7 / 34

  10. Outline Goal 1 Related Works 2 Runtime Masking and Cleansing (RMC) 3 Experiments 4 Train-Time Attacks Defense-Aware Attacks Implications & Conclusion 5 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 8 / 34

  11. Goal Most existing defenses try to get more data at training time Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 9 / 34

  12. Goal Most existing defenses try to get more data at training time We propose a runtime defense Adapts network weights θ for a test point ˆ 1 x Makes inferecne ˆ y = f ( ˆ x ; θ ) 2 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 9 / 34

  13. Goal Most existing defenses try to get more data at training time We propose a runtime defense Adapts network weights θ for a test point ˆ 1 x Makes inferecne ˆ y = f ( ˆ x ; θ ) 2 Merits: Uses potentially large test data to improve adversarial robustness Is compatible with existing train-time defenses Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 9 / 34

  14. Challenge: Test Data are Unlabeled How to adapt network weights θ for unlabeled ˆ x ? Online adversarial training is not applicable Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 10 / 34

  15. Challenge: Test Data are Unlabeled How to adapt network weights θ for unlabeled ˆ x ? Online adversarial training is not applicable Extension: KNN-based online adversarial training For each ˆ x , find its KNN N ( ˆ x ; D ) from the training set D 1 Augment N ( ˆ x ; D ) with adversarial examples (cyan points) perturbed 2 from N ( ˆ x ; D ) Fine-tune the networks weights θ based on N ( ˆ x ; D ) 3 Inference ˆ y = f ( ˆ x ; θ ) 4 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 10 / 34

  16. Unfortunately, It Does Not Work! Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 11 / 34

  17. Unfortunately, It Does Not Work! Figure (b) shows a histogram of N ( ˆ x ; D ) w.r.t. di ff erent labels (x-axis) Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 11 / 34

  18. Unfortunately, It Does Not Work! Figure (b) shows a histogram of N ( ˆ x ; D ) w.r.t. di ff erent labels (x-axis) N ( ˆ x ; D ) contains examples of the same label The adversarial point ˆ x can mislead KNN selection Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 11 / 34

  19. Unfortunately, It Does Not Work! Figure (b) shows a histogram of N ( ˆ x ; D ) w.r.t. di ff erent labels (x-axis) N ( ˆ x ; D ) contains examples of the same label The adversarial point ˆ x can mislead KNN selection Therefore, the fine-tuned θ ends up being less robust Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 11 / 34

  20. Runtime Masking and Cleansing (RMC) RMC precomputes adversarial examples Augment D with adversarial examples to get D 0 1 x ; D ) 0 from D 0 Given a test point ˆ x , find its KNN N ( ˆ 2 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 12 / 34

  21. Runtime Masking and Cleansing (RMC) RMC precomputes adversarial examples Augment D with adversarial examples to get D 0 1 x ; D ) 0 from D 0 Given a test point ˆ x , find its KNN N ( ˆ 2 Adapt the networks weights θ based on N ( ˆ x ; D 0 ) 3 Inference ˆ y = f ( ˆ x ; θ ) 4 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 12 / 34

  22. Why Does It Work? x ; D 0 ) is no longer misled by the adversarial ˆ As Figure (c) shows, N ( ˆ x Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 13 / 34

  23. Why Does It Work? x ; D 0 ) is no longer misled by the adversarial ˆ As Figure (c) shows, N ( ˆ x Defense e ff ects: The diverse-labeled N ( ˆ x ; D 0 ) cleanses the θ of the non-robust patterns Also, dynamically masks the network gradients Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 13 / 34

  24. Outline Goal 1 Related Works 2 Runtime Masking and Cleansing (RMC) 3 Experiments 4 Train-Time Attacks Defense-Aware Attacks Implications & Conclusion 5 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 14 / 34

  25. Datasets MNIST CIFAR-10 ImageNet Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 15 / 34

  26. Outline Goal 1 Related Works 2 Runtime Masking and Cleansing (RMC) 3 Experiments 4 Train-Time Attacks Defense-Aware Attacks Implications & Conclusion 5 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 16 / 34

  27. MNIST & CIFAR-10 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 17 / 34

  28. ImageNet Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 18 / 34

  29. ImageNet For all datasets, RMC achieves the state-of-the-art robustness RMC yields significantly higher clean accuracy Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 18 / 34

  30. ImageNet For all datasets, RMC achieves the state-of-the-art robustness RMC yields significantly higher clean accuracy RMC does not enforce a smooth decision boundary Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 18 / 34

  31. ImageNet For all datasets, RMC achieves the state-of-the-art robustness RMC yields significantly higher clean accuracy RMC does not enforce a smooth decision boundary For gray- black-box attacks, please refer to our main paper Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 18 / 34

  32. Outline Goal 1 Related Works 2 Runtime Masking and Cleansing (RMC) 3 Experiments 4 Train-Time Attacks Defense-Aware Attacks Implications & Conclusion 5 Y.H. Wu, C.H. Yuan, S.H. Wu Runtime Masking and Cleansing ICML’20 19 / 34

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Reinforcing Adversarial Robustness using Model Confidence Induced by Adversarial Training Xi Wu

Reinforcing Adversarial Robustness using Model Confidence Induced by Adversarial Training Xi Wu

Reinforcing Adversarial Robustness using Model Confidence Induced by Adversarial Training Xi Wu xiwu@cs.wisc.edu Joint work with Uyeong Jang, Jiefeng Chen, Lingjiao Chen, and Somesh Jha July 19, 2018 Xi Wu Model Confidence and Adversarial

740 views • 9 slides
Adversarial Training and Robustness for Multiple Perturbations Poster #87 Florian Tramr &

Adversarial Training and Robustness for Multiple Perturbations Poster #87 Florian Tramr &

Adversarial Training and Robustness for Multiple Perturbations Poster #87 Florian Tramr & Dan Boneh NeurIPS 2019 Adversarial examples 88% Tabby Cat 99% Guacamole Szegedy et al., 2014 Goodfellow et al., 2015 Athalye, 2017 Adversarial

564 views • 23 slides
Adversarial Domain Adaptation and Adversarial Robustness Judy Hoffman + = Big Deep success

Adversarial Domain Adaptation and Adversarial Robustness Judy Hoffman + = Big Deep success

Adversarial Domain Adaptation and Adversarial Robustness Judy Hoffman + = Big Deep success data learning Benchmark Performance 100 95 Accuracy 90 85 Millions of Images 80 Deep models 75 Challenge to recognize 1000

1.2k views • 60 slides
Improving Adversarial Robustness via Promoting Ensemble Diversity Tianyu

Improving Adversarial Robustness via Promoting Ensemble Diversity Tianyu

Improving Adversarial Robustness via Promoting Ensemble Diversity Tianyu Pang, Kun Xu, Chao Du, Ning Chen and Jun Zhu Department of Computer Science and Technology Tsinghua University TSAIL ICML | 2019 Adversarial

864 views • 14 slides
Adversarial Approaches to Bayesian Learning and Bayesian Approaches to Adversarial Robustness

Adversarial Approaches to Bayesian Learning and Bayesian Approaches to Adversarial Robustness

Adversarial Approaches to Bayesian Learning and Bayesian Approaches to Adversarial Robustness Ian Goodfellow, OpenAI Research Scientist NIPS 2016 Workshop on Bayesian Deep Learning Barcelona, 2016-12-10 Speculation on Three Topics Can we

769 views • 21 slides
Limits on Robustness to Adversarial Examples Elvis Dohmatob Criteo AI Lab October 2, 2019 Elvis

Limits on Robustness to Adversarial Examples Elvis Dohmatob Criteo AI Lab October 2, 2019 Elvis

Preliminaries on adversarial robustness Classifier-dependent lower bounds Universal lower bounds Limits on Robustness to Adversarial Examples Elvis Dohmatob Criteo AI Lab October 2, 2019 Elvis Dohmatob Limits on Robustness to Adversarial

763 views • 74 slides
Adversarial Robustness for Code Pavol Bielik , Martin Vechev pavol.bielik@inf.ethz.ch,

Adversarial Robustness for Code Pavol Bielik , Martin Vechev pavol.bielik@inf.ethz.ch,

ICML 2020 Adversarial Robustness for Code Pavol Bielik , Martin Vechev pavol.bielik@inf.ethz.ch, martin.vechev@inf.ethz.ch Department of Computer Science 1 Adversarial Robustness panda gibbon Vision + = Explaining and Harnessing

429 views • 32 slides
Certified Adversarial Robustness via Randomized Smoothing Jeremy Cohen Elan Rosenfeld

Certified Adversarial Robustness via Randomized Smoothing Jeremy Cohen Elan Rosenfeld

Certified Adversarial Robustness via Randomized Smoothing Jeremy Cohen Elan Rosenfeld Zico Kolter Carnegie Mellon University Introduction We study a certified adversarial defense in " norm which scales to ImageNet

1.5k views • 12 slides
On the Connection Between Adversarial Robustness and Saliency Map Interpretability Christian

On the Connection Between Adversarial Robustness and Saliency Map Interpretability Christian

On the Connection Between Adversarial Robustness and Saliency Map Interpretability Christian Etmann , 1 , 3 , Sebastian Lunz , 2 , Peter Maass 1 , Carola-Bibiane Sch onlieb 2 13th June, 2019 1: ZeTeM, University of Bremen, 2: Cambridge

673 views • 19 slides
Adversarial Robustness of Machine Learning Models for Graphs Prof. Dr. Stephan Gnnemann

Adversarial Robustness of Machine Learning Models for Graphs Prof. Dr. Stephan Gnnemann

Adversarial Robustness of Machine Learning Models for Graphs Prof. Dr. Stephan Gnnemann Department of Informatics Technical University of Munich 28.10.2019 Adversarial Robustness of Machine Learning Models for Graphs S. Gnnemann Can you

669 views • 27 slides
ON THE ADVERSARIAL ROBUSTNESS OF UNCERTAINTY AWARE DEEP NEURAL NETWORKS APRIL 29 TH , 2019

ON THE ADVERSARIAL ROBUSTNESS OF UNCERTAINTY AWARE DEEP NEURAL NETWORKS APRIL 29 TH , 2019

ON THE ADVERSARIAL ROBUSTNESS OF UNCERTAINTY AWARE DEEP NEURAL NETWORKS APRIL 29 TH , 2019 PREPARED BY: ALI HARAKEH QUESTION Can a neural network mitigate the effects of adversarial attacks by estimating the uncertainty in its predictions ?

1.14k views • 26 slides
Adversarial Robustness: Theory and Practice Zico Kolter Aleksander Mdry madry-lab.ml

Adversarial Robustness: Theory and Practice Zico Kolter Aleksander Mdry madry-lab.ml

Adversarial Robustness: Theory and Practice Zico Kolter Aleksander Mdry madry-lab.ml Tutorial website: @zicokolter @aleks_madry adversarial-ml-tutorial.org Machine Learning: The Success Story Image classification Reinforcement Learning

528 views • 49 slides
Toward Adversarial Robustness by Diversity in an Ensemble of Specialized Deep Neural Networks

Toward Adversarial Robustness by Diversity in an Ensemble of Specialized Deep Neural Networks

Toward Adversarial Robustness by Diversity in an Ensemble of Specialized Deep Neural Networks presented at Canadian AI 2020 Mahdieh Abbasi 1 , Arezoo Rajabi 2 , Christian Gagn 1,3 , and Rakesh B. Bobba 2 1. IID, Universit Laval, Qubec,

728 views • 20 slides
Adversarial Robustness for Aligned AI Ian Goodfellow, Sta ff Research NIPS 2017 Workshop on

Adversarial Robustness for Aligned AI Ian Goodfellow, Sta ff Research NIPS 2017 Workshop on

Adversarial Robustness for Aligned AI Ian Goodfellow, Sta ff Research NIPS 2017 Workshop on Aligned Artificial Intelligence Many thanks to Catherine Olsson for feedback on drafts The Alignment Problem (This is now fixed. Dont try it!)

626 views • 30 slides
Certified Robustness to Adversarial Examples with Di ff erential Privacy Mathias Lcuyer,

Certified Robustness to Adversarial Examples with Di ff erential Privacy Mathias Lcuyer,

Certified Robustness to Adversarial Examples with Di ff erential Privacy Mathias Lcuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, Suman Jana Columbia University Code: https://github.com/columbia/pixeldp Contact:

680 views • 48 slides
Benchmarking Adversarial Robustness on Image Classification Yinpeng Dong, Qi-An Fu, Xiao Yang,

Benchmarking Adversarial Robustness on Image Classification Yinpeng Dong, Qi-An Fu, Xiao Yang,

Benchmarking Adversarial Robustness on Image Classification Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Zihao Xiao, Hang Su, Jun Zhu Dept. of Comp. Sci. and Tech., BNRist Center, Institute for AI, THBI Lab, Tsinghua University, Beijing,

417 views • 6 slides
Mag Net A Two-Pronged Defense against Adversarial Examples Dongyu Meng Hao Chen ShanghaiTech

Mag Net A Two-Pronged Defense against Adversarial Examples Dongyu Meng Hao Chen ShanghaiTech

Mag Net A Two-Pronged Defense against Adversarial Examples Dongyu Meng Hao Chen ShanghaiTech University, China University of California, Davis, USA Neural networks in real-life applications user authentication autonomous

599 views • 30 slides
Defense Against Adversarial Images using Web-Scale Nearest-Neighbor Search Abhimanyu Dubey,

Defense Against Adversarial Images using Web-Scale Nearest-Neighbor Search Abhimanyu Dubey,

Defense Against Adversarial Images using Web-Scale Nearest-Neighbor Search Abhimanyu Dubey, Laurens van der Maaten, I. Zeki Yalniz, Yixuan Li and Dhruv Mahajan Adversarial Images adversarial swan pelican perturbation

457 views • 8 slides
Hash-flooding DoS reloaded: Hash flooding begins? attacks and defenses July 1998 article

Hash-flooding DoS reloaded: Hash flooding begins? attacks and defenses July 1998 article

Hash-flooding DoS reloaded: Hash flooding begins? attacks and defenses July 1998 article Jean-Philippe Aumasson, Designing and attacking Kudelski Security (NAGRA) port scan detection tools by Solar Designer (Alexander D. J. Bernstein,

1.83k views • 164 slides
Lecture 08 Control-flow Hijacking Defenses Stephen Checkoway University of Illinois at

Lecture 08 Control-flow Hijacking Defenses Stephen Checkoway University of Illinois at

Lecture 08 Control-flow Hijacking Defenses Stephen Checkoway University of Illinois at Chicago CS 487 Fall 2017 Slides adapted from Miller, Bailey, and Brumley Control Flow Hijack: Always control + computation shellcode (aka payload)

652 views • 36 slides
Offensive Threat Modeling for Attackers turning threat modeling on its head Rafal M. Los

Offensive Threat Modeling for Attackers turning threat modeling on its head Rafal M. Los

Offensive Threat Modeling for Attackers turning threat modeling on its head Rafal M. Los Chief Security Evangelist HP Software Shane MacDougall Principal Tactical Intelligence Modern threat modeling is a defensive response to

783 views • 52 slides
Cybercrime Kill Chain vs. Effec4veness of Defense Layers

Cybercrime Kill Chain vs. Effec4veness of Defense Layers

Cybercrime Kill Chain vs. Effec4veness of Defense Layers Dr. Stefan Frei & Francisco Arts @stefan_frei @franklyfranc

787 views • 60 slides
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples Anish Athalye* 1 , Nicholas Carlini * 2 , and David Wagner 3 1 Massachusetts Institute of Technology 2 University of California, Berkeley (now

1.13k views • 67 slides
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples Anish Athalye* 1 , Nicholas Carlini * 2 , and David Wagner 3 1 Massachusetts Institute of Technology 2 University of California, Berkeley (now

540 views • 50 slides

More recommend