SLIDE 1
Fabrizio Falchi
ISTI, CNR, Pisa, italy
www.fabriziofalchi.it
ATTACKING DEEP NEURAL NETWORKS
WITH ADVERSARIAL IMAGES
COST ACTION CA16101 - Dubrovnik, November 7th
fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it fabrizio.falchi@cnr.it - - PowerPoint PPT Presentation
fabrizio.falchi@cnr.it fabrizio.falchi@cnr.it - - PowerPoint PPT Presentation
A TTACKING D EEP N EURAL N ETWORKS WITH A DVERSARIAL I MAGES Fabrizio Falchi ISTI, CNR, Pisa, italy www.fabriziofalchi.it COST ACTION CA16101 - Dubrovnik, November 7th fabrizio.falchi@cnr.it fabrizio.falchi@cnr.it fabrizio.falchi@cnr.it W HAT
SLIDE 2
SLIDE 3
fabrizio.falchi@cnr.it
SLIDE 4
fabrizio.falchi@cnr.it
SLIDE 5
fabrizio.falchi@cnr.it
WHAT’S THAT?
SLIDE 6
fabrizio.falchi@cnr.it
WHAT’S THAT?
SLIDE 7
fabrizio.falchi@cnr.it
WHAT’S THAT?
SLIDE 8
fabrizio.falchi@cnr.it
WHAT’S THAT?
SLIDE 9
fabrizio.falchi@cnr.it
ADVERSARIAL EXAMPLES
SLIDE 10
fabrizio.falchi@cnr.it
ILLUSIONS
Edward H. Adelson
SLIDE 11
fabrizio.falchi@cnr.it
ILLUSIONS
Edward H. Adelson
SLIDE 12
fabrizio.falchi@cnr.it
SLIDE 13
fabrizio.falchi@cnr.it
DUBROVNIK
SLIDE 14
fabrizio.falchi@cnr.it
DUBROVNIK – DEEP DREAM
SLIDE 15
KNOW YOUR ENEMY
SLIDE 16
fabrizio.falchi@cnr.it
ADVERSARY
Goal Knowledge Capability
SLIDE 17
ADVERSARY’S GOAL
SLIDE 18
fabrizio.falchi@cnr.it
GENUINE IMAGES
… Mushrooms Pineapple Toucan …
SLIDE 19
fabrizio.falchi@cnr.it
NON-TARGETED ATTACK
19
… Mushrooms <whatever> …
+ =
NON-TARGETED
Goal
SLIDE 20
fabrizio.falchi@cnr.it
TARGETED ATTACK
20
… Mushrooms Toucan …
+ =
TARGETED
Goal
SLIDE 21
fabrizio.falchi@cnr.it
Goal Knowledge Capability
SLIDE 22
fabrizio.falchi@cnr.it
Slide credit: Biggio
SLIDE 23
fabrizio.falchi@cnr.it
ATTACKING DEEP NEURAL NETWORKS
SLIDE 24
fabrizio.falchi@cnr.it
BLACK BOX ADVERSARIAL EXAMPLE ATTACKS
Practical Black-Box Attacks against Machine Learning Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, Ananthram Swami
SLIDE 25
ATTACKING FACE RECOGNITION SYSTEMS
SLIDE 26
fabrizio.falchi@cnr.it
ADVERIAL FACES
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi
SLIDE 27
fabrizio.falchi@cnr.it
ADVERSARIAL FACES
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi
SLIDE 28
fabrizio.falchi@cnr.it
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi
SLIDE 29
fabrizio.falchi@cnr.it
ADVERSARIAL FACES
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi
SLIDE 30
fabrizio.falchi@cnr.it
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi
SLIDE 31
fabrizio.falchi@cnr.it
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi
SLIDE 32
ATTACKING IN REAL WORLD
SLIDE 33
fabrizio.falchi@cnr.it
ADVERSARIAL IMAGE
33 Photo: labsix
SLIDE 34
fabrizio.falchi@cnr.it
ROTATE ADVERSARIAL IMAGE
34 Photo: labsix
SLIDE 35
fabrizio.falchi@cnr.it
SLIDE 36
fabrizio.falchi@cnr.it
36
SLIDE 37
fabrizio.falchi@cnr.it
Robust Physical-World Attacks on Deep Learning Models Eykholt, Evtimov, Fernandes, Bo Li, Rahmati, Xiao, Prakash, Kohno, Song
SLIDE 38
fabrizio.falchi@cnr.it
SLIDE 39
fabrizio.falchi@cnr.it
Adversarial Generative Nets: Neural Network Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter
SLIDE 40
fabrizio.falchi@cnr.it
ATTACKING DNN IN REAL WORLD
Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter
SLIDE 41
fabrizio.falchi@cnr.it
ATTACKING DNN IN REAL WORLD
Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter
SLIDE 42
fabrizio.falchi@cnr.it
ATTACKING DNN IN REAL WORLD
Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter
SLIDE 43
ATTACKING FACE VERIFICATION SYSTEMS
SLIDE 44
fabrizio.falchi@cnr.it
ID1 ID2 ID3 ... ID10 ... IDn
FACE RCOGNITION
SLIDE 45
fabrizio.falchi@cnr.it
FACE VERIFICATION
SLIDE 46
fabrizio.falchi@cnr.it
FACE VERIFIATION
Unravelling Robustness of Deep Learning based Face Recognition Against Adversarial Attacks Goswami, Ratha, Agarwal, Singh, Vatsa
46
SLIDE 47
fabrizio.falchi@cnr.it
FACE VERIFIATION
Unravelling Robustness of Deep Learning based Face Recognition Against Adversarial Attacks Goswami, Ratha, Agarwal, Singh, Vatsa
47
SLIDE 48
ADVERSARY-AWARE MACHINE LEARNING
SLIDE 49
fabrizio.falchi@cnr.it
ADVERSARY-AWARE MACHINE LEARNING
49
Machine learning system should be aware of the arms race with the adversary
Security evaluation of pattern classifiers under attack Biggio, Fumera, Roli
SLIDE 50
fabrizio.falchi@cnr.it
SLIDE 51
ADVERSARIAL EXAMPLE DETECTION
SLIDE 52
fabrizio.falchi@cnr.it
GENIUNE IMAGES
52
… Mushrooms Pineapple Toucan …
SLIDE 53
fabrizio.falchi@cnr.it
NON-TARGETED ATTACK
53
… Mushrooms <whatever> …
+ =
SLIDE 54
fabrizio.falchi@cnr.it
DEFENSE
54
… Mushrooms …
+ =
Increase robustness
SLIDE 55
fabrizio.falchi@cnr.it
… Mushrooms …
DETECTION
55
+ =
Attack detection
SLIDE 56
fabrizio.falchi@cnr.it
ADVERSARIAL EXAMPLES DETECTION
Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter
SLIDE 57
OUR APPROACH
SLIDE 58
fabrizio.falchi@cnr.it
DEEP LEARNING (FROM NATURE)
AI
Machine Learning Repres. Learning Deep Learning
SLIDE 59
fabrizio.falchi@cnr.it
DEEP LEARNING (FROM NATURE)
Representation learning methods that allow a machine to be fed with raw data and to automatically discover the representations needed for detection or classification. Deep-learning are representation learning methods
- with multiple levels of representation, obtained by
- composing simple but non-linear modules that each
- transform the representation at one level
into a representation at a higher, slightly more abstract level.
SLIDE 60
fabrizio.falchi@cnr.it
MULTIPLE LEVELS OF ABSTRACTION
SLIDE 61
fabrizio.falchi@cnr.it
MULTIPLE LEVELS OF ABSTRACTION
SLIDE 62
fabrizio.falchi@cnr.it
OUR APPROACH
A detection scheme for adversarial images based on internal representation (aka deep features) of the neural network classifier.
- Main intuition: look at the evolution of features, i.e. the path formed by their positions in
the feature spaces, during the forward pass of the network.
- Claim: The trajectories traced by authentic inputs and adversarial examples differ and
can be used to discern them.
Adversarial examples detection in features distance spaces
- F. Carrara, R. Becarelli, R. Caldelli, F. Falchi, G. Amato
ECCV WOCM Workshop 2018
SLIDE 63
fabrizio.falchi@cnr.it
MULTIPLE LEVELS OF ABSTRACTION
SLIDE 64
fabrizio.falchi@cnr.it
OUR APPROACH: RESULTS
SLIDE 65
fabrizio.falchi@cnr.it
EASY TO IDENTIFY ADVERSARIAL IMAGES
66
SLIDE 66
fabrizio.falchi@cnr.it
HARD TO IDENTIFY ADVERSARIAL IMAGES
SLIDE 67
fabrizio.falchi@cnr.it
OTHER DETECTION APPROACHES
- Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods [2017]
Nicholas Carlini, David Wagner
- On Detecting Adversarial Perturbations [2017]
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, Bastian Bischoff
- Trace and detect adversarial attacks on CNNs using feature response maps [2018]
Mohammadreza, Friedhelm, Thilo
- Adversarial examples detection in features distance spaces [2018]
- F. Carrara, R. Becarelli, R. Caldelli, F. Falchi, G. Amato
SLIDE 68
RELATED TOPICS
SLIDE 69
fabrizio.falchi@cnr.it
DETECTING FACE MORPHING ATTACKS
Detection of Face Morphing Attacks by Deep Learning
- C. Seibold, W. Samek, A. Hilsmann, P. Eisert
SLIDE 70
fabrizio.falchi@cnr.it
ADVERSARIAL EXAMPLES DETECTION
HiDDeN: Hiding Data With Deep Networks Jiren Zhu, Russell Kaplan, Justin Johnson, Li Fei-Fei
SLIDE 71
fabrizio.falchi@cnr.it
SLIDE 72
fabrizio.falchi@cnr.it
SLIDE 73
fabrizio.falchi@cnr.it
THANKS!
Questions are welcomed
Fabrizio Falchi
fabrizio.falchi@cnr.it
SLIDE 74
fabrizio.falchi@cnr.it
CONCLUSIONS
- Machine Learning and Deep Learning in particular can be attacked
- Slightly modifying images but also in real world
- Even if our neural network is a black box for the enemy
- Many approaches have been proposed to make DL more robust
- Adversarial examples detection is its early stages
- We need adversary-aware machine learning