SLIDE 1
Cryptanalytic Extraction
- f Neural Network Models
Nicholas Carlini1, Matthew Jagielski12, Ilya Mironov13
1Google, 2Northeastern, 3Facebook
W
Given: Given: Given: Solve For
Cryptanalytic Extraction of Neural Network Models Nicholas Carlini 1 - - PowerPoint PPT Presentation
Cryptanalytic Extraction of Neural Network Models Nicholas Carlini 1 - - PowerPoint PPT Presentation
Cryptanalytic Extraction of Neural Network Models Nicholas Carlini 1 , Matthew Jagielski 12 , Ilya Mironov 13 1 Google, 2 Northeastern, 3 Facebook Solve For W Given: Given: Given: CAT Our Question: Given query access to a neural network,
SLIDE 2
SLIDE 3
Our Question:
Given query access to a neural network, can we extract the hidden parameters?
SLIDE 4
Two views of the problem
Machine Learning (function approximation) Mathematical (direct analysis)
SLIDE 5
Our Question:
Given query access to a neural network, can we extract the hidden parameters?
SLIDE 6
Our Result: Yes.*
* For small fully connected neural networks with ReLU activations with a few layers evaluated in float64 precision and fully precise inputs and outputs as long as the network isn't pathologically worst-case (e.g., a reduction from 3-SAT) and even then we can only get functional equivalence because exact extraction is provably impossible and even then we only get up to 40 bits of precision when we could theoretically hope for up to 56 bits of precision with float64.
SLIDE 7
Neural Networks 101
SLIDE 8
SLIDE 9
x y x y x y
SLIDE 10
x y x y x y h1 h2 h3
SLIDE 11
h1 h2 h3 h1 h2 h3 h1 h2 h3
SLIDE 12
h1 h2 h3 h1 h2 h3 h1 h2 h3
SLIDE 13
h4 h5 h6
SLIDE 14
h4 h5 h6
SLIDE 15
z
SLIDE 16
SLIDE 17
Σ a1 a2 x1 x2
SLIDE 18
Σ x1 x2 a1 a2
SLIDE 19
Σ x1 x2 a1 a2
SLIDE 20
ReLU(x) = max(x, 0)
SLIDE 21
Σ x1 x2 a1 a2
SLIDE 22
Σ
SLIDE 23
Σ Σ Σ Σ Σ Σ Σ
SLIDE 24
Extracting Neural Networks
SLIDE 25
Given (oracle) query access to a neural network, can we extract model? the exact
SLIDE 26
can we extract Given (oracle) query access to a neural network, model? a functionally equivalent
SLIDE 27
Given (oracle) query access to a neural network, can we extract model? a functionally equivalent
SLIDE 28
Given (oracle) query access to a neural network, can we extract model? a functionally equivalent learned through stochastic gradient descent,
SLIDE 29
Given (oracle) query access to a neural network, can we extract model? a functionally equivalent learned through stochastic gradient descent, This paper: yes (empirically)
SLIDE 30
Reduced Round Attack: 1 Hidden Layer
[MSDH19, JCB+20]
SLIDE 31
Visual Intuition
[MSDH19, JCB+20]
SLIDE 32
[MSDH19, JCB+20]
SLIDE 33
SLIDE 34
SLIDE 35
SLIDE 36
SLIDE 37
(+, +, +) (-, -, -) (+, -, -) (-, +, +) (
- ,
- ,
+ )
SLIDE 38
(+, +, +) (-, -, -) (+, -, -)
SLIDE 39
SLIDE 40
SLIDE 41
Observation #1: location of the critical hyperplanes almost completely determines the neural network
SLIDE 42
SLIDE 43
[MSDH19, JCB+20]
x y u v w
SLIDE 44
[MSDH19, JCB+20]
x+ε y u' ? v' w+ɑε
SLIDE 45
[MSDH19, JCB+20]
x+ε y+δ u'' v'' w+ɑε +ɣδ
SLIDE 46
[MSDH19, JCB+20]
x+ε y+δ u'' v'' w+ɑε +ɣδ a1 a2
SLIDE 47
[MSDH19, JCB+20]
x z w a1 a2 = a--ε b+δ
SLIDE 48
however....
SLIDE 49
ε δ
SLIDE 50
SLIDE 51
SLIDE 52
SLIDE 53
Observation #2: local information is insufficient to recover neuron signs
SLIDE 54
SLIDE 55
SLIDE 56
SLIDE 57
Finding witnesses to each neuron
SLIDE 58
u v
SLIDE 59
SLIDE 60
u v
SLIDE 61
Our Contributions
SLIDE 62
Our Contributions
- 1. Extract deep models
- 2. Efficient extraction
- 3. High Fidelity Extraction
SLIDE 63
Our Contributions
- 1. Extract deep models
- 2. Efficient extraction
- 3. High Fidelity Extraction
SLIDE 64
Our Contributions
- 1. Extract
- 2. Efficient extraction
- 3. High Fidelity Extraction
deep models
SLIDE 65
Our Contributions
- 1. Extract
- 2. Efficient extraction
- 3. High Fidelity Extraction
deep models 2-
SLIDE 66
Our Contributions
- 1. Extract
deep models 2-
- 2. Efficient extraction
- 3. High Fidelity Extraction
SLIDE 67
Our Contributions
- 1. Extract
deep models 2-
- a. Recover weight values
- b. Recover neuron signs
SLIDE 68
2-deep Neural Network
SLIDE 69
(+,+,+) (+,-,-) (-,+,+) (+,-,-) (-,-,-) (-,+,+)
SLIDE 70
SLIDE 71
SLIDE 72
Recovering the first layer (up to sign)
SLIDE 73
SLIDE 74
SLIDE 75
SLIDE 76
SLIDE 77
SLIDE 78
SLIDE 79
SLIDE 80
SLIDE 81
SLIDE 82
SLIDE 83
SLIDE 84
SLIDE 85
SLIDE 86
Recovering the first layer sign
SLIDE 87
SLIDE 88
SLIDE 89
SLIDE 90
SLIDE 91
SLIDE 92
SLIDE 93
Hyperplane Following
SLIDE 94
SLIDE 95
SLIDE 96
SLIDE 97
SLIDE 98
SLIDE 99
SLIDE 100
SLIDE 101
... then peel off first weight, and re-run attack from there
SLIDE 102
Key Challenges
(that I don't have time to discuss in this talk, but make up most of the technical work that we had to do in the paper)
SLIDE 103
Bounded floating point precision
SLIDE 104
Not all hidden states are reachable
SLIDE 105
Results
SLIDE 106
Results
SLIDE 107
Results
SLIDE 108
Results
SLIDE 109
Results
SLIDE 110
Results
SLIDE 111
Results
SLIDE 112
Results
SLIDE 113
Results
SLIDE 114
Results
SLIDE 115
Results
SLIDE 116
Results
SLIDE 117
Conclusions
SLIDE 118
Direct analysis
- f neural networks
SLIDE 119
"Secure Inference" maybe isn't so secure ...
SLIDE 120
Don't put neural networks in your ideal functionalities
- A talk by Matthew Jagielski
SLIDE 121
Code: https://github.com/google-research/cryptanalytic-model-extraction
Live Q&A: Friday 8:00 PT / 15:00 UTC
After-the-fact Q&A: nicholas@carlini.com
SLIDE 122
Code: https://github.com/google-research/cryptanalytic-model-extraction
Live Q&A: Friday 8:00 PT / 15:00 UTC
After-the-fact Q&A: nicholas@carlini.com