Privacy in Machine Learning Fatemehsadat Mireshghallah WiMLDS - - PowerPoint PPT Presentation

Privacy in Machine Learning

Fatemehsadat Mireshghallah WiMLDS NeurIPS19 Meet-up

Why is privacy a concern in ML?

Patient History Genetic Data Search History

Famous incidents - Anonymization

• “A Face Is Exposed for AOL Searcher No.

4417749” [Barbaro & Zeller ’06]

• “Robust De-anonymization of Large

Datasets (How to Break Anonymity of the Netflix 
 Prize Dataset)”[Narayanan & Shmatikov ’08]

• “Matching Known Patients to Health

Records in Washington State Data” [Sweeney ’13] 
 


Machine Learning Models that Remember Too Much [Song’17]

Membership Inference Attacks Against Machine Learning Models [Shokri’17] Practical Black-Box Attacks against Machine Learning [Papernot’17] Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures [Fredrikson’15]

Solutions for Private Aggregation / Training

• Differential Privacy [Dwork’06] over 5000 papers on this
• Privacy-Preserving Deep Learning [Shokri’15]
• Federated Learning [Konecny’15 & ’16] over 300 papers on this
• Deep Learning with Differential Privacy [Abadi’16]

https://1.bp.blogspot.com/-K65Ed68KGXk/WOa9jaRWC6I/AAAAAAAABsM/gglycD_anuQSp-i67fxER1FOlVTulvV2gCLcB/s640/ FederatedLearning_FinalFiles_Flow%2BChart1.png

Inference Problem

Inverting Visual Representations with Convolutional Networks [Dosovitskiy’16]

Solutions

• MiniONN[Liu’17]
• Arden[Wang’18]
• Deep Private Feature Extraction[Osia’18]
• Shredder [Mireshghallah’19]

Privacy Accuracy Loss

C

• m

p u t a t i

• n

a l C

• s

t

~ ~

Shredder

Undesirable Region Accuracy-Agnostic Noise Addition Homomorphic Encryption

CryptoNets[19] Minion[21]

Shredder

… … …

✕

Edge Cloud

L(x, θ1) R(a0, θ2)

Activation

Noisy Activation

y = f 0(x, θ, n)

input x a a0

Noise Tensor

+

n2

Noise Tensor

n1

Shredder

• Non-intrusive
• 66.7% loss in information
• 97.3% misclassifaction of private labels
• 1.7% loss in classification of publica labels

fmireshg@ucsd.edu http://cseweb.ucsd.edu/~fmireshg/ Dec 14th (saturday) - Privacy in Machine Learning Workshop (East Meeting Rooms 8+15) -11:30 AM

No Nois ise dis e distribut ibutio ion T n Training phas aining phase

Network Partitioner Intermediate Activation Adder Noisy Activation Network Output (logits) Noise Tensor Training Data Transformer with Edge Partition

5 8 8 2 3 4 1 7 3 5 8 8 2 3 4 1 7 3

Loss Input Generator with Cloud Partition Batch of Data From the Training Dataset Loss Function and Optimizer Calculated Gradients Noise Tensor Update Noise Tensor Initializer DNN Topology and Pretrained Weights Accuracy > Desired

• f Noise Elements Collector

Noise Tensor to Laplace Distribution Fitter Yes Yes No No

Computation and Communication Costs Desired Accuracy 1 2 3 4 5 6 7 8 9 Distribution Parameters and Order of Noise Elements Collector Distance of the Distribution and Noise Tensor < Desired

In Infer eren ence p ce phas ase

Intermediate Activation Adder Noisy Activation (to be sent) Transmission Noisy Activation (received) Classification Result Sampler Collection of Distributions and Orders Noise Tensor (Mist)

Edge

Edge Partition Cloud Partition

5 8 8 2 3 4 1 7 3 5 8 8 2 3 4 1 7 3

1 2 3 4 5

Cloud

5

Cross Entropy Loss Learn and remove private labels using only the given public labels, through self-supervision.

Shredder reduces the mutual information between the input and the communicated data and removes sensitive information through self- supervision. Shredder stabilizes privacy at a high level, while increasing accuracy of the primary task and decreasing the accuracy of the private task.

Shredder can perform well with any cutting point. Using an edge GPU, shredder can offer speed-ups.

Shredder’s privacy/accuracy trade-off on public and private labels (Primary and private task).

Comparison with Deep Private Feature Extraction (DPFE) method, which needs provide labels and modifies network weights, unlike Shredder.