Comprehensive Privacy Analysis of Deep Learning: Passive and Active - - PowerPoint PPT Presentation

β–Ά
comprehensive privacy analysis of deep learning
SMART_READER_LITE
LIVE PREVIEW

Comprehensive Privacy Analysis of Deep Learning: Passive and Active - - PowerPoint PPT Presentation

Feb 25, 2023 276 likes β€’574 views

Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning Milad Nasr 1 , Reza Shokri 2 , Amir Houmansadr 1 1 University of Massachusetts Amherst, 2 National

slide-1
SLIDE 1

Comprehensive Privacy Analysis of Deep Learning:

Passive and Active White-box Inference Attacks against Centralized and Federated Learning

Milad Nasr1, Reza Shokri2, Amir Houmansadr1

1University of Massachusetts Amherst, 2National University of Singapore

1

slide-2
SLIDE 2

Deep learning Tasks

2

Medical Location Financial Personal History

slide-3
SLIDE 3

Privacy Threats

  • We provide a comprehensive privacy analysis of deep learning

algorithms.

  • Our objective is to measure information leakage of deep

learning models about their training data

  • In particular we emphasize on membership inference

attacks

  • Can an adversary infer whether or not a particular data

record was part of the training set?

3

slide-4
SLIDE 4

Membership Inference

4

  • Trained

Model

  • Output Vector

Member Non Member

Output Pattern

What is the cause of this behavior ?

Train Data

Data Distribution

slide-5
SLIDE 5

Training a Model

5

0.5 1 1.5 2 2.5 3 3.5 0.5 1 1.5 2 2.5 3 3.5

Train Data SGD: 𝑿 =𝑿 βˆ’πœ· π›‚β€‹πŒβ†“π±

Model parameters change in the

  • pposite direction
  • f each training

data point’s gradient

𝑿 Model parameters 𝐌 Loss π›‚β€‹πŒβ†“π± Loss gradient w.r.t parameters

slide-6
SLIDE 6

Training a Model

6

0.5 1 1.5 2 2.5 3 3.5 4 0.5 1 1.5 2 2.5 3 3.5

Train Data

slide-7
SLIDE 7

Training a Model

7

0.5 1 1.5 2 2.5 3 3.5 4 0.5 1 1.5 2 2.5 3 3.5

Train Data Non member data

Gradients leak information by behaving differently for non-member data

  • vs. member data.
slide-8
SLIDE 8

Gradients Leak Information

0.4 0.14 0.1 0.09 0.07 0.07 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 100 200 300 400 500

Gradient Norm Distribution

Members Non-Member

8

Separable distributions

slide-9
SLIDE 9

Different Learning/Attack Settings

  • Fully trained
  • Black/ White box
  • Fine-tuning
  • Federated learning
  • Central/ local Attacker
  • Passive/ Active

9

slide-10
SLIDE 10

Federated Model

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training … Collaborator 1 Collaborator 2 Collaborator 3 Collaborator n

  • Central

Model

  • 10
slide-11
SLIDE 11

Federated Learning

11

0.5 1 1.5 2 2.5 3 3.5 4 0.5 1 1.5 2 2.5 3 3.5 0.5 1 1.5 2 2.5 3 3.5 0.5 1 1.5 2 2.5 3 3.5 0.5 1 1.5 2 2.5 3 3.5 4 0.5 1 1.5 2 2.5 3 3.5

Multiple observations:

Epoch 1 Epoch 2 Epoch n

Every point leave traces on the target function

slide-12
SLIDE 12

Active Attack on Federated Learning

12

Target member Target non-member

0.5 1 1.5 2 2.5 3 3.5 4 0.5 1 1.5 2 2.5 3 3.5

Active attacker change the parameters in the direction of the gradient

slide-13
SLIDE 13

Active Attack on Federated Learning

13

For the data points that are in the training dataset, local training will compensate for the active attacker

0.5 1 1.5 2 2.5 3 3.5 4 0.5 1 1.5 2 2.5 3 3.5

slide-14
SLIDE 14

Active Attacks in Federated Model

1 2 3 4 5 6 1 2 3 4 10 20 30 40 50 60 70 80

Gradient norm Epochs

Target Members Target Non- member Member instances Non-member instances

14

slide-15
SLIDE 15

Attacker

Scenario 1: Fully Trained Model

  • Training
  • Trained

Model

  • Dataset
  • Trained

Model

  • Not Observable

Input Output vector Cat Dog

15

Outputs of all layers Loss Gradients of all layers

slide-16
SLIDE 16

Scenario 2: Central Attacker in Federated Model

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training … Collaborator 1 Collaborator 2 Collaborator 3 Collaborator n

  • Central

Model

  • 16
slide-17
SLIDE 17
  • Local

Model

Training

  • Local

Model

Training

  • Local

Model

Training

  • Local

Model

Training

  • Central

Model

  • Not observable

Not observable Not observable Not observable

Scenario 2: Central Attacker in Federated Model

…

Target individual collaborators

L

  • c

a l U p d a t e s

  • Isolated

Model

  • Isolated any

collaborators

In addition to the local attacker

  • bservations:
slide-18
SLIDE 18

Scenario 3: Local Attacker in Federated Learning

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training

Local Data

  • Local

Model

Training … Collaborator 1 Collaborator 2 Collaborator 3 Collaborator n

  • Central

Model

  • Active

18

Not Observable Outputs of all layers Loss Gradients of all layers Epoch 1: Outputs of all layers Loss Gradients of all layers Epoch 2: . . . Outputs of all layers Loss Gradients of all layers Epoch N:

slide-19
SLIDE 19

Score function

19

Outputs of all layers Loss Gradients of all layers Epoch 1: Outputs of all layers Loss Gradients of all layers Epoch 2: . . . Outputs of all layers Loss Gradients of all layers Epoch N: Different observation:

Input Score

Member ? Non-member

slide-20
SLIDE 20

Experimental Setup

  • Unlike previous works, we used publicly available pretrained models
  • We used all common regularization techniques
  • We implemented our attacks in PyTorch
  • We used following datasets:
  • CIFAR100
  • Purchase100
  • Texas100

20

slide-21
SLIDE 21

Results

21

slide-22
SLIDE 22

Pretrained Models Attacks

22

Last layer contains the most information

Gradients leak significant information

slide-23
SLIDE 23

Federated Attacks

23

Global attack is more powerful than the local attacker An active attacker can force SGD to leak more information

slide-24
SLIDE 24

Conclusions

  • We go beyond black-box scenario and try to understand why a deep

learning model leak information

  • Gradients leak information about the training dataset
  • Attacker in the federated learning can take the advantage of multiple
  • bservations to leak more information
  • In the federated setting, an attacker can actively force SGD to leak

information

24

Questions ?

slide-25
SLIDE 25

Overall Attack Model

25

Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label Layer 1 output Layer 1 gradient Layer 2 output Layer 2 gradient Layer n output Layer n gradient

. . .

Loss Label

Epoch 1: Epoch n :

Output Component Gradient Component Output Component Gradient Component Output Component Gradient Component

. . .

Loss Component Label Component

Member ? Non-member

slide-26
SLIDE 26

Scenario 4: Fine-Tuning Model

Dataset

  • Training
  • General

Model

  • Specialized

Dataset

  • Training
  • Fine-Tuned

Model

  • Not Observable

Attacker

  • General

Model

  • Fine-Tuned

Model

  • Outputs of all layers

Loss Gradients of all layers Outputs of all layers Loss Gradients of all layers

General dataset Specialized dataset None

26

slide-27
SLIDE 27

Fine-tuning Attacks

27

Both specialized and general datasets are vulnerable to the membership attacks

Dataset Arch Distinguishing specialized/general datasets Distinguishing general / non-member datasets Distinguishing Specialized / non- member datasets

slide-28
SLIDE 28

Federated Attacks

28

slide-29
SLIDE 29

Fine-Tuning Model Leakage

Dataset

  • Training
  • General

Model

  • Specialized

Dataset

  • Training
  • Fine-Tuned

Model

  • Not Observable

Attacker

  • General

Model

  • Fine-Tuned

Model

  • Outputs of all layers

Loss Gradients of all layers Outputs of all layers Loss Gradients of all layers

General dataset Specialized dataset None

29