Advanced Probabilistic Couplings for Differential Privacy Gilles - - PowerPoint PPT Presentation

▶
advanced probabilistic couplings for differential privacy
SMART_READER_LITE
LIVE PREVIEW

Advanced Probabilistic Couplings for Differential Privacy Gilles - - PowerPoint PPT Presentation

Nov 01, 2023 421 likes •1.03k views

Advanced Probabilistic Couplings for Differential Privacy Gilles Barthe, Nomie Fong, Marco Gaboardi, Benjamin Grgoire, Justin Hsu, Pierre-Yves Strub October 25, 2016 1 A new approach to formulating privacy goals: the risk to ones

slide-1
SLIDE 1

Advanced Probabilistic Couplings for Differential Privacy

Gilles Barthe, Noémie Fong, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, Pierre-Yves Strub October 25, 2016

1

slide-2
SLIDE 2

A new approach to formulating privacy goals: the risk to one’s privacy, or in general, any type of risk . . . should not substantially increase as a result of participating in a statistical database. This is captured by differential privacy.

— Cynthia Dwork

2

slide-3
SLIDE 3

Increasing interest

In research. . .

3

slide-4
SLIDE 4

Increasing interest

In research. . . . . . and beyond

3

slide-5
SLIDE 5

4

slide-6
SLIDE 6

Dwork, McSherry, Nissim, and Smith

Let ǫ, δ ≥ 0 be parameters, and suppose there is a binary adjacency relation Adj on D. A randomized algorithm M : D → Distr(R) is (ǫ, δ)-differentially private if for every set of outputs S ⊆ R and every pair of adjacent inputs d1, d2, we have

Prx∼M(d1)[x ∈ S] ≤ exp(ǫ) · Prx∼M(d2)[x ∈ S] + δ.

5

slide-7
SLIDE 7

Dwork, McSherry, Nissim, and Smith

Let ǫ, δ ≥ 0 be parameters, and suppose there is a binary adjacency relation Adj on D. A randomized algorithm M : D → Distr(R) is (ǫ, δ)-differentially private if for every set of outputs S ⊆ R and every pair of adjacent inputs d1, d2, we have

Prx∼M(d1)[x ∈ S] ≤ exp(ǫ) · Prx∼M(d2)[x ∈ S] + δ.

How to formally verify?

5

slide-8
SLIDE 8

Differential privacy is a: relational property of probabilistic programs.

6

slide-9
SLIDE 9

Composition properties Program is (ǫ + ǫ′, δ + δ′)-private

7

slide-10
SLIDE 10

Composition properties Program is (ǫ + ǫ′, δ + δ′)-private

Formally

Consider randomized algorithms M : D → Distr(R) and M : R → D → Distr(R′). If M is (ǫ, δ)-private and for every r ∈ R, M′(r) is (ǫ′, δ′)-private, then the composition is (ǫ + ǫ′, δ + δ′)-private:

r

$

← M(d); res

$

← M(r, d); return(res)

7

slide-11
SLIDE 11

When privacy follows from composition

8

slide-12
SLIDE 12

When privacy follows from composition

(Linear types, refinement types, self products, relational Hoare logics, . . . )

8

slide-13
SLIDE 13

When privacy doesn’t follow from composition

9

slide-14
SLIDE 14

Complicated privacy proofs

— Lyu, Su, Dong

10

slide-15
SLIDE 15

Complicated privacy proofs

— Lyu, Su, Dong

How to verify these proofs?

10

slide-16
SLIDE 16

Recent progress (2016) Differential privacy ≈ Approximate couplings

11

slide-17
SLIDE 17

Recent progress (2016) Differential privacy ≈ Approximate couplings Approximate couplings ≈ Proofs in the logic apRHL

11

slide-18
SLIDE 18

Recent progress (2016) Differential privacy ≈ Approximate couplings Approximate couplings ≈ Proofs in the logic apRHL

Only proofs beyond composition for (ǫ, 0)-privacy

11

slide-19
SLIDE 19

Enhance the logic

New coupling constructions

⇓

New proof rules

⇓

Richer formal proofs of privacy

12

slide-20
SLIDE 20

Our work: formal privacy proofs with:

Accuracy-dependent privacy Advanced composition Adaptive inputs

13

slide-21
SLIDE 21

Our work: formal privacy proofs with:

Accuracy-dependent privacy Advanced composition Adaptive inputs

13

slide-22
SLIDE 22

A crash course: the program logic apRHL [BKOZB]

Imperative language with random sampling

x

$

← Lǫ(e)

14

slide-23
SLIDE 23

A crash course: the program logic apRHL [BKOZB]

Imperative language with random sampling

x

$

← Lǫ(e)

approximate probabilistic Relational Hoare Logic

⊢ {Φ} c1 ∼(ǫ,δ) c2 {Ψ}

14

slide-24
SLIDE 24

A crash course: the program logic apRHL [BKOZB]

Imperative language with random sampling

x

$

← Lǫ(e)

approximate probabilistic Relational Hoare Logic

⊢ {Φ} c1 ∼(ǫ,δ) c2 {Ψ}

Non-probablistic, relational (x1 = x2)

14

slide-25
SLIDE 25

A crash course: the program logic apRHL [BKOZB]

Imperative language with random sampling

x

$

← Lǫ(e)

approximate probabilistic Relational Hoare Logic

⊢ {Φ} c1 ∼(ǫ,δ) c2 {Ψ}

Numeric index

14

slide-26
SLIDE 26

Approximate couplings [BKOZB, BO]

Definition

Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ1, µ2 ∈ Distr(A) are related by an (ǫ, δ)-approximate coupling with support R if there exists µL, µR ∈ Distr(A × A) with:

15

slide-27
SLIDE 27

Approximate couplings [BKOZB, BO]

Definition

Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ1, µ2 ∈ Distr(A) are related by an (ǫ, δ)-approximate coupling with support R if there exists µL, µR ∈ Distr(A × A) with:

◮ support in R ; 15

slide-28
SLIDE 28

Approximate couplings [BKOZB, BO]

Definition

Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ1, µ2 ∈ Distr(A) are related by an (ǫ, δ)-approximate coupling with support R if there exists µL, µR ∈ Distr(A × A) with:

◮ support in R ; ◮ π1(µL) = µ1 and π2(µR) = µ2 ; 15

slide-29
SLIDE 29

Approximate couplings [BKOZB, BO]

Definition

Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ1, µ2 ∈ Distr(A) are related by an (ǫ, δ)-approximate coupling with support R if there exists µL, µR ∈ Distr(A × A) with:

◮ support in R ; ◮ π1(µL) = µ1 and π2(µR) = µ2 ; ◮ for every S ⊆ A × A,

Prz∼µL[z ∈ S] ≤ exp(ǫ) · Prz∼µR[z ∈ S] + δ

15

slide-30
SLIDE 30

Approximate couplings [BKOZB, BO]

Definition

Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ1, µ2 ∈ Distr(A) are related by an (ǫ, δ)-approximate coupling with support R if there exists µL, µR ∈ Distr(A × A) with:

◮ support in R ; ◮ π1(µL) = µ1 and π2(µR) = µ2 ; ◮ for every S ⊆ A × A,

Prz∼µL[z ∈ S] ≤ exp(ǫ) · Prz∼µR[z ∈ S] + δ

15

slide-31
SLIDE 31

Approximate couplings [BKOZB, BO]

Definition

Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ1, µ2 ∈ Distr(A) are related by an (ǫ, δ)-approximate coupling with support R if there exists µL, µR ∈ Distr(A × A) with:

◮ support in R ; ◮ π1(µL) = µ1 and π2(µR) = µ2 ; ◮ for every S ⊆ A × A,

Prz∼µL[z ∈ S] ≤ exp(ǫ) · Prz∼µR[z ∈ S] + δ

Write: µ1 R♯

(ǫ,δ)

µ2

15

slide-32
SLIDE 32

Interpreting judgments

⊢ {Φ} c1 ∼(ǫ,δ) c2 {Ψ}

16

slide-33
SLIDE 33

Interpreting judgments

⊢ {Φ} c1 ∼(ǫ,δ) c2 {Ψ}

Two memories related by Φ

16

slide-34
SLIDE 34

Interpreting judgments

⊢ {Φ} c1 ∼(ǫ,δ) c2 {Ψ}

Two memories related by Φ ⇓

Two distributions related by Ψ♯

(ǫ,δ)

16

slide-35
SLIDE 35

Differential privacy in apRHL

⊢ {Adj(d1, d2)} c ∼(ǫ,δ) c {res1 = res2}

17

slide-36
SLIDE 36

Differential privacy in apRHL

⊢ {Adj(d1, d2)} c ∼(ǫ,δ) c {res1 = res2}

(ǫ, δ)-differential privacy

17

slide-37
SLIDE 37

Proof rules Proof rule ≈ Recipe to combine couplings

18

slide-38
SLIDE 38

Proof rules Proof rule ≈ Recipe to combine couplings

Sequence rule ≈ standard composition of privacy

Seq ⊢ {Φ} c1 ∼(ǫ,δ) c2 {Ψ}

⊢ {Ψ} c′

1 ∼(ǫ′,δ′) c′ 2 {Θ}

⊢ {Φ} c1; c′

1 ∼(ǫ+ǫ′,δ+δ′) c2; c′ 2 {Θ} 18

slide-39
SLIDE 39

Proof rules Proof rule ≈ Recipe to combine couplings

Sequence rule ≈ standard composition of privacy

Seq ⊢ {Φ} c1 ∼(ǫ,δ) c2 {Ψ}

⊢ {Ψ} c′

1 ∼(ǫ′,δ′) c′ 2 {Θ}

⊢ {Φ} c1; c′

1 ∼(ǫ+ǫ′,δ+δ′) c2; c′ 2 {Θ} 18

slide-40
SLIDE 40

Our work: formal privacy proofs with:

Accuracy-dependent privacy Advanced composition Adaptive inputs

19

slide-41
SLIDE 41

Accuracy-dependent privacy

20

slide-42
SLIDE 42

Accuracy-dependent privacy

Rough intuition

◮ Think of δ in (ǫ, δ)-privacy as failure probability ◮ “Algorithm is private except with small probability δ” ◮ “If the noise added is not too large, then . . . ”

Similar to up-to-bad reasoning

◮ Common tool in crypto proofs ◮ “If bad event doesn’t happen, then protocol is safe” 21

slide-43
SLIDE 43

In apRHL: up-to-bad rule

UtB

⊢ {Φ} c1 ∼(ǫ,δ) c2 {¬Ψ1 → x1 = x2} | = m ∈ Θ = ⇒ Pr

[ [c1] ](m1)[Ψ1] < δ′

⊢ {Φ} c1 ∼(ǫ,δ+δ′) c2 {x1 = x2}

22

slide-44
SLIDE 44

In apRHL: up-to-bad rule

UtB

⊢ {Φ} c1 ∼(ǫ,δ) c2 {¬Ψ1 → x1 = x2} | = m ∈ Θ = ⇒ Pr

[ [c1] ](m1)[Ψ1] < δ′

⊢ {Φ} c1 ∼(ǫ,δ+δ′) c2 {x1 = x2}

Notes

◮ Ψ1 is “bad event”, only mentions c1 22

slide-45
SLIDE 45

In apRHL: up-to-bad rule

UtB

⊢ {Φ} c1 ∼(ǫ,δ) c2 {¬Ψ1 → x1 = x2} | = m ∈ Θ = ⇒ Pr

[ [c1] ](m1)[Ψ1] < δ′

⊢ {Φ} c1 ∼(ǫ,δ+δ′) c2 {x1 = x2}

Notes

◮ Ψ1 is “bad event”, only mentions c1 ◮ If bad event doesn’t happen, have privacy 22

slide-46
SLIDE 46

In apRHL: up-to-bad rule

UtB

⊢ {Φ} c1 ∼(ǫ,δ) c2 {¬Ψ1 → x1 = x2} | = m ∈ Θ = ⇒ Pr

[ [c1] ](m1)[Ψ1] < δ′

⊢ {Φ} c1 ∼(ǫ,δ+δ′) c2 {x1 = x2}

Notes

◮ Ψ1 is “bad event”, only mentions c1 ◮ If bad event doesn’t happen, have privacy ◮ Bound probability of Ψ after c1 22

slide-47
SLIDE 47

23

slide-48
SLIDE 48

Advanced composition theorem

Compose n mechanisms, each (ǫ, δ)-private

◮ Standard composition: (n · ǫ, n · δ)-private ◮ Advanced composition: (ǫ∗, δ∗)-private

ǫ∗ ≈ √n · ǫ and δ∗ ≈ n · δ + δ′

24

slide-49
SLIDE 49

Advanced composition theorem

Compose n mechanisms, each (ǫ, δ)-private

◮ Standard composition: (n · ǫ, n · δ)-private ◮ Advanced composition: (ǫ∗, δ∗)-private

ǫ∗ ≈ √n · ǫ and δ∗ ≈ n · δ + δ′

Trade off ǫ and δ

24

slide-50
SLIDE 50

Advanced composition theorem

Compose n mechanisms, each (ǫ, δ)-private

◮ Standard composition: (n · ǫ, n · δ)-private ◮ Advanced composition: (ǫ∗, δ∗)-private

ǫ∗ ≈ √n · ǫ and δ∗ ≈ n · δ + δ′

Trade off ǫ and δ

24

slide-51
SLIDE 51

Advanced composition theorem

Compose n mechanisms, each (ǫ, δ)-private

◮ Standard composition: (n · ǫ, n · δ)-private ◮ Advanced composition: (ǫ∗, δ∗)-private

ǫ∗ ≈ √n · ǫ and δ∗ ≈ n · δ + δ′

Trade off ǫ and δ

24

slide-52
SLIDE 52

In apRHL: new while rule

AC

| = Θ → e1 = e2 ⊢ {Θ ∧ e1} c1 ∼(ǫ,δ) c2 {Θ} while e1 do c1 exceutes at most n iterations ⊢ {Θ} while e1 do c1 ∼(ǫ∗,δ∗) while e2 do c2 {Θ ∧ ¬e1}

Notes

◮ Surprising: generalization to approximate couplings ◮ More surprising: privacy composition directly generalizes 25

slide-53
SLIDE 53

Putting it all together

26

slide-54
SLIDE 54

A brief preview: the Between Thresholds algorithm

Variant of a mechanism by Bun, Steinke, Ullman (2016)

Formalized (ǫ, δ)-privacy in EasyCrypt

27

slide-55
SLIDE 55

Formal proof combines many different features:

◮ Accuracy-dependent privacy ◮ Advanced composition ◮ Adaptively chosen inputs ◮ “Subset” coupling 28

slide-56
SLIDE 56

Formal proof combines many different features:

◮ Accuracy-dependent privacy ◮ Advanced composition ◮ Adaptively chosen inputs ◮ “Subset” coupling 28

slide-57
SLIDE 57

Formal proof combines many different features:

◮ Accuracy-dependent privacy ◮ Advanced composition ◮ Adaptively chosen inputs ◮ “Subset” coupling

Please see the paper!

28

slide-58
SLIDE 58

Our work: formal privacy proofs with:

Accuracy-dependent privacy Advanced composition Adaptive inputs

29

slide-59
SLIDE 59

Our work: formal privacy proofs with:

Accuracy-dependent privacy Advanced composition Adaptive inputs

29