Type-checking Linear Dependent Types Arthur Azevedo de Amorim 1 2 - - PowerPoint PPT Presentation

Type-checking Linear Dependent Types

Arthur Azevedo de Amorim 1 2 Marco Gaboardi 3 Emilio Jesús Gallego Arias 1 Justin Hsu 1

1University of Pennsylvania 2INRIA Paris-Rocquencourt 3University of Dundee 1

2

2

2

Movie Ratings Internet Anonymization

2

2

2

How to allow database queries and retain privacy guarantees?

3

Differential Privacy

• Rigorous bound on “privacy loss” [Dwork, 2006]
• Informally: adding one’s data doesn’t change query results

by much

• Many available algorithms
• Statistical analyses, combinatorial optimizations, machine

learning, ...

4

Ensuring Differential Privacy

Deterministic Query

+

Random Noise Differentially Private Probabilistic Query

5

Ensuring Differential Privacy

k-sensitive Deterministic Query

+

Random Noise ∝ k Differentially Private Probabilistic Query

5

Ensuring Differential Privacy

k-sensitive Deterministic Query

+

Random Noise ∝ k Differentially Private Probabilistic Query Bound on result variation

5

Ensuring Differential Privacy

k-sensitive Deterministic Query

+

Random Noise ∝ k Differentially Private Probabilistic Query Bound on result variation Requires tedious proofs

5

Types to the Rescue

• DFuzz [Reed&Pierce10,Gaboardi13] is a type system for

function sensitivity (hence, differential privacy)

• Capable of expressing many differentially private algorithms
• Metatheory ensures differential privacy
• Type-checking algorithm: proof automation

6

Challenge: Checking and Inference

The DFuzz type system combines interesting features:

• Linear indexed types
• Dependent types
• Subtyping

Their interplay makes it difficult to reuse existing techniques directly

7

Our Contributions

• A type-checking and type-inference algorithm for a system

combining linear and dependent types in the presence of subtyping

• Showing how ideas from the type-checking literature for

those domains can be adapted to a type system built around a special-purpose index language

8

Outline

• DFuzz and function sensitivity
• Type checking and inference for DFuzz

9

DFuzz and Function Sensitivity

10

Function Sensitivity

Bound output variation based on input variation f is k-sensitive: d(f(x), f(y)) ≤ k · d(x, y)

11

Function Sensitivity

Bound output variation based on input variation f is k-sensitive: d(f(x), f(y)) ≤ k · d(x, y) Distance functions

11

Function Sensitivity

x y D f(x) f(y) f f ≤ k · D

11

DFuzz in a Nutshell

• !kσ ⊸ τ: k-sensitive function (= linear)

12

DFuzz in a Nutshell

• !kσ ⊸ τ: k-sensitive function (= linear)

Multivariate polynomial

12

DFuzz in a Nutshell

• !kσ ⊸ τ: k-sensitive function (= linear)
• listn σ: list of length n (mechanisms that depend on input

size)

12

DFuzz in a Nutshell

• !kσ ⊸ τ: k-sensitive function (= linear)
• listn σ: list of length n (mechanisms that depend on input

size)

• σ ⊑ τ: sensitivity weakening (e.g. (!1σ ⊸ τ) ⊑ (!2σ ⊸ τ))

12

A Basic Example

Consider the standard map function function map f l { case l of { | [] => [] | x :: l’ => f x :: map f l’ } } How to bound “distance” between results of two calls?

13

Analyzing Example

d(map(f, x), map(g, y))

14

Analyzing Example

d(map(f, x), map(g, y)) k-sensitive

14

Analyzing Example

d(map(f, x), map(g, y)) length n

14

Analyzing Example

d(map(f, x), map(g, y)) =

n

• i=1

d(f(xi), g(yi)) Distance for lists

14

Analyzing Example

d(map(f, x), map(g, y)) =

n

• i=1

d(f(xi), g(yi)) ≤

n

• i=1

[d(f(xi), g(xi)) + d(g(xi), g(yi))] Triangle inequality

14

Analyzing Example

d(map(f, x), map(g, y)) =

n

• i=1

d(f(xi), g(yi)) ≤

n

• i=1

[d(f(xi), g(xi)) + d(g(xi), g(yi))] ≤

n

• i=1

[d(f, g) + k · d(xi, yi)] Max difference between f and g

14

Analyzing Example

d(map(f, x), map(g, y)) =

n

• i=1

d(f(xi), g(yi)) ≤

n

• i=1

[d(f(xi), g(xi)) + d(g(xi), g(yi))] ≤

n

• i=1

[d(f, g) + k · d(xi, yi)] Definition of sensitivity

14

Analyzing Example

d(map(f, x), map(g, y)) =

n

• i=1

d(f(xi), g(yi)) ≤

n

• i=1

[d(f(xi), g(xi)) + d(g(xi), g(yi))] ≤

n

• i=1

[d(f, g) + k · d(xi, yi)] = n · d(f, g) + k · d( x, y)

14

Typing Example in DFuzz

map : !n(!kσ ⊸ τ) ⊸ !klistn σ ⊸ listn τ

15

Typing Example in DFuzz

map : !n(!kσ ⊸ τ) ⊸ !klistn σ ⊸ listn τ Argument sensitivities

15

Some Rules

Γ, x :k σ ⊢ e : τ Γ ⊢ λx :k σ.e : !kσ ⊸ τ (⊸ I)

16

Some Rules

Γ, x :k σ ⊢ e : τ Γ ⊢ λx :k σ.e : !kσ ⊸ τ (⊸ I) Keep track of sensitivity

16

Some Rules

Γ, x :k σ ⊢ e : τ Γ ⊢ λx :k σ.e : !kσ ⊸ τ (⊸ I) Propagate sensitivity to type

16

Some Rules

Γ ⊢ e1 :!kσ ⊸ τ ∆ ⊢ e2 : σ Γ + k · ∆ ⊢ e1 e2 : τ (⊸ E) Context split, combine sensitivities

16

Some Rules

Γ ⊢ e1 :!kσ ⊸ τ ∆ ⊢ e2 : σ Γ + k · ∆ ⊢ e1 e2 : τ (⊸ E) Composition: multiply sensitivities

16

Some Rules

∆ ⊢ e : listn σ Γ ⊢ enil : τ Γ, h :k σ, t :k listi σ ⊢ econs : τ Γ + k · ∆ ⊢ case e of [] → enil | h :: t → econs : τ (list E)

16

Some Rules

∆ ⊢ e : listn σ Γ ⊢ enil : τ Γ, h :k σ, t :k listi σ ⊢ econs : τ Γ + k · ∆ ⊢ case e of [] → enil | h :: t → econs : τ (list E) Assuming n = 0

16

Some Rules

∆ ⊢ e : listn σ Γ ⊢ enil : τ Γ, h :k σ, t :k listi σ ⊢ econs : τ Γ + k · ∆ ⊢ case e of [] → enil | h :: t → econs : τ (list E) Assuming n = i + 1

16

Some Rules

∆ ⊢ e : listn σ Γ ⊢ enil : τ Γ, h :k σ, t :k listi σ ⊢ econs : τ Γ + k · ∆ ⊢ case e of [] → enil | h :: t → econs : τ (list E) Track sensitivity on list

16

Type Checking and Inference

17

Plan

DFuzz Program

18

Plan

DFuzz Program Inference

18

Plan

DFuzz Program Inference σ Constraints

18

Plan

DFuzz Program Inference σ Constraints Polynomial inequalities from subtyping

18

Plan

DFuzz Program Inference σ Constraints Solver

18

Plan

DFuzz Program Inference σ Constraints Solver Yes

18

Plan

DFuzz Program Inference σ Constraints Solver ⊑ τ?

18

Plan

DFuzz Program Inference σ Constraints Solver ⊑ τ? Provided by annotation

18

Plan

DFuzz Program Inference σ Constraints Solver Yes ⊑ τ?

18

Important Points

• Context splitting imposes a bottom-up strategy: start with

leaves, combine sensitivities progressively

19

Important Points

• Context splitting imposes a bottom-up strategy: start with

leaves, combine sensitivities progressively . . . e1 . . . e2 . . .

19

Important Points

• Context splitting imposes a bottom-up strategy: start with

leaves, combine sensitivities progressively . . . e1 . . . e2 . . . e1 : σ1 e2 : σ2

19

Important Points

• Context splitting imposes a bottom-up strategy: start with

leaves, combine sensitivities progressively . . . e1 . . . e2 . . . : τ e1 : σ1 e2 : σ2

19

Important Points

• Context splitting imposes a bottom-up strategy: start with

leaves, combine sensitivities progressively

• Restrict subtyping to essential places (e.g. application)

19

Important Points

• Context splitting imposes a bottom-up strategy: start with

leaves, combine sensitivities progressively

• Restrict subtyping to essential places (e.g. application)
• Assume sensitivities on higher-order types are given

19

Important Points

• Context splitting imposes a bottom-up strategy: start with

leaves, combine sensitivities progressively

• Restrict subtyping to essential places (e.g. application)
• Assume sensitivities on higher-order types are given

E.g !_(!kα ⊸ α) ⊸ α

19

Problems

• Language not rich enough to express minimal sensitivities
• E.g. point-wise maximum of two polynomials is not a

polynomial

• Solution: enrich sensitivity language with new operators

20

Problems

• Language not rich enough to express minimal sensitivities
• E.g. point-wise maximum of two polynomials is not a

polynomial

• Solution: enrich sensitivity language with new operators
• cf. literature on subtyping

20

Problems

• Language not rich enough to express minimal sensitivities
• E.g. point-wise maximum of two polynomials is not a

polynomial

• Solution: enrich sensitivity language with new operators
• Type checking is undecidable
• Can encode equality of integer polynomials (Hilbert’s tenth

problem)

• Completeness relative to a decider of sensitivity inequalities

20

Syntax-Directed Rules

Equivalent to previous ones, but directly translatable to algorithm Input Term, argument type annotations Output Minimal sensitivities, minimal type

21

Syntax-Directed Rules

Γ ⊢ e1 :!kσ ⊸ τ ∆ ⊢ e2 : σ′ σ′ ⊑ σ Γ + k · ∆ ⊢ e1 e2 : τ (⊸ E)

21

Syntax-Directed Rules

Γ ⊢ e1 :!kσ ⊸ τ ∆ ⊢ e2 : σ′ σ′ ⊑ σ Γ + k · ∆ ⊢ e1 e2 : τ (⊸ E) Not necessarily equal

21

Syntax-Directed Rules

Γ ⊢ e1 :!kσ ⊸ τ ∆ ⊢ e2 : σ′ σ′ ⊑ σ Γ + k · ∆ ⊢ e1 e2 : τ (⊸ E) Subtype check

21

Syntax-Directed Rules

∆ ⊢ e : listn σ Γ ⊢ enil : τnil Γ, h :k σ, t :k listi σ ⊢ econs : τcons τ = case(n, τnil, i, τcons) Γ + k · ∆ ⊢ case e of [] → enil | h :: t → econs : τ (list E)

21

Syntax-Directed Rules

∆ ⊢ e : listn σ Γ ⊢ enil : τnil Γ, h :k σ, t :k listi σ ⊢ econs : τcons τ = case(n, τnil, i, τcons) Γ + k · ∆ ⊢ case e of [] → enil | h :: t → econs : τ (list E) Sensitivity-level case lifted to types

21

Solver Integration

• Need to convert constraints so that standard solvers

understand them

• Avoid alternating quantifiers

22

Solver Integration

k ≥ case(n, k0, i, ks) (n = 0 ⇒ k ≥ k0) ∧ (∀i, n = i + 1 ⇒ k ≥ ks)

22

Wrapping Up

23

Conclusion

• Type-checking system with linear and dependent types
• Standard ideas adapted to exploit application domain and

index structure

• Recover minimal sensitivities by extending index language
• As simple as possible, no need for much expressive power (cf

[DalLago&Petit13])

24

Implementation

Available at http://cis.upenn.edu/~emilioga/dFuzz.tar.gz Capable of checking most of the original DFuzz examples

25

Future Directions

• Let-generalization for sensitivities (remove higher-order

annotations)

• Identify decidable fragment of DFuzz

26

Questions?

27

Some Metric Spaces

dR(x, y) = |x − y| dσ→τ(f, g) = sup

x∈σ dτ(f(x), g(x))

dlist σ(l1, l2) =

• ∞

if length(l1) = length(l2)

• i dσ(l1[i], l2[i])
• therwise

dset σ(s1, s2) = |s1 \ s2 ∪ s2 \ s1| dP(σ)(µ, ν) =

• σ

log dµ dν

• dµ

28

More Typing Rules

Γ, x :1 σ ⊢ x : σ (Var)

29

More Typing Rules

Γ, x :∞ σ ⊢ e : σ ∞ · Γ ⊢ fix x : σ.e : σ (Fix)

29

More Typing Rules

∆ ⊑ Γ Γ ⊢ e : σ σ ⊑ τ ∆ ⊢ e : τ (⊑)

29

Metric Preservation

Suppose ⊢ e :!kσ ⊸ τ ⊢ v1 : σ ⊢ v2 : σ e v1 →∗ v′

1

There exists v′

2 such that e v2 →∗ v′ 2 and

dτ(v′

1, v′ 2) ≤ k · dσ(v1, v2)

30