CSC2412: The Projection Mechanism Sasho Nikolov 1 Motivation - - PowerPoint PPT Presentation

CSC2412: The Projection Mechanism

Sasho Nikolov

1

Motivation

Reminder: Query Release

Recall the query release problem:

• Workload Q = {q1, . . . , qk} of k counting queries

Q(X) = B @ q1(X) . . . qk(X) 1 C A 2 [0, 1]k.

• Compute, with (", )-DP, some Y 2 Rk so that

k

max

i=1 |Yi qi(X)|  ↵,

with probability 1 .

2

qi :D → do

I}

gilt ) - tu j

Yik;)

Private MW: running time

Recall: PMW answers `-way marginals on X = {0, 1}d with error ↵ when n `d log(d) ↵3" n p `d log(d) log(1/) ↵2" What about running time?

3

an

l- way marginal query

( over 2--40, Bd )

is

specified by e

• f the

d

attributes

and

values

for

them

and

asks

whether

a data point

has

the

prescribed values Both

bounds

are

at

f

t

most

linear

E

• DP

( E , f)

• DP

in ol

Polynomial

in

u

t

exponential

in

• l

↳ bounded

mistake

learner

maintain

201

values

Gaussian noise: running time

Recall: The Gaussian noise mechanism answers `-way marginals on X = {0, 1}d with error ↵ when n d`/2p ` log(d) log(1/) ↵" What about running time?

4

It

Much

more

than with

PM w

Olden l )

Open problem

Find an (", )-DP mechanism running in time polynomial in d` and n which answers `-way marginals with error ↵ when n p `d log(d) log(1/) ↵2"

5

penumbra queries

↳ what

you

need

for PNW

• I. e

can

we

get

the

running

time

• f

Gaussian

noise week

but the

error

guarantees

• f

PM w

e

• t

:

123

? ?

÷

• a. well

as raw Il

Effingham

"'m

I

The Projection Mechanism

Feasible query answers

What can actual query answers look like? I.e., what is the set {Q(X) : X 2 X n}? When n = 1: SQ = {Q({x}) : x 2 X} ✓ {0, 1}k General n:

6

Yet of all possible

query

answers

→e will think of

this

set geometrically

• →

all possible vectors of

answers

• n

a single

data point For any

qi

C-Q

,

gift )

• I¥

, qicxjl

⇒

QCX )

= In IZ

Q (f x

;})

→ an

average

• f

some

collection

• f

points

in Sq

Example

X = {0, 1}2, Q = {q1, q2}. q1(x) = 1{x1 = 1}, q2(x) = 1{x1 = 0 OR x2 = 1}.

7

+sedition :B

sa -71%161,41 's

*Yolk:B

So

,

it

is:÷÷!:*

"it

!.EE#!!sete:fiae

condition

is true

* stoning

tix,%:

a

:*.ae

.

ah

• fist 's

Hitomi's)

Feasible polytope

KQ = conv(SQ) = {1y1 + . . . + mym : y1, . . . , ym 2 SQ, 1, . . . , m 0, 1 + . . . + m = 1}. Key observation: for any dataset X, Q(X) 2 KQ.

8

quiver

hull

:qo

°

Qlt )

= th

( Itis)

t toys Htt

+

and'm)

e tha

tf i

Qcsxi})

c- So

,

Post-processing?

Projection mechanism:

• Run the Gaussian noise mechanism to get ˜

Y = Q(X) + Z, Z ⇠ N(0,

k ⇢n2 I).

• Project ˜

Y onto KQ: ˆ Y = arg min{ky ˜ Y k2 : y 2 KQ}. Output ˆ Y .

9

• ftp.y-QLH-Z

Ka

• QH)
• ↳

ee, di

• DP

go , 5-

if

bogus)

So

, output

the

closest

feasible vector of

answers

to

the

• utput of the

Gauss

meeh

Privacy analysis

10

B

is

post

processing

• f

the output

g

• f

the Gaussian

noise

mechanism

Privacy

• f

5

t

composition

theorem

mean

that

I

is

CE

• Dp

Accuracy analysis?

We can show that projection decreases the error on average. The average error of a mechanism X is max

X∈X n

v u u tE1 k

k

X

i=1

(M(X)i qi(X))2 = max

X∈X n

r E1 k kM(X) Q(X)k2

2.

The projection mechanism has average error at most ↵ as long as n p log |X| · q log 1

• ↵2 · "

11

→ Root

mean squared error

• ↳ we respect to

randomness of µ

Usually avg error guarantees

can

be turned into

worst

• case error guarantees

using private

boosting

( related

to

Mw )

• For

l- wise marginals

197¥39

• n. rr

d'

e

d

'

E

Mean width

Support function: hKQ(z) = max{hy, zi : y 2 KQ} = max{hy, zi : y 2 SQ} Mean width: `∗(KQ) = E hKQ(G), where G ⇠ N(0, I). Geometric lemma: The Projection Mechanism has average error ↵ if n

`∗(KQ)· q log 1

δ

↵2·"· √ k

.

12

÷÷÷÷ii÷÷

Hk ) e El

Running time

Running time bottleneck is solving arg min{ky ˜ Y k2 : y 2 KQ}. Sufficient (more or less): have an efficient algorithm to decide if y 2 KQ. Can we solve this problem if Q = 2-way marginals?

• No: would allow solving NP-hard problems like Max-Cut

13

{

}

A work-around

Saving grace: there exists an L so that

• 1

10L ✓ KQ ✓ L

• can efficiently solve arg min{ky ˜

Y k2 : y 2 L}

14

For

Q

• { 2- way marginals}

A

• c. hotel
• 1
• →

tz

hazy

• a. www.iz)

⇒ l*k)t①l"k

= ion

b- L

contains

all

feasible query

answers /

So

,

project

• nto

L

Cau

shall

get

rather

than Ka avg

error

d when

u x trolley

He