with Constant Multiplicative Error Uri Stemmer Ben-Gurion - - PowerPoint PPT Presentation

Differentially Private k-Means

with Constant Multiplicative Error

Uri Stemmer Ben-Gurion University

joint work with Haim Kaplan

What is 𝒍-Means Clustering?

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

What is 𝒍-Means Clustering?

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

✓ Probably the most well-studied clustering problem ✓ Tons of applications ✓ Super popular

What is 𝒍-Means Clustering?

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

What is Differentially Private 𝒍-Means?

[Dwork, McSherry, Nissim, Smith 06] (informal)

 Every data point 𝒚𝒋 represents the (private) information of one individual  Goal: the output (the set of centers) does not reveal information that is specific to any single individual

What is 𝒍-Means Clustering?

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

What is Differentially Private 𝒍-Means?

[Dwork, McSherry, Nissim, Smith 06] (informal)

 Every data point 𝒚𝒋 represents the (private) information of one individual  Goal: the output (the set of centers) does not reveal information that is specific to any single individual  Requirement: the output distribution is insensitive to any arbitrarily change of a single input point (an algorithm satisfying this requirement is differentially private)

What is 𝒍-Means Clustering?

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

What is Differentially Private 𝒍-Means?

[Dwork, McSherry, Nissim, Smith 06] (informal)

 Every data point 𝒚𝒋 represents the (private) information of one individual  Goal: the output (the set of centers) does not reveal information that is specific to any single individual  Requirement: the output distribution is insensitive to any arbitrarily change of a single input point (an algorithm satisfying this requirement is differentially private)

Why is that a good privacy definition?

Even if an observer knows all other data point but mine, and now she sees the outcome

• f the computation, then she still cannot learn “anything” on my data point

Differentially Private 𝒍-Means Clustering

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

Requirement: the output distribution is insensitive to any arbitrarily change of a single input point

Differentially Private 𝒍-Means Clustering

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

Requirement: the output distribution is insensitive to any arbitrarily change of a single input point

Observe: With privacy we must have additive error

• Assume 𝒍 = 𝒐 = 𝟒
• OPT’s cost = 0

Differentially Private 𝒍-Means Clustering

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

Requirement: the output distribution is insensitive to any arbitrarily change of a single input point

Observe: With privacy we must have additive error

𝚳

• Assume 𝒍 = 𝒐 = 𝟒
• OPT’s cost = 0
• Move one point
• OPT’s cost = 0

Differentially Private 𝒍-Means Clustering

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

Requirement: the output distribution is insensitive to any arbitrarily change of a single input point

Observe: With privacy we must have additive error

𝚳

• Assume 𝒍 = 𝒐 = 𝟒
• OPT’s cost = 0
• Move one point
• OPT’s cost = 0
• Each solution must remain approx. equally likely
• On at least one of these inputs our cost is ≈ 𝚳𝟑

Differentially Private 𝒍-Means Clustering

Given: Data points 𝑻 = 𝒚𝟐, … , 𝒚𝒐 ∈ ℝ𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗𝟐, … , 𝒗𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨ℓ 𝒚𝒋 − 𝒗ℓ 𝟑

𝒋

Requirement: the output distribution is insensitive to any arbitrarily change of a single input point

Observe: With privacy we must have additive error

𝚳

• Assume 𝒍 = 𝒐 = 𝟒
• OPT’s cost = 0
• Move one point
• OPT’s cost = 0
• Each solution must remain approx. equally likely
• On at least one of these inputs our cost is ≈ 𝚳𝟑

⟹ We assume that input points come from the unit ball

Ref Model Runtime Bounds GLMRT’10

differential privacy

𝒐𝒆 𝐏 𝟐 ⋅𝐏𝐐𝐔 + 𝑷 𝒍𝟑 ⋅ 𝒆 NCBN’16

differential privacy

𝐪𝐩𝐦𝐳 𝐏 𝐦𝐩𝐡 𝒍 ⋅𝐏𝐐𝐔 + 𝑷 𝒐 FXZR’17

differential privacy

𝐪𝐩𝐦𝐳 𝑷 𝒍 𝐦𝐩𝐡 𝒐 ⋅𝐏𝐐𝐔 + 𝑷 𝒍𝟒/𝟑 ⋅ 𝒆 BDLMZ’17

differential privacy

𝐪𝐩𝐦𝐳 𝑷 𝐦𝐩𝐡𝟒𝒐 ⋅𝐏𝐐𝐔 + 𝑷 𝒍𝟑 + 𝒆 NS’18

differential privacy

𝐪𝐩𝐦𝐳 𝑷 𝒍 ⋅𝐏𝐐𝐔 + 𝑷 𝒍𝟐.𝟔𝟐 ⋅ 𝒆𝟏.𝟔𝟐 New

differential privacy

𝐪𝐩𝐦𝐳 𝑷 𝟐 ⋅𝐏𝐐𝐔 + 𝑷 𝒍𝟐.𝟏𝟐 ⋅ 𝒆𝟏.𝟔𝟐 + 𝒍𝟒/𝟑

Previous and New Bounds