of Geometric Concepts Uri Stemmer Ben-Gurion University joint work - - PowerPoint PPT Presentation

β–Ά
of geometric concepts
SMART_READER_LITE
LIVE PREVIEW

of Geometric Concepts Uri Stemmer Ben-Gurion University joint work - - PowerPoint PPT Presentation

Aug 02, 2023 218 likes β€’380 views

POSTER #124 Differentially Private Learning of Geometric Concepts Uri Stemmer Ben-Gurion University joint work with Haim Kaplan, Yishay Mansour, and Yossi Matias Privately Learning Union of Polygons POSTER #124 Given: points in

slide-1
SLIDE 1

Differentially Private Learning

  • f Geometric Concepts

Uri Stemmer Ben-Gurion University

joint work with

Haim Kaplan, Yishay Mansour, and Yossi Matias POSTER #124

slide-2
SLIDE 2

Privately Learning Union of Polygons

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error POSTER #124

slide-3
SLIDE 3

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data:

Privately Learning Union of Polygons

POSTER #124

slide-4
SLIDE 4

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data: οƒΌ Every labeled example represents the (private) information of one individual οƒΌ Goal: the output hypothesis does not reveal information that is specific to any single individual

Privately Learning Union of Polygons

POSTER #124

slide-5
SLIDE 5

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data: οƒΌ Every labeled example represents the (private) information of one individual οƒΌ Goal: the output hypothesis does not reveal information that is specific to any single individual οƒΌ Requirement: the output distribution is insensitive to any arbitrarily change of a single input example (an algorithm satisfying this requirement is differentially private)

Privately Learning Union of Polygons

POSTER #124

slide-6
SLIDE 6

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data: οƒΌ Every labeled example represents the (private) information of one individual οƒΌ Goal: the output hypothesis does not reveal information that is specific to any single individual οƒΌ Requirement: the output distribution is insensitive to any arbitrarily change of a single input example (an algorithm satisfying this requirement is differentially private)

Why y is tha hat t a go good

  • d pr

privacy vacy definiti nition?

  • n?

Even if an observer knows all other data point but mine, and now she sees the

  • utcome of the computation, then she still cannot learn β€œanything” on my data point

Privately Learning Union of Polygons

POSTER #124

slide-7
SLIDE 7

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data

Privately Learning Union of Polygons

POSTER #124

slide-8
SLIDE 8

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data

Motivation: Analyzing Users’ Location Reports

  • Analyzing GPS navigation data
  • Learning the shape of a flood or a fire based on reports
  • Identifying regions with poor cellular reception based on reports

Privately Learning Union of Polygons

POSTER #124

slide-9
SLIDE 9

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data

Privately Learning Union of Polygons

POSTER #124

slide-10
SLIDE 10

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data

Differential Privacy and Discretization

  • Impossibility results for differential privacy show that this problem (and even much simpler problems) cannot

be solved over infinite domains

Privately Learning Union of Polygons

POSTER #124

slide-11
SLIDE 11

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data

Differential Privacy and Discretization

  • Impossibility results for differential privacy show that this problem (and even much simpler problems) cannot

be solved over infinite domains

  • We assume that input points come from 𝒆 πŸ‘ = 𝟐, πŸ‘, … , 𝒆 Γ— 𝟐, πŸ‘, … , 𝒆 for a discretization parameter 𝒆

Privately Learning Union of Polygons

POSTER #124

slide-12
SLIDE 12

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data

Differential Privacy and Discretization

  • Impossibility results for differential privacy show that this problem (and even much simpler problems) cannot

be solved over infinite domains

  • We assume that input points come from 𝒆 πŸ‘ = 𝟐, πŸ‘, … , 𝒆 Γ— 𝟐, πŸ‘, … , 𝒆 for a discretization parameter 𝒆
  • Furthermore, the sample complexity must grow with the size of the discretization

Privately Learning Union of Polygons

POSTER #124

slide-13
SLIDE 13

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data 𝒆 πŸ‘

Privately Learning Union of Polygons

POSTER #124

slide-14
SLIDE 14

Previous Result

Private learner with sample complexity 𝐏 𝒍 β‹… 𝐦𝐩𝐑 𝒆 and runtime β‰ˆ 𝒆𝒍 (using a generic tool of MT’07) Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data 𝒆 πŸ‘

Privately Learning Union of Polygons

POSTER #124

slide-15
SLIDE 15

Previous Result New Result

Private learner with sample complexity 𝐏 𝒍 β‹… 𝐦𝐩𝐑 𝒆 and runtime β‰ˆ 𝒆𝒍 (using a generic tool of MT’07)

Private learner with sample complexity 𝐏 𝒍 β‹… 𝐦𝐩𝐑 𝒆 and runtime πͺ𝐩𝐦𝐳 𝒍, 𝐦𝐩𝐑 𝒆

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data 𝒆 πŸ‘

Privately Learning Union of Polygons

POSTER #124

slide-16
SLIDE 16

Previous Result New Result

Private learner with sample complexity 𝐏 𝒍 β‹… 𝐦𝐩𝐑 𝒆 and runtime β‰ˆ 𝒆𝒍 (using a generic tool of MT’07)

Private learner with sample complexity 𝐏 𝒍 β‹… 𝐦𝐩𝐑 𝒆 and runtime πͺ𝐩𝐦𝐳 𝒍, 𝐦𝐩𝐑 𝒆

Construction Idea (oversimplified)

  • Via triangulation, positive examples can be covered using 𝒍 triangles
  • Search for β‰ˆ 𝒍 β‹… 𝐦𝐩𝐑 𝒐 triangles in iterations using a private variant of the greedy algorithm for set-cover

Summary

οƒΌ New algorithm for privately learning union of polygons οƒΌ Efficient runtime and sample complexity οƒΌ Applications to privately analyzing users’ location data

Given: 𝒐 points in β„πŸ‘ with binary labels: π’šπ’‹, 𝒛𝒋

𝒋=𝟐 𝒐

Assume: βˆƒcollection of polygons π‘ΈπŸ, … , 𝑸𝒖 with a total of al most 𝒍 edges s.t. βˆ€π’‹ ∈ 𝒐 : π’šπ’‹ ∈ π‘Έπ’Œ

π’Œ

⟺ 𝒛𝒋 = 𝟐 Find: Hypothesis π’Š: β„πŸ‘ β†’ 𝟏, 𝟐 with small error, while providing differential privacy for the training data 𝒆 πŸ‘

Privately Learning Union of Polygons

POSTER #124