Cutting Convex Sets With Margin Shay Moran (Google AI and Technion) - - PowerPoint PPT Presentation

▶
cutting convex sets with margin
SMART_READER_LITE
LIVE PREVIEW

Cutting Convex Sets With Margin Shay Moran (Google AI and Technion) - - PowerPoint PPT Presentation

Nov 25, 2022 26 likes •266 views

Cutting Convex Sets With Margin Shay Moran (Google AI and Technion) Background A geometric problem that arises in Density Estimation Density Estimation = Distribution Learning w.r.t Total Variation Progress on problem improved

slide-1
SLIDE 1

Cutting Convex Sets With Margin

Shay Moran (Google AI and Technion)

slide-2
SLIDE 2

Background

  • A geometric problem that arises in Density Estimation
  • Density Estimation = Distribution Learning w.r.t Total Variation
  • Progress on problem ⟹ improved sample complexity bounds

for optimal density estimators

  • Based on joint works with

Olivier Bousquet, Mark Braverman, Klim Efremenko, Daniel Kane, and Gillat Kol

slide-3
SLIDE 3

The Game

  • Fix a norm ! on ℝ! and 𝜗 > 0
  • 𝐶( ⃗

𝑦, 𝑠) = ball of radius 𝑠 around ⃗ 𝑦

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦" ∈ 𝐶"

  • Adversary picks halfspace 𝐼" disjoint from 𝐶 ⃗

𝑦", 𝜗

  • Update 𝐶"#$ = 𝐶" ∩ 𝐼"
  • Player wants to reach 𝐶" = ∅ as fast as possible

𝜗

slide-4
SLIDE 4

Example: ℓ! in 2 dimensions

Cu#ng Game

  • Player versus Adversary
  • 𝑪𝟏 = 𝑪(𝟏, 𝟐)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible
slide-5
SLIDE 5

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks 𝒚𝒖 ∈ 𝑪𝒖
  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • !0 = !(0, 1)
  • At round ( = 0,1, …
  • Player picks *! ∈ ,!
  • Adversary picks halfspace -" disjoint from ! ⃗

/", 0

  • Update !"#$ = !" ∩ -"
  • Player wants to reach !" = ∅ as fast as possible

Round

1

slide-6
SLIDE 6

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝑰𝒖 disjoint from 𝑪 𝒚𝒖, 𝝑
  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

1

slide-7
SLIDE 7

Example: ℓ! in 2 dimensions

Cu#ng Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝑪𝒖"𝟐 = 𝑪𝒖 ∩ 𝑰𝒖
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

1

slide-8
SLIDE 8

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks 𝒚𝒖 ∈ 𝑪𝒖
  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

2

slide-9
SLIDE 9

Example: ℓ! in 2 dimensions

Cu#ng Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝑰𝒖 disjoint from 𝑪 𝒚𝒖, 𝝑
  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

2

slide-10
SLIDE 10

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝑪𝒖"𝟐 = 𝑪𝒖 ∩ 𝑰𝒖
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

2

slide-11
SLIDE 11

Example: ℓ! in 2 dimensions

Cu#ng Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks 𝒚𝒖 ∈ 𝑪𝒖
  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

3

slide-12
SLIDE 12

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝑰𝒖 disjoint from 𝑪 𝒚𝒖, 𝝑
  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

3

slide-13
SLIDE 13

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝑪𝒖"𝟐 = 𝑪𝒖 ∩ 𝑰𝒖
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

3

slide-14
SLIDE 14

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

4

slide-15
SLIDE 15

Example: ℓ! in 2 dimensions

Cu#ng Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝑰𝒖 disjoint from 𝑪 𝒚𝒖, 𝝑
  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

4

slide-16
SLIDE 16

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝑪𝒖"𝟐 = 𝑪𝒖 ∩ 𝑰𝒖
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

4

slide-17
SLIDE 17

Example: ℓ! in 2 dimensions

Cu#ng Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks 𝒚𝒖 ∈ 𝑪𝒖
  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

5

slide-18
SLIDE 18

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝑰𝒖 disjoint from 𝑪 𝒚𝒖, 𝝑
  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

5

slide-19
SLIDE 19

Example: ℓ! in 2 dimensions

Cu#ng Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝑪𝒖"𝟐 = 𝑪𝒖 ∩ 𝑰𝒖
  • Player wants to reach 𝐶! = ∅ as fast as possible

Round

5

slide-20
SLIDE 20

Example: ℓ! in 2 dimensions

Cutting Game

  • Player versus Adversary
  • 𝐶0 = 𝐶(0, 1)
  • At round 𝑢 = 0,1, …
  • Player picks ⃗

𝑦! ∈ 𝐶!

  • Adversary picks halfspace 𝐼! disjoint from 𝐶 ⃗

𝑦!, 𝜗

  • Update 𝐶!"# = 𝐶! ∩ 𝐼!
  • Player wants to reach 𝑪𝒖 = ∅ as fast as possible

Round

6

Player wins at round 𝑢 = 6

slide-21
SLIDE 21

The Problem

  • Goal. Provide tight bounds on 𝑈

" , 𝑒, 𝜗

  • Arbitrary norms?

\\can be further extended to convex sets

  • Norm = ℓ"?

𝑈 % , 𝑒, 𝜗 : = min number of rounds 𝑈 s.t. player has a strategy that guarantees 𝐶# = ∅ against any adversary

slide-22
SLIDE 22

Known Bounds

  • Goal. Provide tight bounds on 𝑈

" , 𝑒, 𝜗

𝑈 % , 𝑒, 𝜗 : = min number of rounds 𝑈 s.t. player has a strategy that guarantees 𝐶# = ∅ against any adversary

  • (∀ norm % ): 𝑈 ≤ 𝑃(𝑒 log 1/𝜗 )
  • (∃ norm % ): 𝑈 ≥ Ω(𝑒 log 1/𝜗 )

\\ ! = ℓ"

slide-23
SLIDE 23

Known Bounds

  • Goal. Provide tight bounds on 𝑈

" , 𝑒, 𝜗

𝑈 % , 𝑒, 𝜗 : = min number of rounds 𝑈 s.t. player has a strategy that guarantees 𝐶# = ∅ against any adversary

  • ℓ$:

\\related to optimal density estimator

  • 𝑈 ≤ 𝑃

%&' ( )!

  • 𝑈 ≥ Ω

%&' ( )

\\ 𝑒 ≥ & Ω

# $&

  • ℓ" for 𝑞 ∈ 1,2 : 𝑈 ≤ 𝑃"

$ )!

\\indepenent of 𝑒

  • ℓ" for 𝑞 ∈ (2, ∞): 𝑈 ≤ 𝑃

(

"#! $

)!

slide-24
SLIDE 24