Submodular Streaming in All Its Glory: Tight Approximation, Minimum - - PowerPoint PPT Presentation

Submodular Streaming in All Its Glory:

Tight Approximation, Minimum Memory and Low Adaptive Complexity

Ehsan Kazemi1, Marko Mitrovic1, Morteza Zadimoghaddam2, Silvio Lattanzi2, Amin Karbasi1

1Yale University and 2Google

Many practical scenarios we need to use streaming algorithms: the data arrives at a very fast pace there is only time to read the data once random access to the entire data is not possible and only a small fraction of the data can be loaded to the main memory

Streaming Algorithms

2 Summary Video from “Britain’s Got Talent”

Many practical scenarios we need to use streaming algorithms: the data arrives at a very fast pace there is only time to read the data once random access to the entire data is not possible and only a small fraction of the data can be loaded to the main memory

Streaming Algorithms

2

Is it possible to summarize a massive data set “on the fly”, i.e., when at any point of time we have access only to a small fraction of data?

Summary Video from “Britain’s Got Talent”

Submodularity

3

Submodularity

3

s1 s2 s1 s2 s3 s4

A = {s1, s2} B = {s1, s2, s3, s4}

Submodularity

3

s1 s2 s1 s2 s3 s4

A = {s1, s2} B = {s1, s2, s3, s4}

Submodularity

Submodularity

3

s1 s2 s1 s2 s3 s4 s s

∀ A ⊆ B ⊆ V and s ∈ V \ B A = {s1, s2} B = {s1, s2, s3, s4}

Submodularity

Submodularity

3

s1 s2 s1 s2 s3 s4 s s

∀ A ⊆ B ⊆ V and s ∈ V \ B A = {s1, s2} B = {s1, s2, s3, s4}

Submodularity

Submodularity

3

s1 s2 s1 s2 s3 s4 s s

∀ A ⊆ B ⊆ V and s ∈ V \ B A = {s1, s2} B = {s1, s2, s3, s4} f(A ∪ {s}) − f(A) ≥ f(B ∪ {s}) − f(B)

Submodularity

Submodularity

3

s1 s2 s1 s2 s3 s4 s s

∀ A ⊆ B ⊆ V and s ∈ V \ B A = {s1, s2} B = {s1, s2, s3, s4} f(A ∪ {s}) − f(A) ≥ f(B ∪ {s}) − f(B)

Submodularity Monotonicity

∀A ⊆ B ⊆ V

f(A) ≤ f(B)

Can we find a good approximation of ? Submodularity: Find better lower bounds for as more elements arrive

Closing the Gap: SIEVE-STREAMING++

m ≤ OPT ≤ k · m, m = max

e∈V f({e})

OPT

] ]

τ ∗

m

k · m

τ = (1 + ε)i

Choosing elements with marginal gain at least returns a set with an objective value of at least S OPT τ ∗ = OPT/(2k) f(S) ≥ OPT/2

] ]

m

k · m

τ = (1 + ε)i

Lower bound onOPT

“Submodular Streaming in All Its Glory: Tight Approximation, Minimum Memory and Low Adaptive Complexity”, ICML’19

[Kazemi, Mitrovic, Zadimoghaddam, Lattanzi, Karbasi]

For monotone submodular functions, SIEVE-STREAMING++ with a memory gives constant factor approximation using only function evaluation per element

O(log(k)/ε) O(k)

f (SSIEVE-STREAMING++) ≥ (1/2) OPT

Streaming Algorithms: Cardinality Constraint

5

Approximation Factor Memory Complexity

Ω(n/k)

Hard

[Norouzi-Fard et al]

O(k)

Impossible

Streaming Algorithms: Cardinality Constraint

5

Approximation Factor Memory Complexity

Ω(n/k)

Hard

[Norouzi-Fard et al]

O(k)

Impossible

[Buchbinder et al., SODA’2015]

Streaming Algorithms: Cardinality Constraint

5

Approximation Factor Memory Complexity

Ω(n/k)

Hard

[Norouzi-Fard et al]

O(k log k) O(k)

Impossible

[Buchbinder et al., SODA’2015] [Badanidiyuru et al., KDD’2014]

Streaming Algorithms: Cardinality Constraint

5

Approximation Factor Memory Complexity

Ω(n/k)

Hard

[Norouzi-Fard et al]

O(k log k) O(k)

Impossible

[Buchbinder et al., SODA’2015] [Badanidiyuru et al., KDD’2014]

[Kazemi et al., ICML’2019]

BATCH-SIEVE-STREAMING++

6

Data Stream Buffer

τ

Sτ

f(T | Sτ) |T| ≥ (1 − ε)τ

Sample set of size 4

T

For each threshold Full No Filter

T

Yes

Multi-source Setting

7

· · ·

Thank You !