MELODI M achin E L earning, O ptimization, & D ata I - - PowerPoint PPT Presentation

▶
melodi
SMART_READER_LITE
LIVE PREVIEW

MELODI M achin E L earning, O ptimization, & D ata I - - PowerPoint PPT Presentation

Dec 08, 2022 6 likes •946 views

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints (SCSC/ SCSK) Rishabh Iyer Jeff Bilmes University of Washington, Seattle

slide-1
SLIDE 1

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints (SCSC/ SCSK)

Rishabh Iyer Jeff Bilmes

University of Washington, Seattle

NIPS-2013

MELODI

MachinE Learning, Optimization,

& Data Interpretation @ UW

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 0 / 20

slide-2
SLIDE 2

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Outline

1

Introduction to Submodular Functions

2

Problem Formulation of SCSC/ SCSK

3

Algorithmic Framework

4

Empirical Results

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 1 / 20

slide-3
SLIDE 3

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Set functions f : 2V → R

{

V =

, , , , , , , ,}

V is a finite “ground” set of objects. A set function f : 2V → R produces a value for any subset A ⊆ V .

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 2 / 20

slide-4
SLIDE 4

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Set functions f : 2V → R

} {

A=

, , , , }

For example, f (A) = 22,

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 2 / 20

slide-5
SLIDE 5

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Functions

Special class of set functions. f (A ∪ v) − f (A) ≥ f (B ∪ v) − f (B), if A ⊆ B (1)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

slide-6
SLIDE 6

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Functions

Special class of set functions. f (A ∪ v) − f (A) ≥ f (B ∪ v) − f (B), if A ⊆ B (1)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

slide-7
SLIDE 7

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Functions

Special class of set functions. f (A ∪ v) − f (A) ≥ f (B ∪ v) − f (B), if A ⊆ B (1) Gain = 1

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

slide-8
SLIDE 8

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Functions

Special class of set functions. f (A ∪ v) − f (A) ≥ f (B ∪ v) − f (B), if A ⊆ B (1) Gain = 1 Gain = 0

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

slide-9
SLIDE 9

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Functions

Special class of set functions. f (A ∪ v) − f (A) ≥ f (B ∪ v) − f (B), if A ⊆ B (1) Gain = 1 Gain = 0 Monotonicity: f (A) ≤ f (B), if A ⊆ B.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

slide-10
SLIDE 10

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Functions

Special class of set functions. f (A ∪ v) − f (A) ≥ f (B ∪ v) − f (B), if A ⊆ B (1) Gain = 1 Gain = 0 Monotonicity: f (A) ≤ f (B), if A ⊆ B. Modular function f (X) =

i∈X f (i) analogous to linear functions.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

slide-11
SLIDE 11

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Two Sides of Submodularity

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

slide-12
SLIDE 12

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Two Sides of Submodularity

Submodular Minimization Solve min{f (X)|X ⊆ V }. Polynomial-time. Relation to convexity. Models cooperation.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

slide-13
SLIDE 13

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Two Sides of Submodularity

Submodular Minimization Solve min{f (X)|X ⊆ V }. Polynomial-time. Relation to convexity. Models cooperation. Submodular Maximization Solve max{g(X)|X ⊆ V }. Constant-factor approximable. Relation to concavity. Models diversity and coverage.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

slide-14
SLIDE 14

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Two Sides of Submodularity

Submodular Minimization Solve min{f (X)|X ⊆ V }. Polynomial-time. Relation to convexity. Models cooperation. Submodular Maximization Solve max{g(X)|X ⊆ V }. Constant-factor approximable. Relation to concavity. Models diversity and coverage. Sometimes we want to simultaneously maximize coverage/ diversity (g) while minimizing cooperative costs (f ).

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

slide-15
SLIDE 15

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Two Sides of Submodularity

Submodular Minimization Solve min{f (X)|X ⊆ V }. Polynomial-time. Relation to convexity. Models cooperation. Submodular Maximization Solve max{g(X)|X ⊆ V }. Constant-factor approximable. Relation to concavity. Models diversity and coverage. Sometimes we want to simultaneously maximize coverage/ diversity (g) while minimizing cooperative costs (f ). Often these naturally occur as budget or cover constraints (for example, maximize diversity subject to a budget constraint on the submodular cost).

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

slide-16
SLIDE 16

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Constraints

Historically: DS optimization

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

slide-17
SLIDE 17

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Constraints

Historically: DS optimization

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

slide-18
SLIDE 18

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Constraints

Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12).

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

slide-19
SLIDE 19

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Constraints

Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway:

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

slide-20
SLIDE 20

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Constraints

Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway:

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

slide-21
SLIDE 21

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Constraints

Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway:

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

slide-22
SLIDE 22

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Constraints

Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway: While DS optimization is NP hard to approximate, SCSC and SCSK however, retain approximation guarantees!

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

slide-23
SLIDE 23

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Optimization with Submodular Constraints

Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway: While DS optimization is NP hard to approximate, SCSC and SCSK however, retain approximation guarantees! Throughout this talk, assume f and g are monotone.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

slide-24
SLIDE 24

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Our Main Contributions

Show how SCSC/SCSK subsume a number of important

  • ptimization problems.

Provide a unifying algorithmic framework for these. Provide a complete characterization of the hardness of these problems. Emphasize the scalability and practicality of some of our algorithms!

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 6 / 20

slide-25
SLIDE 25

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

I - Submodular Set Cover and Submodular Knapsack

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 7 / 20

slide-26
SLIDE 26

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

I - Submodular Set Cover and Submodular Knapsack

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 7 / 20

slide-27
SLIDE 27

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

I - Submodular Set Cover and Submodular Knapsack

Sensor Placement (Krause et al’08) Data Subset Selection (Wei et al’13) Document Summarization (Lin-Bilmes’11)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 7 / 20

slide-28
SLIDE 28

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

II - Submodular Cost with Modular Constraints

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 8 / 20

slide-29
SLIDE 29

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

II - Submodular Cost with Modular Constraints

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 8 / 20

slide-30
SLIDE 30

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

II - Submodular Cost with Modular Constraints

the yes don’t know you exactly its not that no see like so
  • kay
what but and wow they did do true we is good
  • h
have great yep think really that’s I it sure mean well right just was a if about i’m too are all to guess then understand probably definitely sounds wonderful interesting anyway absolutely pretty because

SVB-50 D-50

Limited vocabulary speech corpus selection (Lin-Bilmes’11)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 8 / 20

slide-31
SLIDE 31

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

III - Most General Case: SCSC and SCSK

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 9 / 20

slide-32
SLIDE 32

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

III - Most General Case: SCSC and SCSK

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 9 / 20

slide-33
SLIDE 33

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

III - Most General Case: SCSC and SCSK

Sensor Placement with Submodular Costs (I-Bilmes’12) Limited vocabulary and accoustically diverse speech corpus selection (Lin-Bilmes’11, Wei et al’13) Privacy preserving communication (I-Bilmes’13)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 9 / 20

slide-34
SLIDE 34

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Connections between SCSC and SCSK

Bi-criterion factors:

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 10 / 20

slide-35
SLIDE 35

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Connections between SCSC and SCSK

Bi-criterion factors:

min{f (X) : g(X) ≥ c}: [σ, ρ] approximation for SCSC is a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. [σ > 1, ρ < 1]

Approximate Solution Range Approximate Feasible Range Feasible Range

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 10 / 20

slide-36
SLIDE 36

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Connections between SCSC and SCSK

Bi-criterion factors:

min{f (X) : g(X) ≥ c}: [σ, ρ] approximation for SCSC is a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. max{g(X) : f (X) ≤ b}: [ρ, σ] approximation for SCSK is a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb. [σ > 1, ρ < 1]

Approximate Solution Range Approximate Feasible Range Feasible Range Approximate Solution Range Feasible Range Approximate Feasible Range

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 10 / 20

slide-37
SLIDE 37

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Connections between SCSC and SCSK

Bi-criterion factors:

min{f (X) : g(X) ≥ c}: [σ, ρ] approximation for SCSC is a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. max{g(X) : f (X) ≤ b}: [ρ, σ] approximation for SCSK is a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb. [σ > 1, ρ < 1]

Approximate Solution Range Approximate Feasible Range Feasible Range Approximate Solution Range Feasible Range Approximate Feasible Range

Theorem: Given a [σ, ρ] bi-criterion approx. algorithm for SCSC, we can obtain a [(1 + ǫ)ρ, σ] bi-criterion approx. algorithm for SCSK, by running the algorithm for SCSC, O(log 1

ǫ) times.

The other direction also holds!

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 10 / 20

slide-38
SLIDE 38

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Curvature of a Submodular Function

Curvature: κf = 1 − min

j∈V

f (j|V \j) f (j) and κg = 1 − min

j∈V

g(j|V \j) g(j) (2)

κ

cardinality |S| F(S) Curvature is a fundamental “complexity” parameter of a submodular function.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 11 / 20

slide-39
SLIDE 39

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Hardness (Lower bounds) of the problems

Modular g Submodular g (κg = 0) (0 < κg < 1) (κg = 1) Modular f (κf = 0) Submod f (0 < κf < 1) Submod f (κf = 1)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 12 / 20

slide-40
SLIDE 40

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Hardness (Lower bounds) of the problems

Modular g Submodular g (κg = 0) (0 < κg < 1) (κg = 1) Modular f FPTAS (κf = 0) Submod f (0 < κf < 1) Submod f (κf = 1)

Knapsack

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 12 / 20

slide-41
SLIDE 41

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Hardness (Lower bounds) of the problems

Modular g Submodular g (κg = 0) (0 < κg < 1) (κg = 1) Modular f FPTAS

1 κg (1 − e−κg )

1 − 1/e (κf = 0) Submod f (0 < κf < 1) Submod f (κf = 1)

Knapsack SSC/SK

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 12 / 20

slide-42
SLIDE 42

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Hardness (Lower bounds) of the problems

Modular g Submodular g (κg = 0) (0 < κg < 1) (κg = 1) Modular f FPTAS

1 κg (1 − e−κg )

1 − 1/e (κf = 0) Submod f Ω(

√n 1+(√n−1)(1−κf ) )

(0 < κf < 1) Submod f Ω(√n) (κf = 1)

Knapsack SSC/SK SML/SS

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 12 / 20

slide-43
SLIDE 43

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Hardness (Lower bounds) of the problems

Modular g Submodular g (κg = 0) (0 < κg < 1) (κg = 1) Modular f FPTAS

1 κg (1 − e−κg )

1 − 1/e (κf = 0) Submod f Ω(

√n 1+(√n−1)(1−κf ) )

Ω(

√n 1+(√n−1)(1−κf ) )

Ω(

√n 1+(√n−1)(1−κf ) )

(0 < κf < 1) Submod f Ω(√n) Ω(√n) Ω(√n) (κf = 1)

Knapsack SSC/SK SCSC/SCSK SML/SS

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 12 / 20

slide-44
SLIDE 44

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Hardness (Lower bounds) of the problems

Modular g Submodular g (κg = 0) (0 < κg < 1) (κg = 1) Modular f FPTAS

1 κg (1 − e−κg )

1 − 1/e (κf = 0) Submod f Ω(

√n 1+(√n−1)(1−κf ) )

Ω(

√n 1+(√n−1)(1−κf ) )

Ω(

√n 1+(√n−1)(1−κf ) )

(0 < κf < 1) Submod f Ω(√n) Ω(√n) Ω(√n) (κf = 1)

Knapsack SSC/SK SCSC/SCSK SML/SS

Hardness depends (mainly) on κf and not (so much) on that of κg.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 12 / 20

slide-45
SLIDE 45

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Algorithmic framework

Algorithm 1 General algorithmic framework to address both Problems 1 and 2

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 13 / 20

slide-46
SLIDE 46

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Algorithmic framework

Algorithm 1 General algorithmic framework to address both Problems 1 and 2

1: for t = 1, 2, · · · , T do 4: end for

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 13 / 20

slide-47
SLIDE 47

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Algorithmic framework

Algorithm 1 General algorithmic framework to address both Problems 1 and 2

1: for t = 1, 2, · · · , T do 2:

Choose surrogate functions ˆ ft and ˆ gt for f and g respectively.

4: end for

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 13 / 20

slide-48
SLIDE 48

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Algorithmic framework

Algorithm 1 General algorithmic framework to address both Problems 1 and 2

1: for t = 1, 2, · · · , T do 2:

Choose surrogate functions ˆ ft and ˆ gt for f and g respectively.

3:

Obtain X t as the optimizer of SCSC/SCSK with ˆ ft and ˆ gt instead

  • f f and g.

4: end for

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 13 / 20

slide-49
SLIDE 49

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Algorithmic framework

Algorithm 1 General algorithmic framework to address both Problems 1 and 2

1: for t = 1, 2, · · · , T do 2:

Choose surrogate functions ˆ ft and ˆ gt for f and g respectively.

3:

Obtain X t as the optimizer of SCSC/SCSK with ˆ ft and ˆ gt instead

  • f f and g.

4: end for

Surrogate functions: modular upper/ lower bounds or Ellipsoidal Approximations.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 13 / 20

slide-50
SLIDE 50

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Surrogate functions

Modular Lower Bounds: Induced via orderings of elements:

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 14 / 20

slide-51
SLIDE 51

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Surrogate functions

Modular Lower Bounds: Induced via orderings of elements:

f (X) ≤ hσ

Y (X), where hσ Y (σ(i)) = f (Σi) − f (Σi−1)

Y

σ(1) σ(2) σ(3) σ(4) σ(5) σ(6) σ(7) σ(8)

Σ1 Σ2 Σ3 Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 14 / 20

slide-52
SLIDE 52

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Surrogate functions

Modular Lower Bounds: Induced via orderings of elements:

f (X) ≤ hσ

Y (X), where hσ Y (σ(i)) = f (Σi) − f (Σi−1)

Y

σ(1) σ(2) σ(3) σ(4) σ(5) σ(6) σ(7) σ(8)

Σ1 Σ2 Σ3

Modular upper bounds:

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 14 / 20

slide-53
SLIDE 53

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Surrogate functions

Modular Lower Bounds: Induced via orderings of elements:

f (X) ≤ hσ

Y (X), where hσ Y (σ(i)) = f (Σi) − f (Σi−1)

Y

σ(1) σ(2) σ(3) σ(4) σ(5) σ(6) σ(7) σ(8)

Σ1 Σ2 Σ3

Modular upper bounds: Upper bound-I

f (X) ≤ mY ,1(X) = f (Y ) −

  • j∈Y \X

f (j|Y \j) +

  • j∈X\Y

f (j|∅)

X Y V f(j|Y \ j) f(j|∅)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 14 / 20

slide-54
SLIDE 54

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Surrogate functions

Modular Lower Bounds: Induced via orderings of elements:

f (X) ≤ hσ

Y (X), where hσ Y (σ(i)) = f (Σi) − f (Σi−1)

Y

σ(1) σ(2) σ(3) σ(4) σ(5) σ(6) σ(7) σ(8)

Σ1 Σ2 Σ3

Modular upper bounds: Upper bound-II

f (X) ≤ mY ,2(X) = f (Y ) −

  • j∈Y \X

f (j|V \j) +

  • j∈X\Y

f (j|Y )

X Y V f(j|Y ) f(j|V \ j)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 14 / 20

slide-55
SLIDE 55

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Surrogate functions

Modular Lower Bounds: Induced via orderings of elements:

f (X) ≤ hσ

Y (X), where hσ Y (σ(i)) = f (Σi) − f (Σi−1)

Y

σ(1) σ(2) σ(3) σ(4) σ(5) σ(6) σ(7) σ(8)

Σ1 Σ2 Σ3

Modular upper bounds: Upper bound-II

f (X) ≤ mY ,2(X) = f (Y ) −

  • j∈Y \X

f (j|V \j) +

  • j∈X\Y

f (j|Y )

X Y V f(j|Y ) f(j|V \ j)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 14 / 20

slide-56
SLIDE 56

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Surrogate functions

Modular Lower Bounds: Induced via orderings of elements:

f (X) ≤ hσ

Y (X), where hσ Y (σ(i)) = f (Σi) − f (Σi−1)

Y

σ(1) σ(2) σ(3) σ(4) σ(5) σ(6) σ(7) σ(8)

Σ1 Σ2 Σ3

Modular upper bounds: Upper bound-II

f (X) ≤ mY ,2(X) = f (Y ) −

  • j∈Y \X

f (j|V \j) +

  • j∈X\Y

f (j|Y )

X Y V f(j|Y ) f(j|V \ j)

Approximations: Ellipsoidal Approximation gives the tightest approximation to a submodular function.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 14 / 20

slide-57
SLIDE 57

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Cover (SSC) and Submodular Knapsack (SK)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 15 / 20

slide-58
SLIDE 58

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Cover (SSC) and Submodular Knapsack (SK)

Lemma: The greedy algorithm for SSC (Wolsey, 82) and SK (Nemhauser, 78) is special case of Algorithm 1 with g replaced by its modular lower bound.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 15 / 20

slide-59
SLIDE 59

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Submodular Set Cover (SSC) and Submodular Knapsack (SK)

Lemma: The greedy algorithm for SSC (Wolsey, 82) and SK (Nemhauser, 78) is special case of Algorithm 1 with g replaced by its modular lower bound. Approximation guarantees are constant factor – 1 − 1/e respectively.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 15 / 20

slide-60
SLIDE 60

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Iterative Submodular Set Cover (ISSC)/Submodular Knapsack (ISK)

Choose surrogate functions ˆ ft as modular upper bounds.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 16 / 20

slide-61
SLIDE 61

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Iterative Submodular Set Cover (ISSC)/Submodular Knapsack (ISK)

Choose surrogate functions ˆ ft as modular upper bounds. Fast iterative algorithms for SCSC and SCSK – Iteratively solve SSC

  • r SK.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 16 / 20

slide-62
SLIDE 62

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Iterative Submodular Set Cover (ISSC)/Submodular Knapsack (ISK)

Choose surrogate functions ˆ ft as modular upper bounds. Fast iterative algorithms for SCSC and SCSK – Iteratively solve SSC

  • r SK.

Theorem: ISSC and ISK obtain (bi-criterion) approximation factors

σ ρ = O( n 1+(n−1)(1−κf )).

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 16 / 20

slide-63
SLIDE 63

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Iterative Submodular Set Cover (ISSC)/Submodular Knapsack (ISK)

Choose surrogate functions ˆ ft as modular upper bounds. Fast iterative algorithms for SCSC and SCSK – Iteratively solve SSC

  • r SK.

Theorem: ISSC and ISK obtain (bi-criterion) approximation factors

σ ρ = O( n 1+(n−1)(1−κf )).

These algorithms also extend to SML/SS.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 16 / 20

slide-64
SLIDE 64

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Ellipsoidal Approximation Submodular Set Cover (EASSC)/ Submodular Knapsack (EASK)

Choose surrogate functions ˆ ft as Ellipsoidal Approximation, in both SCSC and SCSK.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 17 / 20

slide-65
SLIDE 65

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Ellipsoidal Approximation Submodular Set Cover (EASSC)/ Submodular Knapsack (EASK)

Choose surrogate functions ˆ ft as Ellipsoidal Approximation, in both SCSC and SCSK. Theorem: EASSC and EASK obtain (bi-criterion) approximation factors of σ

ρ = O( √n log n 1+(√n log n−1)(1−κf )).

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 17 / 20

slide-66
SLIDE 66

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Ellipsoidal Approximation Submodular Set Cover (EASSC)/ Submodular Knapsack (EASK)

Choose surrogate functions ˆ ft as Ellipsoidal Approximation, in both SCSC and SCSK. Theorem: EASSC and EASK obtain (bi-criterion) approximation factors of σ

ρ = O( √n log n 1+(√n log n−1)(1−κf )).

These algorithms also extend to SML/SS.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 17 / 20

slide-67
SLIDE 67

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Ellipsoidal Approximation Submodular Set Cover (EASSC)/ Submodular Knapsack (EASK)

Choose surrogate functions ˆ ft as Ellipsoidal Approximation, in both SCSC and SCSK. Theorem: EASSC and EASK obtain (bi-criterion) approximation factors of σ

ρ = O( √n log n 1+(√n log n−1)(1−κf )).

These algorithms also extend to SML/SS. This algorithm matches the hardness of this problem upto log factors.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 17 / 20

slide-68
SLIDE 68

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Limited Vocabulary data subset selection with Accoustic diversity

Accoustic Diversity:

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 18 / 20

slide-69
SLIDE 69

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Limited Vocabulary data subset selection with Accoustic diversity

Accoustic Diversity:

Similarity matrix sij between utterances i and j (string kernel)

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 18 / 20

slide-70
SLIDE 70

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Limited Vocabulary data subset selection with Accoustic diversity

Accoustic Diversity:

Similarity matrix sij between utterances i and j (string kernel) Submodular functions:

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 18 / 20

slide-71
SLIDE 71

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Limited Vocabulary data subset selection with Accoustic diversity

Accoustic Diversity:

Similarity matrix sij between utterances i and j (string kernel) Submodular functions:

1

Facility Location function: g(X) =

i∈V maxj∈X sij Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 18 / 20

slide-72
SLIDE 72

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Limited Vocabulary data subset selection with Accoustic diversity

Accoustic Diversity:

Similarity matrix sij between utterances i and j (string kernel) Submodular functions:

1

Facility Location function: g(X) =

i∈V maxj∈X sij 2

Saturated coverage function g(X) =

i∈V min{ j∈X sij, β j∈V sij}. Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 18 / 20

slide-73
SLIDE 73

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Limited Vocabulary data subset selection with Accoustic diversity

Accoustic Diversity:

Similarity matrix sij between utterances i and j (string kernel) Submodular functions:

1

Facility Location function: g(X) =

i∈V maxj∈X sij 2

Saturated coverage function g(X) =

i∈V min{ j∈X sij, β j∈V sij}.

Limited Vocabulary:

the yes don’t know you exactly its not that no see like so
  • kay
what but and wow they did do true we is good
  • h
have great yep think really that’s I it sure mean well right just was a if about i’m too are all to guess then understand probably definitely sounds wonderful interesting anyway absolutely pretty because SVB-50 D
  • 5

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 18 / 20

slide-74
SLIDE 74

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Limited Vocabulary data subset selection with Accoustic diversity

Accoustic Diversity:

Similarity matrix sij between utterances i and j (string kernel) Submodular functions:

1

Facility Location function: g(X) =

i∈V maxj∈X sij 2

Saturated coverage function g(X) =

i∈V min{ j∈X sij, β j∈V sij}.

Limited Vocabulary:

the yes don’t know you exactly its not that no see like so
  • kay
what but and wow they did do true we is good
  • h
have great yep think really that’s I it sure mean well right just was a if about i’m too are all to guess then understand probably definitely sounds wonderful interesting anyway absolutely pretty because SVB-50 D
  • 5

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 18 / 20

slide-75
SLIDE 75

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Limited Vocabulary data subset selection with Accoustic diversity

Accoustic Diversity:

Similarity matrix sij between utterances i and j (string kernel) Submodular functions:

1

Facility Location function: g(X) =

i∈V maxj∈X sij 2

Saturated coverage function g(X) =

i∈V min{ j∈X sij, β j∈V sij}.

Limited Vocabulary: Bipartite Neighborhood function: |γ(X)|.

the yes don’t know you exactly its not that no see like so
  • kay
what but and wow they did do true we is good
  • h
have great yep think really that’s I it sure mean well right just was a if about i’m too are all to guess then understand probably definitely sounds wonderful interesting anyway absolutely pretty because SVB-50 D
  • 5

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 18 / 20

slide-76
SLIDE 76

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Results

Compare our different algorithms on the TIMIT speech corpus.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 19 / 20

slide-77
SLIDE 77

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Results

Compare our different algorithms on the TIMIT speech corpus. Baseline is choosing random subsets.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 19 / 20

slide-78
SLIDE 78

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Results

Compare our different algorithms on the TIMIT speech corpus. Baseline is choosing random subsets. Observations:

100 200 250 10 20 30 40 50

f(X) g(X)

  • Fac. Location/ Bipartite Neighbor.
ISSC EASSCc ISK Gr EASKc Random

20 40 60 80 100 100 200 300

f(X) g(X) Saturated Sum/ Bipartite Neighbor

ISSC EASSCc ISK Gr EASKc Random

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 19 / 20

slide-79
SLIDE 79

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Results

Compare our different algorithms on the TIMIT speech corpus. Baseline is choosing random subsets. Observations:

1

All the algorithms perform much better than random subset selection.

100 200 250 10 20 30 40 50

f(X) g(X)

  • Fac. Location/ Bipartite Neighbor.
ISSC EASSCc ISK Gr EASKc Random

20 40 60 80 100 100 200 300

f(X) g(X) Saturated Sum/ Bipartite Neighbor

ISSC EASSCc ISK Gr EASKc Random

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 19 / 20

slide-80
SLIDE 80

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Results

Compare our different algorithms on the TIMIT speech corpus. Baseline is choosing random subsets. Observations:

1

All the algorithms perform much better than random subset selection.

2

The iterative and much faster algorithms, perform comparably to the slower and tight Ellipsoidal Approximation based algorithms.

100 200 250 10 20 30 40 50

f(X) g(X)

  • Fac. Location/ Bipartite Neighbor.
ISSC EASSCc ISK Gr EASKc Random

20 40 60 80 100 100 200 300

f(X) g(X) Saturated Sum/ Bipartite Neighbor

ISSC EASSCc ISK Gr EASKc Random

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 19 / 20

slide-81
SLIDE 81

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results

Conclusions/ Future Work

We proposed some very efficient (scalable) algorithms and two tight algorithms for submodular optimization under submodular constraints. In the paper: Extensions to handle multiple constraints, and non-monotone submodular functions. Future Work: Investigate our new algorithms on different real world applications.

Thank You!

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 20 / 20

slide-82
SLIDE 82

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-83
SLIDE 83

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc [σ > 1, ρ < 1].

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-84
SLIDE 84

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. A [ρ, σ] approximation for (2) = ⇒ a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb [σ > 1, ρ < 1].

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-85
SLIDE 85

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. A [ρ, σ] approximation for (2) = ⇒ a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb [σ > 1, ρ < 1]. Algorithm 2 Algorithm for SCSC using an algorithm for SCSK

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-86
SLIDE 86

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. A [ρ, σ] approximation for (2) = ⇒ a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb [σ > 1, ρ < 1]. Algorithm 2 Algorithm for SCSC using an algorithm for SCSK

1: Input: An SCSC instance, c, [ρ, σ] algorithm for SCSK, ǫ > 0.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-87
SLIDE 87

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. A [ρ, σ] approximation for (2) = ⇒ a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb [σ > 1, ρ < 1]. Algorithm 2 Algorithm for SCSC using an algorithm for SCSK

1: Input: An SCSC instance, c, [ρ, σ] algorithm for SCSK, ǫ > 0. 2: Output:

[(1 + ǫ)σ, ρ] approx. for SCSC.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-88
SLIDE 88

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. A [ρ, σ] approximation for (2) = ⇒ a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb [σ > 1, ρ < 1]. Algorithm 2 Algorithm for SCSC using an algorithm for SCSK

1: Input: An SCSC instance, c, [ρ, σ] algorithm for SCSK, ǫ > 0. 2: Output:

[(1 + ǫ)σ, ρ] approx. for SCSC.

3: b ← argminj f (j), ˆ

Xb ← ∅.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-89
SLIDE 89

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. A [ρ, σ] approximation for (2) = ⇒ a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb [σ > 1, ρ < 1]. Algorithm 2 Algorithm for SCSC using an algorithm for SCSK

1: Input: An SCSC instance, c, [ρ, σ] algorithm for SCSK, ǫ > 0. 2: Output:

[(1 + ǫ)σ, ρ] approx. for SCSC.

3: b ← argminj f (j), ˆ

Xb ← ∅.

4: while g( ˆ

Xb) < ρc do

7: end while

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-90
SLIDE 90

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. A [ρ, σ] approximation for (2) = ⇒ a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb [σ > 1, ρ < 1]. Algorithm 2 Algorithm for SCSC using an algorithm for SCSK

1: Input: An SCSC instance, c, [ρ, σ] algorithm for SCSK, ǫ > 0. 2: Output:

[(1 + ǫ)σ, ρ] approx. for SCSC.

3: b ← argminj f (j), ˆ

Xb ← ∅.

4: while g( ˆ

Xb) < ρc do

5:

b ← (1 + ǫ)b

6:

ˆ Xb ← [ρ, σ] approx. for SCSK using b.

7: end while

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-91
SLIDE 91

Extra Slides

Connections between SCSC and SCSK

SCSC and SCSK are closely related, and can be transformed into

  • ne another!

Bi-criterion factor: [σ, ρ] approximation for (1) = ⇒ a set X : f (X) ≤ σf (X ∗) and g(X) ≥ ρc. A [ρ, σ] approximation for (2) = ⇒ a set X : g(X) ≥ ρg(X ∗) and f (X) ≤ σb [σ > 1, ρ < 1]. Algorithm 2 Algorithm for SCSC using an algorithm for SCSK

1: Input: An SCSC instance, c, [ρ, σ] algorithm for SCSK, ǫ > 0. 2: Output:

[(1 + ǫ)σ, ρ] approx. for SCSC.

3: b ← argminj f (j), ˆ

Xb ← ∅.

4: while g( ˆ

Xb) < ρc do

5:

b ← (1 + ǫ)b

6:

ˆ Xb ← [ρ, σ] approx. for SCSK using b.

7: end while 8: Return ˆ

Xb.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 21 / 20

slide-92
SLIDE 92

Extra Slides

Hardness Theorem

Theorem: For any κ > 0, there exists submodular function f with curvature κf = κ such that no polynomial time algorithm for SCSC and SCSK σ

ρ = n1/2−ǫ 1+(n1/2−ǫ−1)(1−κ) for any ǫ > 0.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 22 / 20

slide-93
SLIDE 93

Extra Slides

Hardness Theorem

Theorem: For any κ > 0, there exists submodular function f with curvature κf = κ such that no polynomial time algorithm for SCSC and SCSK σ

ρ = n1/2−ǫ 1+(n1/2−ǫ−1)(1−κ) for any ǫ > 0.

Hardness depends on the curvature of the submodular function f and not on that of g.

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 22 / 20

slide-94
SLIDE 94

Extra Slides

Hardness Theorem

Theorem: For any κ > 0, there exists submodular function f with curvature κf = κ such that no polynomial time algorithm for SCSC and SCSK σ

ρ = n1/2−ǫ 1+(n1/2−ǫ−1)(1−κ) for any ǫ > 0.

Hardness depends on the curvature of the submodular function f and not on that of g.

Modular g Submodular g (κg = 0) (0 < κg < 1) (κg = 1) Modular f FPTAS

1 κg (1 − e−κg )

1 − 1/e (κf = 0) Submod f Ω(

√n 1+(√n−1)(1−κf ) )

Ω(

√n 1+(√n−1)(1−κf ) )

Ω(

√n 1+(√n−1)(1−κf ) )

(0 < κf < 1) Submod f Ω(√n) Ω(√n) Ω(√n) (κf = 1)

Table: Summary of Hardness results for SCSC/ SCSK

Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 22 / 20