melodi
play

MELODI M achin E L earning, O ptimization, & D ata I - PowerPoint PPT Presentation

Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints (SCSC/ SCSK) Rishabh Iyer Jeff Bilmes University of Washington, Seattle


  1. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints (SCSC/ SCSK) Rishabh Iyer Jeff Bilmes University of Washington, Seattle NIPS-2013 MELODI M achin E L earning, O ptimization, & D ata I nterpretation @ UW Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 0 / 20

  2. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Outline Introduction to Submodular Functions 1 Problem Formulation of SCSC/ SCSK 2 Algorithmic Framework 3 Empirical Results 4 Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 1 / 20

  3. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Set functions f : 2 V → R { , } , , , V = , , , , V is a finite “ground” set of objects. A set function f : 2 V → R produces a value for any subset A ⊆ V . Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 2 / 20

  4. } Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Set functions f : 2 V → R { , } , , A = , For example, f ( A ) = 22, Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 2 / 20

  5. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Set Functions Special class of set functions. f ( A ∪ v ) − f ( A ) ≥ f ( B ∪ v ) − f ( B ) , if A ⊆ B (1) Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

  6. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Set Functions Special class of set functions. f ( A ∪ v ) − f ( A ) ≥ f ( B ∪ v ) − f ( B ) , if A ⊆ B (1) Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

  7. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Set Functions Special class of set functions. f ( A ∪ v ) − f ( A ) ≥ f ( B ∪ v ) − f ( B ) , if A ⊆ B (1) Gain = 1 Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

  8. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Set Functions Special class of set functions. f ( A ∪ v ) − f ( A ) ≥ f ( B ∪ v ) − f ( B ) , if A ⊆ B (1) Gain = 1 Gain = 0 Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

  9. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Set Functions Special class of set functions. f ( A ∪ v ) − f ( A ) ≥ f ( B ∪ v ) − f ( B ) , if A ⊆ B (1) Gain = 1 Gain = 0 Monotonicity: f ( A ) ≤ f ( B ) , if A ⊆ B . Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

  10. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Set Functions Special class of set functions. f ( A ∪ v ) − f ( A ) ≥ f ( B ∪ v ) − f ( B ) , if A ⊆ B (1) Gain = 1 Gain = 0 Monotonicity: f ( A ) ≤ f ( B ) , if A ⊆ B . Modular function f ( X ) = � i ∈ X f ( i ) analogous to linear functions. Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 3 / 20

  11. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Two Sides of Submodularity Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

  12. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Two Sides of Submodularity Submodular Minimization Solve min { f ( X ) | X ⊆ V } . Polynomial-time. Relation to convexity. Models cooperation. Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

  13. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Two Sides of Submodularity Submodular Maximization Submodular Minimization Solve max { g ( X ) | X ⊆ V } . Solve min { f ( X ) | X ⊆ V } . Constant-factor approximable. Polynomial-time. Relation to concavity. Relation to convexity. Models diversity and coverage. Models cooperation. Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

  14. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Two Sides of Submodularity Submodular Maximization Submodular Minimization Solve max { g ( X ) | X ⊆ V } . Solve min { f ( X ) | X ⊆ V } . Constant-factor approximable. Polynomial-time. Relation to concavity. Relation to convexity. Models diversity and coverage. Models cooperation. Sometimes we want to simultaneously maximize coverage/ diversity ( g ) while minimizing cooperative costs ( f ). Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

  15. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Two Sides of Submodularity Submodular Maximization Submodular Minimization Solve max { g ( X ) | X ⊆ V } . Solve min { f ( X ) | X ⊆ V } . Constant-factor approximable. Polynomial-time. Relation to concavity. Relation to convexity. Models diversity and coverage. Models cooperation. Sometimes we want to simultaneously maximize coverage/ diversity ( g ) while minimizing cooperative costs ( f ). Often these naturally occur as budget or cover constraints (for example, maximize diversity subject to a budget constraint on the submodular cost). Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 4 / 20

  16. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Constraints Historically: DS optimization Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

  17. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Constraints Historically: DS optimization Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

  18. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Constraints Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

  19. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Constraints Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway: Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

  20. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Constraints Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway: Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

  21. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Constraints Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway: Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

  22. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Constraints Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway: While DS optimization is NP hard to approximate, SCSC and SCSK however, retain approximation guarantees! Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

  23. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Submodular Optimization with Submodular Constraints Historically: DS optimization Unfortunately, NP hard to approximate (Iyer-Bilmes’12). We introduce the following, which is often more natual anyway: While DS optimization is NP hard to approximate, SCSC and SCSK however, retain approximation guarantees! Throughout this talk, assume f and g are monotone. Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 5 / 20

  24. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results Our Main Contributions Show how SCSC/SCSK subsume a number of important optimization problems. Provide a unifying algorithmic framework for these. Provide a complete characterization of the hardness of these problems. Emphasize the scalability and practicality of some of our algorithms! Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 6 / 20

  25. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results I - Submodular Set Cover and Submodular Knapsack Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 7 / 20

  26. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results I - Submodular Set Cover and Submodular Knapsack Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 7 / 20

  27. Submodular Functions Problem Formulation Algorithmic Framework Empirical Results I - Submodular Set Cover and Submodular Knapsack Data Subset Selection Sensor Placement (Wei et al’13) Document Summarization (Krause et al’08) (Lin-Bilmes’11) Iyer & Bilmes, 2013 (UW, Seattle) SCSC/SCSK NIPS-2013 7 / 20

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend