functions maximization Yao Zhang 03/19/2015 Example Deploy - - PowerPoint PPT Presentation

▶
functions maximization
SMART_READER_LITE
LIVE PREVIEW

functions maximization Yao Zhang 03/19/2015 Example Deploy - - PowerPoint PPT Presentation

Sep 10, 2023 299 likes •613 views

A survey of submodular functions maximization Yao Zhang 03/19/2015 Example Deploy sensors in the water distribution network to detect contamination F(S): the performance of the detection when a set S of places is selected 2

slide-1
SLIDE 1

A survey of submodular functions maximization

Yao Zhang 03/19/2015

slide-2
SLIDE 2

Example

  • Deploy sensors in the water distribution

network to detect contamination

  • F(S): the performance of the detection

when a set S of places is selected

2

slide-3
SLIDE 3

Definition of submodular functions

  • finite ground set V={1,2,...n}
  • set function f(S):
  • Marginal gain:
  • Submodular: diminishing return

3

slide-4
SLIDE 4
  • Modular function:
  • Supermodular

Definition of submodular functions

4

slide-5
SLIDE 5

Examples of submodular function

  • Deploy sensors in the water distribution network to

detect contamination

From Krause’s survey

5

slide-6
SLIDE 6

Examples of submodular function

  • From Krause’s survey
  • Weighted coverage functions
  • Entropy
  • Mutual information
  • Cut capacity functions
  • …
  • Influence function (Kempe 03)
  • f(S): the expected number of infected nodes when

nodes in S are infected at the start

  • Propagation model
  • Linear threshold
  • Independent Cascade

6

slide-7
SLIDE 7

Properties of submodular function

  • Linear combination
  • if set functions F1,...,Fm are submodular

functions, and a1,...,am>0

  • then is submodular
  • Concavity

7

slide-8
SLIDE 8
  • Except for the submodularity, we assume:
  • 1. Monotonically non-decreasing
  • 2.

8

Submodularity optimization

slide-9
SLIDE 9

Summary of submodularity optimization

Focus on this part

9

For more details, see http://submodularity.org/

slide-10
SLIDE 10

Maximization of submodular functions

  • Problem:
  • Simplest constraint
  • cardinality constraints
  • for a given k, we require that
  • Greedy algorithm

10

slide-11
SLIDE 11

Greedy Algorithm

11

slide-12
SLIDE 12

Matroid constraints

12

Greedy algorithm is an ½-approximation algorithm

slide-13
SLIDE 13

Knapsack constraint

  • Knapsack constraint:
  • Greedy Algorithms:
  • Analysis:
  • Scb: the solution provided by the cost-benefit

greedy algorithm

  • Suc: the solution returned by returned by the

uniform cost

13

[Leskovec et al. KDD 2007]

slide-14
SLIDE 14

Speeding up the greedy algorithm

  • Lazy evaluation [Leskovec et al. KDD 2007]
  • First iteration as usual
  • Keep an order list of marginal gain Δi from the previous

iteration

  • Re-evaluate the marginal gain only for top element i
  • if Δi stays on top, user it, otherwise re-sort

14

Sorted list in the descending

  • rder: t, s at i-th iteration

In the (i+1)-th iteration:

slide-15
SLIDE 15

Fast algorithms

  • Summary:
  • Randomized Greedy

15

Badanidiyuru, Ashwinkumar, and Jan Vondrák. "Fast algorithms for maximizing submodular functions." SODA2014. w is the threshold

slide-16
SLIDE 16

Lazier Than Lazy Greedy

  • Random Sampling
  • Analysis: (1-1/e-ε) approximation

16

Mirzasoleiman, Baharan, Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondrák, and Andreas Krause. "Lazier Than Lazy Greedy." AAAI 2015.

Cardinality Constraint

slide-17
SLIDE 17

Complex constraints

  • Submodular maximization using the multilinear extension
  • Submodular optimization over graphs
  • the set S forms a path, or a tree on G of weight at most B
  • Robust submodular optimization
  • Consider adversaries (Game theory)
  • Nonmonotone submodular functions
  • E.g., a monotone submodular function f, and a modular cost

function c

  • We want to max. non-monotone function

17

All have good approximations using Greedy based algorithm (See Krause’s survey for details)

slide-18
SLIDE 18
  • The objective may not be known in advance
  • Objectives functions {f1,…,fT} drawn from

some distribution

  • At each round, select certain element
  • Two settings:
  • no-regret setting
  • the choices in any round are not constrained by what one

did in previous rounds and the goal is to perform well on average

  • competitive setting
  • a sequence of irrevocable decisions
  • Previous round choices may affect the decision of the

current round

18

Online maximization of submodular functions

slide-19
SLIDE 19

Adaptive submodularity

  • We wish to adaptively select a set, observing

and taking into account feedback after selecting any particular element.

  • E.g., when placing the next sensor, adaptively

taking into account measurements provided by the sensors selected so far

  • Active learning
  • …

19

Daniel Golovin and Andreas Krause, . Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization. Journal of Artificial Intelligence Research (JAIR), 2011.

slide-20
SLIDE 20

Recent Progress

slide-21
SLIDE 21

Recent Progress

  • Submodular Welfare problem
  • Submodular function over integer lattice
  • Distributed Submodular Maximization
  • Streaming Submodular Maximization
  • Submodular Optimization with Submodular

Cover and Submodular Knapsack Constraints

21

slide-22
SLIDE 22

Submodular Welfare problem

  • Problem:
  • Algorithm:
  • Continuous Greedy Algorithm provides (1-1/e-o(1))-

appriximation

22

Jan Vondrák. "Optimal approximation for the submodular welfare problem in the value oracle model." STOC 2008.

slide-23
SLIDE 23

Submodular Welfare problem

23

Jan Vondrák. "Optimal approximation for the submodular welfare problem in the value oracle model." STOC 2008.

slide-24
SLIDE 24

Submodular function over integer lattice

  • Integer lattice: vector
  • Submodular function satisfies:
  • Greedy algorithm provides (1-1/e)-approx. for

the cardinality constraint (ICML 2014)

  • Recent paper:
  • Consider cardinality, matriod and knapsack

constraint

24

  • 1. Soma, Tasuku, and Yuichi Yoshida. "Maximizing Submodular Functions with

the Diminishing Return Property over the Integer Lattice." arXiv preprint arXiv:1503.01218 (2015)

  • 2. Soma, Tasuku, Naonori Kakimura, Kazuhiro Inaba, and Ken-ichi
  • Kawarabayashi. "Optimal budget allocation: Theoretical guarantee and efficient

algorithm." ICML 2014.

slide-25
SLIDE 25

Distributed Submodular Maximization I

  • Greedy Algorithm for the submodular function

25

Mirzasoleiman, Baharan, Amin Karbasi, Rik Sarkar, and Andreas Krause. "Distributed submodular maximization: Identifying representative elements in massive data." NIPS 2013.

slide-26
SLIDE 26

Distributed Submodular Maximization II

  • Summary of Greedy Algorithm in MapReduce

26

Kumar, Ravi, Benjamin Moseley, Sergei Vassilvitskii, and Andrea Vattani. "Fast greedy algorithms in mapreduce and streaming." SPAA 2013

SPAA 2013 best paper

slide-27
SLIDE 27

Streaming Submodular Maximization

  • Assume elements set V is ordered
  • any streaming algorithm must process V in the given order
  • At each iteration t
  • the algorithm maintains a memory of subset of elements Mt

points;

  • and must be ready to output a candidate feasible solution St
  • When a new point arrives, the algorithm may select

to remember it, and discard previous elements

27

Badanidiyuru, Ashwinkumar, Baharan Mirzasoleiman, Amin Karbasi, and Andreas Krause. "Streaming submodular maximization: Massive data summarization on the fly." KDD 2014.

slide-28
SLIDE 28

Streaming Submodular Maximization

28

  • Simple example
  • our memory can only store one element
  • Algorithm:

Badanidiyuru, Ashwinkumar, Baharan Mirzasoleiman, Amin Karbasi, and Andreas Krause. "Streaming submodular maximization: Massive data summarization on the fly." KDD 2014.

slide-29
SLIDE 29

Submodular knapsack constraint

  • Two types of problem
  • Submodular Cost Submodular Cover (SCSC)
  • Submodular Cost Submodular Knapsack

(SCSK)

  • Both f and g are submodular function

29

Iyer, Rishabh K., and Jeff A. Bilmes. "Submodular optimization with submodular cover and submodular knapsack constraints." NIPS 2013.

slide-30
SLIDE 30

References

1. http://submodularity.org/ 2. Krause, Andreas, and Daniel Golovin. "Submodular function maximization." Tractability: Practical Approaches to Hard Problems 3 (2012): 19. 3. Mirzasoleiman, Baharan, Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondrák, and Andreas Krause. "Lazier Than Lazy Greedy." AAAI 2015. 4. Badanidiyuru, Ashwinkumar, and Jan Vondrák. "Fast algorithms for maximizing submodular functions." SODA2014. 5. Jan Vondrák. "Optimal approximation for the submodular welfare problem in the value oracle model." STOC 2008. 6. Soma, Tasuku, and Yuichi Yoshida. "Maximizing Submodular Functions with the Diminishing Return Property over the Integer Lattice." arXiv preprint arXiv:1503.01218 (2015) 7. Soma, Tasuku, Naonori Kakimura, Kazuhiro Inaba, and Ken-ichi

  • Kawarabayashi. "Optimal budget allocation: Theoretical guarantee

and efficient algorithm." ICML 2014.

30

slide-31
SLIDE 31

References

8. Mirzasoleiman, Baharan, Amin Karbasi, Rik Sarkar, and Andreas

  • Krause. "Distributed submodular maximization: Identifying

representative elements in massive data." NIPS 2013. 9. Kumar, Ravi, Benjamin Moseley, Sergei Vassilvitskii, and Andrea

  • Vattani. "Fast greedy algorithms in mapreduce and streaming."

SPAA 2013.

  • 10. Badanidiyuru, Ashwinkumar, Baharan Mirzasoleiman, Amin

Karbasi, and Andreas Krause. "Streaming submodular maximization: Massive data summarization on the fly." KDD 2014.

  • 11. Iyer, Rishabh K., and Jeff A. Bilmes. "Submodular optimization with

submodular cover and submodular knapsack constraints." NIPS 2013.

31