Minimizing Submodular Functions Satoru Iwata (RIMS, Kyoto - - PowerPoint PPT Presentation

Satoru Iwata (RIMS, Kyoto University)

Minimizing Submodular Functions

Outline

• Submodular Functions

Examples Discrete Convexity

• Submodular Function Minimization

Min-Max Theorem Combinatorial Algorithms

• Applications
• Conclusion

Submodular Functions

• Cut Capacity Functions
• Matroid Rank Functions
• Entropy Functions

) ( ) ( ) ( ) ( Y X f Y X f Y f X f ∪ + ∩ ≥ +

V Y X ⊆ ∀ ,

V

X

Y

R 2 : →

V

f

: V

Finite Set

Cut Capacity Function

Cut Capacity

∑

= } leaving : | ) ( { ) ( X a a c X κ X

s t

Max Flow Value＝Min Cut Capacity

) ( ≥ a c

Matroid Rank Functions

: ) ( ) ( | | ) ( , ρ ρ ρ ρ Y X Y X X X V X ≤ ⇒ ⊆ ≤ ⊆ ∀

] , [ rank ) ( X U A X = ρ

Matrix Rank Function Submodular

= A

X

V

U

Whitney (1935)

Entropy Functions

: ) (X h ) ( = φ h

X

Information Sources

≥

) ( ) ( ) ( ) ( Y X h Y X h Y h X h ∪ + ∩ ≥ +

Entropy of the Joint Distribution Conditional Mutual Information

Positive Definite Symmetric Matrices

) ( = φ f

= A

] [X A ] [ det log ) ( X A X f = X

) ( ) ( ) ( ) ( Y X f Y X f Y f X f ∪ + ∩ ≥ + Ky Fan’s Inequality Extension of the Hadamard Inequality

∏

∈

≤

V i ii

A A det

Discrete Convexity

) ( ) ( ) ( ) ( Y X f Y X f Y f X f ∪ + ∩ ≥ + V X

Y

Convex Function

x

y

Discrete Convexity

} , { b a

} , , { c b a V = } {b

} {a

} {c

} , { c b

φ

Lovász (1983)

f ˆ

f

f ˆ

： Linear Interpolation

： Convex ： Submodular

Murota (2003)

Discrete Convex Analysis

Submodular Function Minimization

X

) (X f

? } | ) ( min{ V Y Y f ⊆

Minimization Algorithm Evaluation Oracle Minimizer ) ( = φ f

Assumption:

Submodular Function Minimization

) (

8 7

n n O + γ

) log (

5

M n O γ ) log (

7

n n O γ

Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige (2000) Schrijver (2000) Iwata (2003) Fleischer, Iwata (2000)

) log ) ((

5 4

M n n O + γ

Orlin (2007)

) (

6 5

n n O + γ

Iwata (2002)

Fully Combinatorial Ellipsoid Method

Cunningham (1985) Iwata, Orlin (2009)

Base Polyhedra

Submodular Polyhedron

)} ( ) ( , , | { ) ( Y f Y x V Y x x f P

V

≤ ⊆ ∀ ∈ = R

∑

∈

=

Y v

v x Y x ) ( ) (

} | { R R → = V x

V

)} ( ) ( ), ( | { ) ( V f V x f P x x f B = ∈ =

Base Polyhedron

Greedy Algorithm

v

⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ )) ( ( )) ( ( )) ( ( ) ( ) ( ) ( 1 1 1 1 1 1

2 1 2 1 n n

v L f v L f v L f v y v y v y M M L O M M M O L

}) { ) ( ( )) ( ( ) ( v v L f v L f v y − − =

Edmonds (1970) Shapley (1971) Extreme Base

) (v L

) ( V v∈

: y

2

v

1

v

n

v

Theorem

Min-Max Theorem

)} ( | ) ( max{ ) ( min f B x V x Y f

V Y

∈ =

− ⊆

)} ( , min{ : ) ( v x v x =

−

) ( ) ( ) ( Y f Y x V x ≤ ≤

−

Edmonds (1970)

Combinatorial Approach

Extreme Base Convex Combination Cunningham (1985)

L L L y

x ∑

Λ ∈

= λ

) ( f B yL ∈

x

| ) ( | max X f M

V X⊆

=

) log (

6

nM M n O γ

Combinatorial Approach

T v v x T v v x ∉ ∀ ≥ ∈ ∀ ≤ , ) ( , ) (

:

L

y L

. ), ( ) ( Λ ∈ ∀ = L T f T yL

L L Ly

x ∑

Λ ∈

= λ

) ( ) ( T f T x = ∴

) ( ) ( ) ( T f T x V x = =

−

T

Extreme Base

: T

Minimizer

Distance Labeling

) ( : Λ ∈ → L V dL Ζ

u v

L

. , ) ( ) ( Λ ∈ ∀ = ⇒ ≤ L u d u x

L

L L Ly

x ∑

Λ ∈

= λ

). ( ) ( v d u d v u

L L L

≤ ⇒ p

. , , , 1 | ) ( ) ( | V u K L u d u d

K L

∈ ∀ Λ ∈ ∀ ≤ −

Labeling

u

:

L

y

Extreme Base

Distance Labeling

: , ) (

min

Y Y v k v d ∀ ∉ ⇒ ≥ . f

k

. ) ( ,

min

k v d V v = ∈ ∃

. 1 ) ( ,

min

− ≠ ∈ ∀ k v d V v

Gap of Level k < Minimizer of

v

k ≥

} | ) ( min{ : ) (

min

Λ ∈ = L u d u d

L

Distance Labeling

: , ) (

min

Y Y v k v d ∀ ∉ ⇒ ≥ . f

k < Minimizer of

v

k ≥

). ( ) ( ) ( ) ( T X f T X x T x T f ∪ ≤ ∪ < =

). ( ) ( T X f X f ∩ > ∴ ⇒ ≠ φ T X \

T

Iteration

n 4 : η δ =

. , | ) ( | such that Find V v v x ∈ ∀ > − δ μ μ

} ) ( , , | ) ( min{ : μ > Λ ∈ ∈ = u x L V u u d l

L

). ( such that and Select u d l L V u

L

= Λ ∈ ∈

η

μ

δ δ

} | ) ( max{ : V v v x ∈ = η

New Permutation

} ) ( , | { l v d V v v S

L

= ∈ = ) , , ( ation New_Permut l L μ

}, ) ( , | { μ > ∈ = v x S v v R R S Q \ =

⎩ ⎨ ⎧ ∈ + ∉ =

′

) ( 1 ) ( ) ( ) ( : ) ( R v v d R v v d v d

L L L

L

Q

R L′ η

μ

Push Operation

α λ λ − =

L L :

) , ( Push L L ′

⎭ ⎬ ⎫ ⎩ ⎨ ⎧ ≠ ∈ − − =

′ ′

) ( ) ( , | ) ( ) ( ) ( min : v y v y S v v y v y v x

L L L L

μ β

α λ =

′ : L

} , min{ : β λ α

L

=

L

λ α =

Saturating Nonsaturating

η

μ

β α =

Potential Function

∑

∈ +

= Φ

V v

v x x

2

) ( ) ( } ), ( max{ ) ( v x v x =

+

x x′

Nonsaturating Push Moves to

3

16 / ) ( ) ( ) ( n x x x Φ ≥ ′ Φ − Φ ⇒

Initially, . ) (

2

nM x ≤ Φ After Nonsaturating Pushes,

) log (

3

nM n O . / 1 ) (

2

n x < Φ . / 1 n < η } | ) ( max{ : V v v x ∈ = η

Algorithm Termination

} | ) ( max{ : V v v x ∈ = η

: 1 V n ⇒ < η

Maximal Minimizer

Z →

V

f 2 : . 1 ) ( 1 ) ( ) ( ) ( ) − = − > ≥

−

V f V x V x X f Q

) log ( | |

3

nM n O = Λ

Running Time Bound

). (Λ Γ

| | Λ

) (Λ Γ

A Saturating Push Decreases

.

2

n

Total Increase of

) log (

5

nM n O ) log (

6

nM n O γ

) (Λ Γ

∑∑

Λ ∈ ∈

− = Λ Γ

L V v L v

d n )] ( [ ) (

A Nonsaturating Push Increases by One and by at Most # Saturating Pushes

) log (

5

nM n O

Running Time

Improvements

• A Simple Algorithm
• A Faster Weakly Polynomial Algorithm
• A Strongly Polynomial Algorithm
• A Fully Combinatorial Algorithm

) log ) ((

8 7

n n n O + γ

) log (

6

nM n O γ ) log ) ((

5 4

nM n n O + γ ) log ) ((

6 5

n n n O + γ

) ( subject to Minimize

2

f B x x ∈

The Minimum-Norm Base

sol.

• pt.

:

∗

x } ) ( | { : < =

∗ −

v x v X } ) ( | { : ≤ =

∗ v

x v X

Minimal Minimizer Maximal Minimizer

∗

x

Theorem

Fujishige (1984)

The minimum-norm base minimizes in for any convex function Remark

∑

∈ V v

v x g )) ( ( ) ( f B

Nagano (2007) . g

Evacuation Problem （Dynamic Flow）

: ) (X

• :

) (a τ

Hoppe, Tardos (2000)

: ) (v b

: ) (a c S X ∩ X T \ T S X X

• X

b ∪ ⊆ ∀ ≤ ), ( ) ( T S

Capacity Transit Time Supply/Demand Maximum Amount of Flow from to . Feasible

Multiterminal Source Coding

X

X

R

Y

Encoder Encoder

Y

R

Decoder

) (Y H

) (X H

) | ( X Y H

) | ( Y X H ) , ( Y X H

) , ( Y X H

X

R

Y

R

Slepian, Wolf (1973)

Multiclass Queueing Systems

i

μ

i

λ

Server

Control Policy Arrival Rate Service Rate Multiclass M/M/1 Preemptive Arrival Interval Service Time

Performance Region

V X S

X i i X i i i X i i i

⊆ ∀ − ≥

∑ ∑ ∑

∈ ∈ ∈

, 1 ρ μ ρ ρ

: s

, :

i i i

μ λ ρ =

Y

R

j

s

i

s

:

j

s

Expected Staying Time of a Job in j Achievable

Coffman, Mitrani (1980)

: s

1 <

∑

∈V i i

ρ

A Class of Submodular Functions

V

z y x

+

∈R , ,

)) ( ( ) ( ) ( ) ( X x h X y X z X f − =

: h

V X S

X i i X i i i X i i i

⊆ ∀ − ≥

∑ ∑ ∑

∈ ∈ ∈

, 1 ρ μ ρ ρ

Nonnegative, Nondecreasing, Convex

) ( V X ⊆

i i i

y μ ρ = : x x h − = 1 1 : ) (

i i i

S z ρ = :

i i

x ρ = :

Submodular Itoko & I. (2005)

Zonotope in 3D

)) ( ), ( ), ( ( ) ( X z X y X x X w =

} | ) ( { conv V X X w Z ⊆ =

) ( ) , , ( ~ x yh z z y x f − =

Zonotope

}

• f

Point Extreme Lower : ) , , ( | ) , , ( ~ min{ } | ) ( min{ Z z y x z y x f V X X f = ⊆

Remark:

) , , ( ~ z y x f

is NOT concave!

z

Line Arrangement

β

i i i

z y x = + β α

α Topological Sweeping Method Edelsbrunner, Guibas (1989)

) (

2

n O

Enumerating All the Cells

Summary

• Submodular Functions Arise Everywhere.
• Discrete Analogue of Convexity.
• General SFM Algorithms Available.
• Exploit Special Structures of Problems.