( ) Outline Submodular - PowerPoint PPT Presentation

劣モジュラ最適化 岩田 覚 ( 京都大学数理解析研究所 )

Outline • Submodular Functions Examples Discrete Convexity • Minimizing Submodular Functions • Symmetric Submodular Functions • Maximizing Submodular Functions • Approximating Submodular Functions

Submodular Functions : V Finite Set → ∀ , ⊆ V : 2 R X Y V f + ≥ ∩ + ∪ ( ) ( ) ( ) ( ) f X f Y f X Y f X Y • Cut Capacity Functions • Matroid Rank Functions • Entropy Functions X Y V

Cut Capacity Function ∑ κ = Cut Capacity ( ) { ( ) | : leaving } X c a a X ≥ ( ) 0 c a t s X Max Flow Value ＝ Min Cut Capacity

) Y Whitney (1935) ( | ρ X ≤ Matroid Rank Functions | ≤ ) ) X Submodular X ( ρ ( ρ ⇒ , V Y ⊆ ⊆ X : ρ ∀ X ] U Matrix Rank Function X , U [ A rank V X X = ) = ( ρ A

Entropy Functions Information φ = ( ) 0 h Sources ( X ) : h Entropy of the Joint Distribution + ≥ ∩ + ∪ ( ) ( ) ( ) ( ) h X h Y h X Y h X Y X ≥ 0 Conditional Mutual Information

Positive Definite Symmetric Matrices X φ = ( ) 0 f = ( ) log det [ ] f X A X [ X ] = A A Ky Fan’s Inequality + ≥ ∩ + ∪ ( ) ( ) ( ) ( ) f X f Y f X Y f X Y Extension of the Hadamard Inequality ∏ ≤ det A A ii ∈ i V

Discrete Concavity ⊆ ⇒ S T ∪ − ≥ ∪ − ( { }) ( ) ( { }) ({ }) f S u f S f T u f u Diminishing Returns S T u V

) Y ∪ V X ( f + ) Y Y Discrete Convexity ∩ X ( f ≥ ) Y ( f + X ) X ( f Convex Function y x

Discrete Convexity Lovász (1983) V = { , , } a b c ˆ { c , } b ： Linear Interpolation f ˆ { , } a b f ： Convex { b } { c } f ： Submodular { a } φ Discrete Convex Analysis Murota (2003)

Submodular Function Minimization φ = ( ) 0 f Assumption: X Minimization Evaluation Algorithm Oracle ( X ) f ⊆ min{ ( ) | } ? f Y Y V Minimizer Ellipsoid Method Grötschel, Lovász, Schrijver (1981)

Base Polyhedra = → V { | } R x V R ∑ = ( ) ( ) x Y x v ∈ v Y Submodular Polyhedron = ∈ ∀ ⊆ ≤ V ( ) { | , , ( ) ( )} P f x x Y V x Y f Y R Base Polyhedron = ∈ = ( ) { | ( ), ( ) ( )} B f x x P f x V f V

Edmonds (1970) Greedy Algorithm Shapley (1971) ( v ) L v v v v 1 2 n v ∈ = − − ( ) V ( ) ( ( )) ( ( ) { }) y v f L v f L v v : y Extreme Base ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ L 1 0 0 ( ) ( ( )) y v f L v 1 1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ O M 1 1 ( ) ( ( )) y v f L v ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ = 2 2 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ M M O M M 0 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ L ⎣ ⎦ ( ) ( ( )) 1 1 1 ⎣ ⎦ ⎣ ⎦ y v f L v n n

Min-Max Theorem Edmonds (1970) Theorem − = ∈ min ( ) max{ ( ) | ( )} f Y x V x B f ⊆ Y V − = ( ) : min{ 0 , ( )} x v x v − ≤ ≤ ( ) ( ) ( ) x V x Y f Y

Combinatorial Approach y L ∈ ( f ) Extreme Base B Convex Combination x ∑ = λ L y L ∈ Λ x L Cunningham (1985) γ 6 ( log ) O n M nM = max | ( ) | M f X X ⊆ V

Submodular Function Minimization Grötschel, Lovász, Schrijver (1981, 1988) Ellipsoid Method γ 5 ( log ) Cunningham (1985) O n M γ + γ 7 8 7 ( ) O n n ( log ) O n n Iwata, Fleischer, Fujishige (2000) Schrijver (2000) Fleischer, Iwata (2000) Iwata (2002) Fully Combinatorial Iwata (2003) Orlin (2007) γ + γ + 4 5 5 6 ( ) (( ) log ) O n n O n n M Iwata, Orlin (2009)

The Minimum-Norm Base 2 Minimize x ∗ ∈ x subject to ( ) x B f Fujishige (1984) Theorem ∗ : opt. sol. x ∗ = < : { | ( ) 0 } X v x v Minimal Minimizer − ∗ v = ≤ : { | ( ) 0 } X v x Maximal Minimizer 0 Wolfe’s Algorithm Practically Efficient

Evacuation Problem （ Dynamic Flow ） T Hoppe, Tardos (2000) S ( a ) : c Capacity τ ( a ) : Transit Time ( v ) : b Supply/Demand X ∩ ( X ) : o T \ S X Maximum Amount of Flow from to . ≤ ∀ ⊆ ∪ ( ) ( ), b X o X X S T Feasible

X R ) Y Multiterminal Source Coding , X ( H ) ( X H ) Y | X ( H ) ) ) Y X Y ( Y R , X | H Y ( ( H H Decoder Slepian, Wolf (1973) X Y R R Encoder Encoder Y X

Multiclass Queueing Systems Service Time Arrival Interval Server μ λ Service Rate Arrival Rate i i Multiclass M/M/1 Control Policy Preemptive

Performance Region : s Expected Staying Time of a Job in j j ∑ ρ = λ μ ρ < : , 1 i i i i : s Achievable ∈ V i s j : s R Y ∑ ρ μ i i ∑ ρ ≥ ∀ ⊆ ∈ i X , S X V ∑ − ρ i i 1 ∈ i X i ∈ i X Coffman, Mitrani (1980) s i

A Class of Submodular Functions ∈ R Itoko & Iwata (2005) V , , x y z + : h Nonnegative, Nondecreasing, Convex = − X ⊆ ( ) ( ) ( ) ( ( )) f X z X y X h x X ( ) V Submodular ρ = = ρ ∑ i : y ρ μ : z S μ i i i i i i ∑ i ρ ≥ ∀ ⊆ ∈ i X , S X V ∑ 1 − ρ i i 1 = ρ = 1 : ( ) : ∈ x h x i X i − i i ∈ x i X

Zonotope in 3D z = ( ) ( ( ), ( ), ( )) w X x X y X z X = ⊆ conv { ( ) | } Z w X X V Zonotope ~ = − ( , , ) ( ) f x y z z yh x ⊆ min{ ( ) | } f X X V ~ = min{ ( , , ) | ( , , ) : Lower Extreme Point of } f x y z x y z Z ~ ( , , ) f x y z Remark: is NOT concave!

Line Arrangement β α + β = x y z i i i α Enumerating All the Cells Topological Sweeping Method 2 ( ) O n Edelsbrunner, Guibas (1989)

Symmetric Submodular Functions → V : 2 f R = ∀ ⊆ ( ) ( \ ), . f X f V X X V Symmetric Crossing Submodular ∩ ≠ φ ∪ ≠ ⇒ , X Y X Y V + ≥ ∪ + ∩ ( ) ( ) ( ) ( ) f X f Y f X Y f X Y Symmetric Submodular Function Minimization φ ≠ ⊂ ≠ min{ ( ) | , } ? f X X V X V

Maximum Adjacency Ordering • Minimum Cut Algorithm by MA-ordering Nagamochi & Ibaraki (1992) • Simpler Proofs Frank (1994), Stoer & Wagner (1997) • Symmetric Submodular Functions Queyranne (1998) • Alternative Proofs Fujishige (1998), Rizzi (2000)

Minimum Degree Ordering Nagamochi (2007) ISAAC’07, Sendai, Japan Finding the family of all extreme sets for symmetric crossing submodular functions 3 γ in time. ( ) O n Symmetric Submodular Function Minimization

Extreme Sets : f Symmetric Crossing Submodular Function : X Extreme Set > ∀ ⊂ φ ≠ ≠ ( ) ( ), : . f Z f X Z X Z X The family of all extreme sets forms a laminar. Y X Y X + ≥ + ( ) ( ) ( \ ) ( \ ) f X f Y f X Y f Y X

Flat Pair for Symmetric Submodular Functions ⊆ ≠ Flat Pair { , } ( ) u v V u v ≥ ∈ ( ) min{ ( ) | } , f X f x x X ∀ ⊆ ∩ = s.t. | { , } | 1 . X V X u v X v u { , } u v { , } u v Shrink into No Extreme Sets . u v a single vertex. Separate and

MD-Ordering for Symmetric Submodular Functions V = : { ,..., } v v MD-ordering 1 i i ∈ , ,..., − , v v v v V 1 2 1 n n X V i Each has minimum value of v j ∈ ( ) f j − 1 v \ . among v V V − 1 j = + ∪ ⊆ ( ) : ( ) ( ) ( \ ) f X f X f V X X V V i i i Symmetric, Crossing Submodular

MD-Ordering for Symmetric Submodular Functions The last two vertices of , v v − 1 n n an MD-ordering form a flat pair. Proof by Induction: { , } : V \ v − v f Flat Pair for on V 1 i n n i = n − 2 ,..., 1 , 0 . i = n − 2 i v v − 1 n n

Time Complexity 2 γ ( ) O n • Finding an MD-ordering in time. 3 γ ( ) O n • Finding all the extreme sets in time. • Minimizing symmetric submodular functions 3 γ ( ) O n in time.

Application to Clustering V X ,..., X Random variables 1 n V \ Partition into and V A A as independent as possible V \ A A ( ; ) Minimize I X A X \ A V φ ≠ A ≠ subject to V = − ( ; ) ( ) ( | ) I X X H X H X X A B B B A = + − ( ) ( ) ( , ) H X H X H X X A B A B

Application to Clustering Greedy Split k ∑ ( ) Minimize f V i = 1 i = ∪ ∪ subject to L V V V 1 k ∩ = φ ≠ ( ) V V i j i j − ( 2 1 ) k -Approximation

Submodurlar Function Maximization Approximation Algorithms Nemhauser, Wolsey, Fisher (1978) Monotone SF / Cardinality Constraint (1 - 1/e)-Approximation Feige, Mirrokni, Vondrák (FOCS 2007) Nonnegative SF 2/5-Approximation Vondrák (STOC 2008) Monotone SF / Matroid Constraint (1 - 1/e)-Approximation