Linear Programming Lecturer: Shi Li Department of Computer Science - PowerPoint PPT Presentation

CSE 431/531: Algorithm Analysis and Design (Spring 2018) Linear Programming Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo

Outline Linear Programming 1 Introduction Network Flow 2 Ford-Fulkerson Method Bipartite Matching Problem 3 2-Approximation for Weighted Vertex Cover 4 Linear Programming Duality 5 2/51

Outline Linear Programming 1 Introduction Network Flow 2 Ford-Fulkerson Method Bipartite Matching Problem 3 2-Approximation for Weighted Vertex Cover 4 Linear Programming Duality 5 3/51

Example of Linear Programming x 2 9 8 min 7 x 1 + 4 x 2 7 x 1 + x 2 ≥ 5 6 x 1 + 2 x 2 ≥ 6 4 x 1 + x 2 ≥ 8 5 4 x 1 + x 2 ≥ 8 Feasible Region 4 x 1 + x 2 ≥ 5 x 1 , x 2 ≥ 0 3 2 x 1 + 2 x 2 ≥ 6 1 optimum point: x 1 = 1 , x 2 = 4 x 1 0 1 2 3 4 5 6 7 value = 7 × 1 + 4 × 4 = 23 4/51

Standard Form of Linear Programming min c 1 x 1 + c 2 x 2 + · · · + c n x n s.t. � A 1 , 1 x 1 + A 1 , 2 x 2 + · · · + A 1 ,n x n ≥ b 1 � A 2 , 1 x 1 + A 2 , 2 x 2 + · · · + A 2 ,n x n ≥ b 2 . . . . . . . . . . . . � A m, 1 x 1 + A m, 2 x 2 + · · · + A m,n x n ≥ b m x 1 , x 2 , · · · , x n ≥ 0 5/51

Standard Form of Linear Programming     x 1 c 1 x 2 c 2     Let x =  , c =  ,  .   .  . .     . .   x n c n     A 1 , 1 A 1 , 2 · · · A 1 ,n b 1 A 2 , 1 A 2 , 2 · · · A 2 ,n b 2     A =  , b =  .  . . . .   .  . . . . .     . . . . .   A m, 1 A m, 2 · · · A m,n b m c T x Then, LP becomes min s.t. Ax ≥ b x ≥ 0 ≥ means coordinate-wise greater than or equal to 6/51

Standard Form of Linear Programming c T x min s.t. Ax ≥ b x ≥ 0 Linear programmings can be solved in polynomial time Algorithm Theory Practice Simplex Method Exponential Time Works Well Ellipsoid Method Polynomial Time Slow Internal Point Methods Polynomial Time Works Well 7/51

Applications of Linear Programming Design polynomial-time exact algorithms Design polynomial-time approximation algorithms Branch-and-bound algorithms to solve integer programmings 8/51

Brewery Problem (from Kevin Wayne’s Notes ∗ ) Small brewery produces ale and beer. Production limited by scarce resources: corn, hops, barley malt. Recipes for ale and beer require different proportions of resources. Corn Hops Malt Profit Beverage (pounds) (pounds) (pounds) ($) Ale (barrel) 5 4 35 13 Beer (barrel) 15 4 20 23 Constraint 480 160 1190 How can brewer maximize profits? ∗ http://www.cs.princeton.edu/~wayne/kleinberg-tardos/pdf/ LinearProgrammingI.pdf 9/51

Brewery Problem (from Kevin Wayne’s Notes ∗ ) Corn Hops Malt Profit Beverage (pounds) (pounds) (pounds) ($) Ale (barrel) 5 4 35 13 Beer (barrel) 15 4 20 23 Constraint 480 160 1190 Devote all resources to ale: 34 barrels of ale ⇒ $442 Devote all resources to beer: 32 barrels of beer ⇒ $736 7.5 barrels of ale, 29.5 barrels of beer ⇒ $776 12 barrels of ale, 28 barrels of beer ⇒ $800 ∗ http://www.cs.princeton.edu/~wayne/kleinberg-tardos/pdf/ LinearProgrammingI.pdf 10/51

Brewery Problem (from Kevin Wayne’s Notes ∗ ) Corn Hops Malt Profit Beverage (pounds) (pounds) (pounds) ($) Ale (barrel) 5 4 35 13 Beer (barrel) 15 4 20 23 Constraint 480 160 1190 max 13 A + 23 B profit 5 A + 15 B ≤ 480 Corn 4 A + 4 B ≤ 160 Hops 35 A + 20 B ≤ 1190 Malt A, B ≥ 0 ∗ http://www.cs.princeton.edu/~wayne/kleinberg-tardos/pdf/ 11/51 LinearProgrammingI.pdf

s - t Shortest Path Input: (directed or undirected) graph G = ( V, E ) , s, t ∈ V w : E → R ≥ 0 Output: shortest path from s to t 5 a b 6 2 8 9 16 10 s c d 12 1 3 4 5 4 7 e t f 12/51

s - t Shortest Path Using Linear Programming max d t d s = 0 d v ≤ d u + w ( u, v ) ∀ ( u, v ) ∈ E � Lemma Let P be any s → t path. Then value of LP ≤ w e . e ∈ P Coro. value of LP ≤ dist ( s, t ) . Lemma Let d v be the length of the shortest path from s to v . Then ( d v ) v ∈ V satisfies all the constraints in LP. Lemma value of LP = dist ( s, t ) . 13/51

Weighted Interval Scheduling Input: n jobs, job i with start time s i and finish time f i each job has a weight (or value) v i > 0 i and j are compatible if [ s i , f i ) and [ s j , f j ) are disjoint Output: a maximum-weight subset of mutually compatible jobs 0 1 2 3 4 5 6 7 8 9 100 50 30 25 50 90 80 80 70 14/51

Weighted Interval Scheduling Problem Integer Programming Linear Programming � � max x j w j max x j w j j ∈ [ n ] j ∈ [ n ] � � x j ≤ 1 ∀ t ∈ [ T ] x j ≤ 1 ∀ t ∈ [ T ] j ∈ [ n ]: t ∈ [ s j ,f j ) j ∈ [ n ]: t ∈ [ s j ,f j ) x j ∈ { 0 , 1 } ∀ j ∈ [ n ] x j ∈ [0 , 1] ∀ j ∈ [ n ] In general, integer programming is an NP-hard problem. Most optimization problems can be formulated as integer programming. However, the above IP is equivalent to the LP! 15/51

Outline Linear Programming 1 Introduction Network Flow 2 Ford-Fulkerson Method Bipartite Matching Problem 3 2-Approximation for Weighted Vertex Cover 4 Linear Programming Duality 5 16/51

Flow Network Abstraction of fluid flowing through edges Digraph G = ( V, E ) with source s ∈ V and sink t ∈ V No edges enter s No edges leave t Edge capacity c ( e ) ∈ R > 0 for every e ∈ E 12 a c 20 16 s t 7 4 9 13 4 b d 14 17/51

Def. An s - t flow is a function f : E → R such that for every e ∈ E : 0 ≤ f ( e ) ≤ c ( e ) (capacity conditions) for every v ∈ V \ { s, t } : � � f ( e ) = f ( e ) . (conservation conditions) e into v e out of v The value of a flow f is � val ( f ) = f ( e ) . e out of s Maximum Flow Problem Input: directed network G = ( V, E ) , capacity function c : E → R > 0 , source s ∈ V and sink t ∈ V Output: an s - t flow f in G with the maximum val ( f ) 18/51

Maximum Flow Problem: Example 12/12 a c 19/20 12/16 0/4 7/7 s t 0/9 11/13 4/4 b d 11/14 19/51

Linear Programming for Max-Flow � max x e e ∈ δ out ( s ) x e ≤ c ( e ) ∀ e ∈ E � � x e = x e ∀ v ∈ V \ { s, t } e ∈ δ in ( v ) e ∈ δ out ( v ) x e ≥ 0 ∀ e ∈ E 20/51

Outline Linear Programming 1 Introduction Network Flow 2 Ford-Fulkerson Method Bipartite Matching Problem 3 2-Approximation for Weighted Vertex Cover 4 Linear Programming Duality 5 21/51

Greedy Algorithm Start with empty flow: f ( e ) = 0 for every e ∈ E Define the residual capacity of e to be c ( e ) − f ( e ) Find an augmenting path: a path from s to t , where all edges have positive residual capacity Augment flow along the path as much as possible Repeat until we got stuck 22/51

Greedy Algorithm: Example 12/12 a c 19/20 12/16 0/4 7/7 s t 0/9 11/13 4/4 b d 11/14 23/51

Greedy Algorithm Does Not Always Give a Optimum Solution a 0/1 0/1 1/1 1/1 s t 0/1 0/1 1/1 b 24/51

Fix the Issue: Allowing “Undo” Flow Sent a 1/1 1/1 0/1 s t 1/1 1/1 b 25/51

Assumption ( u, v ) and ( v, u ) can not both be in E Def. For a s - t flow f , the residual graph G f of G = ( V, E ) w.r.t f contains: the vertex set V , for every e = ( u, v ) ∈ E with f ( e ) < c ( e ) , a forward edge e = ( u, v ) , with residual capacity c f ( e ) = c ( e ) − f ( e ) , for every e = ( u, v ) ∈ E with f ( e ) > 0 , a backward edge e ′ = ( v, u ) , with residual capacity c f ( e ′ ) = f ( e ) . a a 0/1 0/1 1/1 1 1 1/1 s t s t 1 0/1 0/1 1 1 1/1 b b Residual Graph G f Original graph G and f 26/51

Residual Graph: One More Example G f G s s 4/16 0 / 12 13 1 3 4 0/4 4 a b a b 9 / 4/12 4/14 4 5 0 4 8 4 4 1 0/7 7 c d c d 0 4/4 / 20 4 2 0 t t 27/51

Agumenting Path Augmenting the flow along a path P from s to t in G f Augment ( P ) b ← min e ∈ P c f ( e ) 1 for every ( u, v ) ∈ P 2 if ( u, v ) is a forward edge 3 f ( u, v ) ← f ( u, v ) + b 4 else \\ ( u, v ) is a backward edge 5 f ( v, u ) ← f ( v, u ) − b 6 return f 7 28/51

Example for Augmenting Along a Path a a 0/1 0/1 1/1 1 1 1 s t s t 1 / 1 0/1 0/1 1/1 1 1 b b 29/51

Ford-Fulkerson’s Method Ford-Fulkerson ( G, s, t, c ) let f ( e ) ← 0 for every e in G 1 while there is a path from s to t in G f 2 let P be any simple path from s to t in G f 3 f ← augment ( f, P ) 4 return f 5 30/51

Ford-Fulkerson: Example G f G s s 1 8/16 1 / 2 8 1 3 8 11 0/4 4 a b a b 9 / 8/12 11/14 0 9 1 4 8 3 1 7/7 7 c d c d 1 15 5 4/4 / 4 2 5 0 t t 31/51

Correctness of Ford-Fulkerson Method Flow conservation conditions are satisfied When algorithm terminates, there is a cut in the residual graph Running Time of Ford-Fulkerson Method Depends on #iterations #iterations could be exponential if augmenting paths are chosen by adversary #iterations=polynomial if in each iteration, we choose the shortest augmenting path, or the augmenting path with largest bottleneck capacity. 32/51

Outline Linear Programming 1 Introduction Network Flow 2 Ford-Fulkerson Method Bipartite Matching Problem 3 2-Approximation for Weighted Vertex Cover 4 Linear Programming Duality 5 33/51