Linear Programming Lecturer: Shi Li Department of Computer Science - PowerPoint PPT Presentation

CSE 632: Analysis of Algorithms II: Combinatorial Optimization and Linear Programming (Fall 2020) Linear Programming Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo

Outline Linear Programming 1 Introduction Linear Programming Duality 2 2/28

Outline Linear Programming 1 Introduction Linear Programming Duality 2 3/28

Example of Linear Programming x 2 min 7 x 1 + 4 x 2 9 x 1 + x 2 ≥ 5 8 x 1 + 2 x 2 ≥ 6 7 4 x 1 + x 2 ≥ 8 6 x 1 , x 2 ≥ 0 5 4 x 1 + x 2 ≥ 8 Feasible Region 4 x 1 + x 2 ≥ 5 3 optimum point: 2 x 1 = 1 , x 2 = 4 x 1 + 2 x 2 ≥ 6 1 value = 7 × 1 + 4 × 4 = 23 x 1 0 1 2 3 4 5 6 7 4/28

Standard Form of Linear Programming min c 1 x 1 + c 2 x 2 + · · · + c n x n s.t. � A 1 , 1 x 1 + A 1 , 2 x 2 + · · · + A 1 ,n x n ≥ b 1 � A 2 , 1 x 1 + A 2 , 2 x 2 + · · · + A 2 ,n x n ≥ b 2 . . . . . . . . . . . . � A m, 1 x 1 + A m, 2 x 2 + · · · + A m,n x n ≥ b m x 1 , x 2 , · · · , x n ≥ 0 5/28

Standard Form of Linear Programming     x 1 c 1 x 2 c 2     Let x =  , c =  ,  .   .  . .     . .   x n c n     A 1 , 1 A 1 , 2 · · · A 1 ,n b 1 A 2 , 1 A 2 , 2 · · · A 2 ,n b 2     A =  , b =  .  . . . .   .  . . . . .     . . . . .   A m, 1 A m, 2 · · · A m,n b m c T x Then, LP becomes min s.t. Ax ≥ b x ≥ 0 ≥ means coordinate-wise greater than or equal to 6/28

Standard Form of Linear Programming c T x min s.t. Ax ≥ b x ≥ 0 Linear programmings can be solved in polynomial time Algorithm Theory Practice Simplex Method Exponential Time Works Well Ellipsoid Method Polynomial Time Slow Internal Point Methods Polynomial Time Works Well 7/28

Applications of Linear Programming Design polynomial-time exact algorithms Design polynomial-time approximation algorithms Branch-and-bound algorithms to solve integer programmings 8/28

Brewery Problem (from Kevin Wayne’s Notes ∗ ) Small brewery produces ale and beer. Production limited by scarce resources: corn, hops, barley malt. Recipes for ale and beer require different proportions of resources. Corn Hops Malt Profit Beverage (pounds) (pounds) (pounds) ($) Ale (barrel) 5 4 35 13 Beer (barrel) 15 4 20 23 Constraint 480 160 1190 How can brewer maximize profits? ∗ http://www.cs.princeton.edu/~wayne/kleinberg-tardos/pdf/ LinearProgrammingI.pdf 9/28

Brewery Problem (from Kevin Wayne’s Notes ∗ ) Corn Hops Malt Profit Beverage (pounds) (pounds) (pounds) ($) Ale (barrel) 5 4 35 13 Beer (barrel) 15 4 20 23 Constraint 480 160 1190 Devote all resources to ale: 34 barrels of ale ⇒ $442 Devote all resources to beer: 32 barrels of beer ⇒ $736 7.5 barrels of ale, 29.5 barrels of beer ⇒ $776 12 barrels of ale, 28 barrels of beer ⇒ $800 ∗ http://www.cs.princeton.edu/~wayne/kleinberg-tardos/pdf/ LinearProgrammingI.pdf 10/28

Brewery Problem (from Kevin Wayne’s Notes ∗ ) Corn Hops Malt Profit Beverage (pounds) (pounds) (pounds) ($) Ale (barrel) 5 4 35 13 Beer (barrel) 15 4 20 23 Constraint 480 160 1190 max 13 A + 23 B profit 5 A + 15 B ≤ 480 Corn 4 A + 4 B ≤ 160 Hops 35 A + 20 B ≤ 1190 Malt A, B ≥ 0 ∗ http://www.cs.princeton.edu/~wayne/kleinberg-tardos/pdf/ 11/28 LinearProgrammingI.pdf

Recall: Network Flow Problem Abstraction of fluid flowing through edges Digraph G = ( V, E ) with source s ∈ V and sink t ∈ V No edges enter s No edges leave t Edge capacity c ( e ) ∈ R > 0 for every e ∈ E 12 a c 20 16 s t 7 4 9 13 4 b d 14 12/28

Def. An s - t flow is a function f : E → R such that for every e ∈ E : 0 ≤ f ( e ) ≤ c ( e ) (capacity conditions) for every v ∈ V \ { s, t } : � � f ( e ) = f ( e ) . (conservation conditions) e into v e out of v The value of a flow f is � val ( f ) = f ( e ) . e out of s Maximum Flow Problem Input: directed network G = ( V, E ) , capacity function c : E → R > 0 , source s ∈ V and sink t ∈ V Output: an s - t flow f in G with the maximum val ( f ) 13/28

Max-Flow is a Linear Program � max x e e ∈ δ out ( s ) x e ≤ c ( e ) ∀ e ∈ E � � x e = x e ∀ v ∈ V \ { s, t } e ∈ δ in ( v ) e ∈ δ out ( v ) x e ≥ 0 ∀ e ∈ E 14/28

s - t Shortest Path Input: (directed or undirected) graph G = ( V, E ) , s, t ∈ V w : E → R ≥ 0 Output: shortest path from s to t 5 a b 6 8 2 9 10 16 s c d 12 1 3 4 5 4 7 e f t 15/28

s - t Shortest Path Using Linear Programming max d t d s = 0 d v ≤ d u + w ( u, v ) ∀ ( u, v ) ∈ E � Lemma Let P be any s → t path. Then value of LP ≤ w e . e ∈ P Coro. value of LP ≤ dist ( s, t ) . Lemma Let d v be the length of the shortest path from s to v . Then ( d v ) v ∈ V satisfies all the constraints in LP. Lemma value of LP = dist ( s, t ) . 16/28

Weighted Interval Scheduling Input: n jobs, job i with start time s i and finish time f i each job has a weight (or value) v i > 0 i and j are compatible if [ s i , f i ) and [ s j , f j ) are disjoint Output: a maximum-weight subset of mutually compatible jobs 0 1 2 3 4 5 6 7 8 9 100 50 30 25 50 90 80 80 70 Optimum value = 220 17/28

Weighted Interval Scheduling Problem Integer Programming Linear Programming � � max x j w j max x j w j j ∈ [ n ] j ∈ [ n ] � � x j ≤ 1 ∀ t ∈ [ T ] x j ≤ 1 ∀ t ∈ [ T ] j ∈ [ n ]: t ∈ [ s j ,f j ) j ∈ [ n ]: t ∈ [ s j ,f j ) x j ∈ { 0 , 1 } ∀ j ∈ [ n ] x j ∈ [0 , 1] ∀ j ∈ [ n ] In general, integer programming is an NP-hard problem. Most optimization problems can be formulated as integer programming. However, the above IP is equivalent to the LP! 18/28

Approximation for Weighted Vertex Cover Def. Given a graph G = ( V, E ) , a vertex cover of G is a subset S ⊆ V such that for every ( u, v ) ∈ E then u ∈ S or v ∈ S . Weighted Vertex-Cover Problem Input: G = ( V, E ) with vertex weights { w v } v ∈ V Output: a vertex cover S with minimum � v ∈ S w v 19/28

Integer Programming for Weighted Vertex Cover For every v ∈ V , let x v ∈ { 0 , 1 } indicate whether we select v in the vertex cover S The integer programming for weighted vertex cover: � ( IP WVC ) min w v x v s.t. v ∈ V x u + x v ≥ 1 ∀ ( u, v ) ∈ E x v ∈ { 0 , 1 } ∀ v ∈ V ( IP WVC ) ⇔ weighted vertex cover Thus it is NP-hard to solve integer programmings in general 20/28

Integer programming for WVC: � ( IP WVC ) min w v x v s.t. v ∈ V x u + x v ≥ 1 ∀ ( u, v ) ∈ E x v ∈ { 0 , 1 } ∀ v ∈ V Linear programming relaxation for WVC: � ( LP WVC ) min w v x v s.t. v ∈ V x u + x v ≥ 1 ∀ ( u, v ) ∈ E x v ∈ [0 , 1] ∀ v ∈ V let IP = value of ( IP WVC ) , LP = value of ( LP WVC ) Then, LP ≤ IP We can show that IP ≤ 2 · LP ( LP WVC ) gives a 2 -approximation for the problem. 21/28

Outline Linear Programming 1 Introduction Linear Programming Duality 2 22/28

x 2 min 7 x 1 + 4 x 2 9 x 1 + x 2 ≥ 5 8 7 x 1 + 2 x 2 ≥ 6 6 4 x 1 + x 2 ≥ 8 5 4 x 1 + x 2 ≥ 8 Feasible Region x 1 , x 2 ≥ 0 4 x 1 + x 2 ≥ 5 3 optimum point: x 1 = 1 , x 2 = 4 2 value = 7 × 1 + 4 × 4 = 23 1 x 1 + 2 x 2 ≥ 6 x 1 0 1 2 3 4 5 6 7 Q: How can we prove a lower bound for the value? 7 x 1 + 4 x 2 ≥ 2( x 1 + x 2 ) + ( x 1 + 2 x 2 ) ≥ 2 × 5 + 6 = 16 7 x 1 + 4 x 2 ≥ ( x 1 + 2 x 2 ) + 1 . 5(4 x 1 + x 2 ) ≥ 6 + 1 . 5 × 8 = 18 7 x 1 +4 x 2 ≥ ( x 1 + x 2 )+( x 1 +2 x 2 )+(4 x 1 + x 2 ) ≥ 5+6+8 = 19 7 x 1 + 4 x 2 ≥ 4( x 1 + x 2 ) ≥ 4 × 5 = 20 23/28 7 x + 4 x ≥ 3( x + x ) + (4 x + x ) ≥ 3 × 5 + 8 = 23

Primal LP Dual LP min 7 x 1 + 4 x 2 max 5 y 1 +6 y 2 +8 y 3 s.t. x 1 + x 2 ≥ 5 y 1 + y 2 + 4 y 3 ≤ 7 x 1 + 2 x 2 ≥ 6 y 1 + 2 y 2 + y 3 ≤ 4 4 x 1 + x 2 ≥ 8 y 1 , y 2 ≥ 0 x 1 , x 2 ≥ 0 A way to prove lower bound on the value of primal LP 7 x 1 + 4 x 2 ( if 7 ≥ y 1 + y 2 + 4 y 3 and 4 ≥ y 1 + 2 y 2 + y 3 ) ≥ y 1 ( x 1 + x 2 ) + y 2 ( x 1 + 2 x 2 ) + y 3 (4 x 1 + x 2 ) ( if y 1 , y 2 , y 3 ≥ 0) ≥ 5 y 1 + 6 y 2 + 8 y 3 . Goal: need to maximize 5 y 1 + 6 y 2 + 8 y 3 24/28

Primal LP Dual LP min 7 x 1 + 4 x 2 max 5 y 1 +6 y 2 +8 y 3 s.t. x 1 + x 2 ≥ 5 y 1 + y 2 + 4 y 3 ≤ 7 x 1 + 2 x 2 ≥ 6 y 1 + 2 y 2 + y 3 ≤ 4 4 x 1 + x 2 ≥ 8 y 1 , y 2 ≥ 0 x 1 , x 2 ≥ 0 � 7     1 1 5 � A = 1 2 b = 6 c =     4 4 1 8 c T x b T y min s.t. max s.t. A T y ≤ c Ax ≥ b x ≥ 0 y ≥ 0 25/28

Primal LP Dual LP c T x b T y min s.t. max s.t. A T y ≤ c Ax ≥ b x ≥ 0 y ≥ 0 P = value of primal LP D = value of dual LP Theorem (weak duality theorem) D ≤ P . Theorem (strong duality theorem) D = P . Can always prove the optimality of the primal solution, by adding up primal constraints. 26/28

Example Primal LP Dual LP min 5 x 1 + 6 x 2 + x 3 s.t. max 2 y 1 + 5 y 2 + 7 y 3 s.t. 2 x 1 + 5 x 2 − 3 x 3 ≥ 2 2 y 1 + 3 y 2 + y 3 ≤ 5 3 x 1 − 2 x 2 + x 3 ≥ 5 5 y 1 − 2 y 2 + 2 y 3 ≤ 6 x 1 + 2 x 2 + 3 x 3 ≥ 7 − 3 y 1 + y 2 + 3 y 3 ≥ 1 x 1 , x 2 , x 3 ≥ 0 y 1 , y 2 , y 3 ≥ 0 Primal Solution Dual Solution x 1 = 1 . 6 , x 2 = 0 . 6 y 1 = 1 , y 2 = 5 / 8 x 3 = 1 . 4 , value = 13 y 3 = 9 / 8 , value = 13 27/28