Basics on Linear Programming Combinatorial Problem Solving (CPS) - PowerPoint PPT Presentation

Basics on Linear Programming Combinatorial Problem Solving (CPS) Javier Larrosa Albert Oliveras Enric Rodr´ ıguez-Carbonell May 6, 2020

Linear Programs (LP’s) A linear program is an optimization problem of the form ■ min c T x A 1 x ≤ b 1 A 2 x = b 2 A 3 x ≥ b 3 x ∈ R n c ∈ R n , b i ∈ R m i , A i ∈ R m i × n , i = 1 , 2 , 3 x is the vector of variables ■ c T x is the cost or objective function ■ A 1 x ≤ b 1 , A 2 x = b 2 and A 3 x ≥ b 3 are the constraints ■ 2 / 31

Linear Programs (LP’s) A linear program is an optimization problem of the form ■ min c T x A 1 x ≤ b 1 A 2 x = b 2 A 3 x ≥ b 3 x ∈ R n c ∈ R n , b i ∈ R m i , A i ∈ R m i × n , i = 1 , 2 , 3 x is the vector of variables ■ c T x is the cost or objective function ■ A 1 x ≤ b 1 , A 2 x = b 2 and A 3 x ≥ b 3 are the constraints ■ Example: ■ min x + y + z x + y = 3 0 ≤ x ≤ 2 0 ≤ y ≤ 2 2 / 31

Notes on the Definition of LP Solving minimization or maximization is equivalent: ■ max { f ( x ) | x ∈ S } = − min { − f ( x ) | x ∈ S } Satisfiability problems are a particular case: ■ take arbitrary cost function, e.g., c = 0 3 / 31

Equivalent Forms of LP’s (1) This form is not the most convenient for algorithms ■ WLOG we can transform such a problem as follows 1. Split = constraints into ≥ and ≤ constraints min c T x min c T x A 1 x ≤ b 1 A 1 x ≤ b 1 = ⇒ A 2 x ≤ b 2 A 2 x = b 2 A 2 x ≥ b 2 A 3 x ≥ b 3 A 3 x ≥ b 3 Now all constraints are ≤ or ≥ 4 / 31

Equivalent Forms of LP’s (2) Example of step 1.: min x + y + z min x + y + z x + y ≤ 3 x + y = 3 = ⇒ x + y ≥ 3 0 ≤ x ≤ 2 0 ≤ x ≤ 2 0 ≤ y ≤ 2 0 ≤ y ≤ 2 5 / 31

Equivalent Forms of LP’s (3) 2. Transform ≥ constraints into ≤ constraints by multiplying by -1 min c T x min c T x = ⇒ A 1 x ≤ b 1 A 1 x ≤ b 1 A 2 x ≥ b 2 − A 2 x ≤ − b 2 Now all constraints are ≤ 6 / 31

Equivalent Forms of LP’s (4) Example of step 2.: min x + y + z min x + y + z x + y ≤ 3 x + y ≤ 3 = ⇒ x + y ≥ 3 − x − y ≤ − 3 0 ≤ x ≤ 2 0 ≤ x ≤ 2 0 ≤ y ≤ 2 0 ≤ y ≤ 2 7 / 31

Equivalent Forms of LP’s (5) 3. Replace variables x by y − z , where y , z are vectors of fresh variables, and add constraints y ≥ 0 , z ≥ 0 min c T y − c T z min c T x = ⇒ Ay − Az ≤ b Ax ≤ b y, z ≥ 0 Now all constraints are ≤ and all variables have to be ≥ 0 8 / 31

Equivalent Forms of LP’s (6) Actually only needed for variables which are not already non-negative. (in the example, only z ) Example of step 3.: min x + y + u − v min x + y + z x + y ≤ 3 x + y ≤ 3 − x − y ≤ − 3 = ⇒ − x − y ≤ − 3 0 ≤ x ≤ 2 0 ≤ x ≤ 2 0 ≤ y ≤ 2 0 ≤ y ≤ 2 u, v ≥ 0 9 / 31

Equivalent Forms of LP’s (7) 4. Add a slack variable to each ≤ constraint to convert it into = min c T x min c T x = ⇒ Ax ≤ b Ax + s = b x ≥ 0 x, s ≥ 0 Now all constraints are = and all variables have to be ≥ 0 10 / 31

Equivalent Forms of LP’s (8) Example of step 4.: min x + y + u − v min x + y + u − v x + y ≤ 3 x + y + s 1 = 3 − x − y ≤ − 3 − x − y + s 2 = − 3 = ⇒ 0 ≤ x ≤ 2 x + s 3 = 2 0 ≤ y ≤ 2 y + s 4 = 2 u, v ≥ 0 x, y, u, v, s 1 , s 2 , s 3 , s 4 ≥ 0 11 / 31

Equivalent Forms of LP’s (9) Altogether: min x + y + u − v min x + y + z x + y + s 1 = 3 x + y = 3 − x − y + s 2 = − 3 = ⇒ 0 ≤ x ≤ 2 x + s 3 = 2 0 ≤ y ≤ 2 y + s 4 = 2 x, y, u, v, s 1 , s 2 , s 3 , s 4 ≥ 0 12 / 31

Equivalent Forms of LP’s (10) In the end we get a problem in standard form: ■ min c T x Ax = b x ≥ 0 c ∈ R n , b ∈ R m , A ∈ R m × n , n ≥ m, rank( A ) = m These transformations are not strictly necessary ■ (they increase no. of constraints and variables), but are convenient in a first formulation of the algorithms Often variables are identified with columns of the matrix, ■ and constraints are identified with rows 13 / 31

Methods for Solving LP’s Simplex algorithms ■ Interior-point algorithms ■ 14 / 31

Methods for Solving LP’s Simplex algorithms ■ Interior-point algorithms ■ 14 / 31

Basic Definitions (1) min c T x Ax = b x ≥ 0 Any vector x such that Ax = b is called a solution ■ A solution x satisfying x ≥ 0 is called a feasible solution ■ An LP with feasible solutions is called feasible; ■ otherwise it is called infeasible A feasible solution x ∗ is called optimal ■ if c T x ∗ ≤ c T x for all feasible solution x A feasible LP with no optimal solution is unbounded ■ 15 / 31

Basic Definitions (2) max x + 2 y x + y + s 1 = 3 x + s 2 = 2 y + s 3 = 2 x, y, s 1 , s 2 , s 3 ≥ 0 ( x, y, s 1 , s 2 , s 3 ) = ( − 1 , − 1 , 5 , 3 , 3) is a solution but not feasible ■ ( x, y, s 1 , s 2 , s 3 ) = (1 , 1 , 1 , 1 , 1) is a feasible solution ■ 16 / 31

Basic Definitions (3) max x + βy x + y + s 1 = α x + s 2 = 2 y + s 3 = 2 x, y, s 1 , s 2 , s 3 ≥ 0 If α = − 1 the LP is not feasible ■ If α = 3 , β = 2 then ■ ( x, y, s 1 , s 2 , s 3 ) = (1 , 2 , 0 , 1 , 0) is the (only) optimal solution 17 / 31

Basic Definitions (3) max x + βy x + y + s 1 = α x + s 2 = 2 y + s 3 = 2 x, y, s 1 , s 2 , s 3 ≥ 0 If α = − 1 the LP is not feasible ■ If α = 3 , β = 2 then ■ ( x, y, s 1 , s 2 , s 3 ) = (1 , 2 , 0 , 1 , 0) is the (only) optimal solution There may be more than one optimal solution: ■ If α = 3 and β = 1 then { (1 , 2 , 0 , 1 , 0) , (2 , 1 , 0 , 0 , 1) , ( 3 2 , 3 2 , 0 , 1 2 , 1 2 ) } are optimal 17 / 31

Basic Definitions (4) y max x + 2 y max x + y y ≤ 2 (1 , 2) x + y ≤ 3 (2 , 1) x ≥ 0 x ≤ 2 y ≥ 0 x 18 / 31

Basic Definitions (5) y y ≤ 2 x + y ≤ 3 min x + 2 y x + y ≤ 3 0 ≤ x ≤ 2 y ≤ 2 x ≥ 0 x ≤ 2 x Unbounded LP min x + 2 y 19 / 31

Basic Definitions (6) y max x + 2 y max x + 2 y y ≤ 2 x + y ≤ 3 x + y ≤ 3 0 ≤ x ≤ 2 (1 , 2) y ≤ 2 LP is bounded, x ≥ 0 x ≤ 2 but set of feasible x solutions is not 20 / 31

Bases (1) Let us denote by a 1 , ..., a n the columns of A Recall that n ≥ m, rank( A ) = m . A matrix of m columns ( a k 1 , ..., a k m ) is a basis ■ if the columns are linearly independent Note that a basis is a square matrix! ■ If ( a k 1 , ..., a k m ) is a basis, ■ then the variables ( x k 1 , ..., x k m ) are called basic We usually denote ■ by B the list of indices ( k 1 , ..., k m ) , and by R the list of indices (1 , 2 , ..., n ) − B ; and by B the matrix ( a i | i ∈ B ) , and by R the matrix ( a i | i ∈ R ) x B the basic variables, x R the non-basic ones 21 / 31

Bases (2) max x + 2 y x y s 1 s 2 s 3 x + y + s 1 = 3 x + s 2 = 2   1 1 1 0 0 y + s 3 = 2 A = 1 0 0 1 0   x, y, s 1 , s 2 , s 3 ≥ 0 0 1 0 0 1 ( x, s 1 , s 2 ) do not form a basis: ■   1 1 0  does not have linearly independent columns 1 0 1  0 0 0 ( s 1 , s 2 , s 3 ) form a basis, where x B = ( s 1 , s 2 , s 3 ) , x R = ( x, y ) ■     1 0 0 1 1 0 1 0 1 0 B = R =     0 0 1 0 1 22 / 31

Bases (3) If B is a basis, then the following holds ■ Bx B + Rx R = b Hence: x B = B − 1 b − B − 1 Rx R Non-basic variables determine values of basic ones If non-basic variables are set to 0 , we get the solution ■ x R = 0 , x B = B − 1 b Such a solution is called a basic solution If a basic solution satisfies x B ≥ 0 then it is called a ■ basic feasible solution, and the basis is feasible 23 / 31

Bases (4) Consider basis ( s 1 , s 2 , s 3 ) max x + 2 y     x + y + s 1 = 3 1 0 0 1 1 x + s 2 = 2 B = 0 1 0 R = 1 0     y + s 3 = 2 0 0 1 0 1 x, y, s 1 , s 2 , s 3 ≥ 0 Equations x B = B − 1 b − B − 1 Rx R are  s 1 = 3 − x − y  s 2 = 2 − x s 3 = 2 − y  Basic solution is � 0   3 � σ B = 2 σ R =   0 2 So basis ( s 1 , s 2 , s 3 ) is feasible 24 / 31

Bases (5) Basis ( x, y, s 1 ) is not feasible max x + 2 y     x + y + s 1 = 3 1 1 1 0 0 x + s 2 = 2 B = 1 0 0 R = 1 0     y + s 3 = 2 0 1 0 0 1 x, y, s 1 , s 2 , s 3 ≥ 0 � 0    x = 2 − s 2 2 �  y = 2 − s 3 σ B = 2 σ R =   0 s 1 = − 1 + s 2 + s 3 − 1  25 / 31

Bases (6) A basis is called degenerate when at least one component of its basic solution x B is null For example: max x + 2 y     x + y + s 1 = 4 1 1 0 1 0 x + s 2 = 2 B = 1 0 1 R = 0 0     y + s 3 = 2 0 1 0 0 1 x, y, s 1 , s 2 , s 3 ≥ 0    x = 2 + s 3 − s 1 2  y = 2 − s 3 σ B = 2   s 2 = s 1 − s 3 0  26 / 31

Geometry of LP’s (1) Set of feasible solutions of an LP is a convex polyhedron ■ Basic feasible solutions are vertices of the convex polyhedron ■ 27 / 31

Geometry of LP’s (2) max x + 2 y x + y + s 1 = 3 y x + s 2 = 2 ■ y + s 3 = 2 y ≤ 2 (0 , 2) (1 , 2) x, y, s 1 , s 2 , s 3 ≥ 0 1 2 x + y ≤ 3 x B 1 = ( y, s 1 , s 2 ) ■ 3 (2 , 1) x B 2 = ( x, y, s 2 ) ■ x ≥ 0 x ≤ 2 x ≤ 2 x ≤ 2 x B 3 = ( x, y, s 3 ) ■ 5 4 y ≥ 0 x (0 , 0) (2 , 0) x B 4 = ( x, s 1 , s 3 ) ■ x B 5 = ( s 1 , s 2 , s 3 ) ■ 28 / 31