Linear Programming II Lecture 19 October 29, 2018 1/26 1 LP - PowerPoint PPT Presentation

CS 473: Algorithms, Fall 2018 Linear Programming II Lecture 19 October 29, 2018 1/26 1

LP feasibility... Clicker question 2/26 5 equality constraints). with the correct value (one need two variables because of the 4 variable with the appropriate value. 2 3 changing the value of one of the variables. 2 1 Then we have the following: Let L be an instance of LP with n variables and m constraints. L is always feasible. L might not be feasible, but it can be made feasible by L might not be feasible, but can be fjxed by adding a single L might not be feasible, but can be fjxed by adding two variable L might not be feasible, and this can not be fjxed.

19.1: The Simplex Algorithm in Detail 3

Simplex algorithm (*) 4/26 return “No solution” 4 Simplex ( � L a LP ) Transform � L into slack form. Let L be the resulting slack form. L ′ ← Feasible ( L ) x ← LPStartSolution ( L ′ ) x ′ ← SimplexInner ( L ′ , x ) z ← objective function value of x ′ if z > 0 then x ′′ ← SimplexInner ( L , x ′ ) return x ′′

Simplex algorithm... 1 5/26 feasible solution (all nonbasic vars assigned value 0 ). need to describe SimplexInner : solve LP in slack form given a 9 Use solution in SimplexInner on L . 8 7 6 5 4 5 3 2 zero to all the nonbasic variables is feasible. SimplexInner : solves a LP if the trivial solution of assigning L ′ = Feasible ( L ) returns a new LP with feasible solution. Done by adding new variable x 0 to each equality. Set target function in L ′ to min x 0 . ⇒ LP L ′ has feasible solution with original LP L feasible ⇐ x 0 = 0 . Apply SimplexInner to L ′ and solution computed (for L ′ ) by LPStartSolution ( L ′ ). If x 0 = 0 then have a feasible solution to L .

Notations B - Set of indices of basic variables N - Set of indices of nonbasic variables of inequalities) 6/26 6 n = | N | - number of original variables b , c - two vectors of constants m = | B | - number of basic variables (i.e., number A = { a ij } - The matrix of coeffjcients N ∪ B = { 1 , . . . , n + m } v - objective function constant. LP in slack form is specifjed by a tuple ( N , B , A , b , c , v ) .

s.t. The corresponding LP 7/26 7 � max z = v + c j x j , j ∈ N � x i = b i − a ij x j for i ∈ B , j ∈ N x i ≥ 0 , ∀ i = 1 , . . . , n + m .

Reminder - basic/nonbasic 8/26 8 Nonbasic variables z = 29 − 1 9 x 3 − 1 9 x 5 − 2 max 9 x 6 x 1 = 8 + 1 6 x 3 + 1 6 x 5 − 1 3 x 6 x 2 = 4 − 8 3 x 3 − 2 3 x 5 + 1 3 x 6 x 4 = 18 − 1 2 x 3 + 1 2 x 5 Basic variables

19.2: The SimplexInner Algorithm 9

The SimplexInner Algorithm x e : nonbasic variable with positive coeffjcient in objective 10/26 ... one of basic variables is going to vanish (i.e., become zero). 9 8 7 . j Description SimplexInner algorithm: 6 function. Formally: e is one of the indices of 5 3 1 LP is in slack form. 2 feasible. 10 objective value for this solution is v . 4 Trivial solution x = τ (i.e., all nonbasic variables zero), is Reminder: Objective function is z = v + � j ∈ N c j x j . � � � � � c j > 0 , j ∈ N x e is the entering variable (enters set of basic variables). If increase value x e (from current value of 0 in τ )...

Choosing the leaving variable set all nonbasic to 0 zero, except x e 11/26 9 8 7 1 6 11 5 How do we now which variable is x l ? 4 3 x l : leaving variable – vanishing basic variable. 2 x e : entering variable increase value of x e till x l becomes zero. x i = b i − a ie x e , for all i ∈ B . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . = ⇒ x e ≤ ( b i / a ie ) l = arg min i b i / a ie 10 If more than one achieves min i b i / a ie , just pick one.

Pivoting on x e ... a le 12/26 Alternatively, do Gaussian elimination remove any appearance of 5 4 Cleanup: remove all appearances (on right) in LP of x e . 3 a le a lj 1 12 2 (Every basic variable has an equation in the LP !) 2 1 Determined x e and x l . Rewrite equation for x l in LP . x l = b l − � j ∈ N a lj x j � ⇒ x e = b l − x j , where a ll = 1 . = j ∈ N ∪{ l } Substituting x e into the other equalities, using above. x e on right side LP (including objective). Transfer x l on the left side, to the right side.

Pivoting continued... which are non-basic. 13/26 n n Can do at most 9 ...because improve objective in each pivoting step. 8 variable) already visited. 1 7 Pivoting never returns to a combination (of basic/non-basic LP is completely defjned by which variables are basic, and 3 End of this process: have new equivalent LP . 2 6 13 4 End of this pivoting stage: LP objective function value increased. 5 Made progress. basic variables: B ′ = ( B \ { l } ) ∪ { e } non-basic variables: N ′ = ( N \ { e } ) ∪ { l } . � n + m � � n + m � n . ≤ · e 10 examples where 2 n pivoting steps are needed.

Simplex algorithm summary... 1 Each pivoting step takes polynomial time in n and m . 2 Running time of Simplex is exponential in the worst case. 3 In practice, Simplex is extremely fast. 14/26 14

Pivoting with zeroes? In an LP the constant in a constraint can never be zero, so this 15/26 Simplex might pivot in a loop forever. The pivoting step would not improve the LP objective function. 5 entering/leaving variables. If there is any problem, it can be solved by choosing a difgerent 4 is an impossible scenario. 3 Clicker question There is no problem. 2 Doing the pivoting step would involve division by zero, and as 1 15 Consider a pivoting step, with x e as the entering variable, and x ℓ as x ℓ = 0 − � the leaving variable, with the relevant constraint in the LP being: j ∈ N a lj x j . such the Simplex algorithm would fail.

Degeneracies 1 Simplex might get stuck if one of the b i s is zero. 2 same point. 3 Result: might not be able to make any progress at all in a pivoting step. 4 Solution I: add tiny random noise to each coeffjcient. Can be done symbolically. Intuitively, the degeneracy, being a local phenomena on the polytope disappears with high probability. 16/26 16 More than > m hyperplanes (i.e., equalities) passes through the

Degeneracies – cycling 1 Might get into cycling: a sequence of pivoting operations that do not improve the objective function, and the bases you get are cyclic (i.e., infjnite loop). 2 Solution II: Bland’s rule . Always choose the lowest index variable for entering and leaving out of the possible candidates. (Not prove why this work - but it does.) 17/26 17

19.2.1: Correctness of linear programming 18

Correctness of LP unbounded. 19/26 Proof: is constructive by running the simplex algorithm. If an optimal solution exists, then a basic optimal solution exists. 3 If a feasible solution exists, then a basic feasible solution exists. 2 If there is no optimal solution, the problem is either infeasible or Defjnition 1 For an arbitrary linear program, the following statements are true: Theorem Simplex algorithm deals only with basic solutions. the nonbasic variables to zero. A solution to an LP is a basic solution if it the result of setting all 19

19.2.2: On the ellipsoid method and interior point methods 20

On the ellipsoid method and interior point methods 5 21/26 algorithm for linear programming? BIG OPEN QUESTION: Is there strongly polynomial time 7 simplex method. Result in arm race between the interior-point method and the 6 Also weakly polynomial. Quite useful in practice. interior-point method . 1 In 1984, Karmakar came up with a difgerent method, called the 4 Useless in practice. Khachian in 1979 came up with it. 3 It is polynomial in the number of bits of the input. ellipsoid method is weakly polynomial. 2 Simplex has exponential running time in the worst case. 21

Solving LPs without ever getting into a loop - symbolic perturbations Details in the class notes. 22/26 22

Notes 23/26 23

Notes 24/26 24

Notes 25/26 25

Notes 26/26 26