Revised Simplex Method Marco Chiarandini Department of Mathematics - PowerPoint PPT Presentation

DM545 Linear and Integer Programming Lecture 7 Revised Simplex Method Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

Revised Simplex Method Outline Efficiency Issues 1. Revised Simplex Method 2. Efficiency Issues 2

Revised Simplex Method Motivation Efficiency Issues Complexity of single pivot operation in standard simplex: • entering variable O ( n ) • leaving variable O ( m ) • updating the tableau O ( mn ) Problems with this: • Time: we are doing operations that are not actually needed Space: we need to store the whole tableau: O ( mn ) floating point numbers • Most problems have sparse matrices (many zeros) sparse matrices are typically handled efficiently the standard simplex has the ’Fill in’ effect: sparse matrices are lost • accumulation of Floating Point Errors over the iterations 3

Revised Simplex Method Outline Efficiency Issues 1. Revised Simplex Method 2. Efficiency Issues 4

Revised Simplex Method Revised Simplex Method Efficiency Issues Several ways to improve wrt pitfalls in the previous slide, requires matrix description of the simplex. n max c T x max { c T x | A x = b , x ≥ 0 } � max c j x j A x = b j = 1 n x ≥ 0 � a ij x j ≤ b i i = 1 .. m A ∈ R m × ( n + m ) j = 1 c ∈ R ( n + m ) , b ∈ R m , x ∈ R n + m x j ≥ 0 j = 1 .. n At each iteration the simplex moves from a basic feasible solution to another. For each basic feasible solution: • x N = 0 • B = { 1 . . . m } basis • x B ≥ 0 • N = { m + 1 . . . m + n } • A B = [ a 1 . . . a m ] basis matrix • A N = [ a m + 1 . . . a m + n ] 5

Revised Simplex Method Efficiency Issues       A N A B 0 b           c T c T 1 0 N B A x = A N x N + A B x B = b A B x B = b − A N x N Theorem Basic feasible solution ⇐ ⇒ A B is non-singular x B = A − 1 B b − A − 1 B A N x N 6

for the objective function: z = c T x = c T B x B + c T N x N Substituting for x B from above: B ( A − 1 B b − A − 1 z = c T B A N x N ) + c T N x N = = c T B A − 1 B b + ( c T N − c T B A − 1 B A N ) x N Collecting together: x B = A − 1 B b − A − 1 B A N x N B A − 1 B A − 1 z = c T B b + ( c T N − c T B A N ) x N � �� � ¯ A In tableau form, for a basic feasible solution corresponding to B we have:      A − 1 A − 1  B A N I 0 B b We do not need to compute     all elements of ¯   A     B A − 1 B A − 1 c T N − c T − c T B A N 0 1 B b

Revised Simplex Method Example Efficiency Issues max x 1 + x 2 max x 1 + x 2 − x 1 + x 2 ≤ 1 − x 1 + x 2 + x 3 = 1 x 1 ≤ 3 x 1 + x 4 = 3 x 2 ≤ 2 x 2 + x 5 = 2 x 1 , x 2 ≥ 0 x 1 , x 2 , x 3 , x 4 , x 5 ≥ 0 Initial tableau After two iterations x 1 x 2 x 3 x 4 x 5 − z x 1 x 2 x 3 x 4 x 5 − z b b − 1 1 1 0 0 0 1 1 0 − 1 0 1 0 1 1 0 0 1 0 0 3 0 1 0 0 1 0 2 0 1 0 0 1 0 2 0 0 1 1 − 1 0 2 1 1 0 0 0 1 0 0 0 1 0 − 2 1 3 Basic variables x 1 , x 2 , x 4 . Non basic: x 3 , x 5 . From the initial tableau:       − 1 1 0 1 0 x 1 � � x 3 A B = 1 0 1 A N = 0 0 x B = x 2 x N =       x 5 0 1 0 0 1 x 4 � 1 1 0 � � 0 0 � c T c T B = N = 8

Revised Simplex Method Efficiency Issues • Entering variable : B A − 1 in std. we look at tableau, in revised we need to compute: c T N − c T B A N 1. find y T = c T B A − 1 (by solving y T A B = c T B , the latter can be done more efficiently) B 2. calculate c T N − y T A N 9

Revised Simplex Method Efficiency Issues Step 1:   − 1 1 0 � y 1 y 2 y 3 � � 1 1 0 �  = y T A B = c T 1 0 1  B 0 1 0   − 1 0 1 � 1 1 0 � � − 1 0 2 �  = c T B A − 1 = y T 0 0 1  B 1 1 − 1 Step 2:   1 0 � 0 0 � � − 1 0 2 � � 1 − 2 �  = c T N − y T A N − 0 0  0 1 (Note that they can be computed individually: c j − y T a j > 0) Let’s take the first we encounter x 3 10

• Leaving variable we increase variable by largest feasible amount θ R1: x 1 − x 3 + x 5 = 1 x 1 = 1 + x 3 ≥ 0 R2: x 2 + 0 x 3 + x 5 = 2 x 2 = 2 ≥ 0 R3: − x 3 + x 4 − x 5 = 2 x 4 = 2 − x 3 ≥ 0 B − A − 1 x B = x ∗ B A N x N d is the column of A − 1 B A N that corresponds to x B = x ∗ B − d θ the entering variable, ie, d = A − 1 B a where a is the entering column 3. Find θ such that x B stays positive: Find d = A − 1 B a (by solving A B d = a ) Step 3:             d 1 − 1 0 1 1 − 1 1 − 1  =  =  =  −  θ ≥ 0 d 2 0 0 1 0 ⇒ d = 0 ⇒ x B = 2 0        d 3 1 1 − 1 0 1 2 1 2 − θ ≥ 0 = ⇒ θ ≤ 2 � x 4 leaves

Revised Simplex Method Efficiency Issues • So far we have done computations, but now we save the pivoting update. The update of A B is done by replacing the leaving column by the entering column       x 1 − d 1 θ 3 − 1 1 1  = x ∗ B = x 2 − d 2 θ 2 A B = 1 0 0      2 0 1 0 θ • Many implementations depending on how y T A B = c T B and A B d = a are solved. They are in fact solved from scratch. • many operations saved especially if many variables! • special ways to call the matrix A from memory • better control over numerical issues since A − 1 can be recomputed. B 12

Revised Simplex Method Outline Efficiency Issues 1. Revised Simplex Method 2. Efficiency Issues 13

Revised Simplex Method Solving the two Systems of Equations Efficiency Issues A B x = b solved without computing A − 1 B (costly and likely to introduce numerical inaccuracy) Recall how the inverse is computed: For a 2 × 2 matrix the matrix inverse is � d � d � T � a b � � A − 1 = 1 − c 1 − b A = = c d | A | − b a ad − bc − c a For a 3 × 3 matrix the matrix inverse is T   a 11 a 12 a 13 � � � � � �  a 22 a 23 a 21 a 23 a 21 a 22  � � � � � � + � − � + A = a 21 a 22 a 23 � � � � � � a 32 a 33 a 31 a 33 a 31 a 32   � � � �   a 31 a 32 a 33    � � � � � �  1 a 12 a 13 a 11 a 13 a 11 a 12 A − 1 =  � � � � � �  − � + � −  � � � � � �  | A | a 32 a 33 a 31 a 33 a 31 a 32  � � � �      � � � � � � a 12 a 13 a 11 a 13 a 11 a 12   � � � � � � + � − � + � � � � � � a 22 a 23 a 21 a 23 a 21 a 22 � � � � 14

Revised Simplex Method Eta Factorization of the Basis Efficiency Issues Let B := A B , k th iteration B k be the matrix with col p differing from B k − 1 Column p is the a column appearing in B k − 1 d = a solved at 3) Hence: B k = B k − 1 E k E k is the eta matrix differing from id. matrix in only one column       − 1 1 1 − 1 1 0 1 − 1  = 1 0 0 1 0 1 1 0      0 1 0 0 1 0 1 No matter how we solve y T B k − 1 = c T B and B k − 1 d = a , their update always relays on B k = B k − 1 E k with E k available. Plus when initial basis by slack variable B 0 = I and B 1 = E 1 , B 2 = E 1 E 2 · · · : B k = E 1 E 2 . . . E k eta factorization (((( y T E 1 ) E 2 ) E 3 ) · · · ) E k = c T u T E 4 = c T B , v T E 3 = u T , w T E 2 = v T , y T E 1 = w T B , ( E 1 ( E 2 · · · E k d )) = a , E 1 u = a , E 2 v = u , E 3 w = v , E 4 d = w 15

Revised Simplex Method Efficiency Issues • Solving y T B k = c T B also called backward transformation (BTRAN) • Solving B k d = a also called forward transformation (FTRAN) • E i matrices can be stored by only storing the column and the position • If sparse columns then can be stored in compact mode, ie only nonzero values and their indices 19

Revised Simplex Method More on LP Efficiency Issues • Tableau method is unstable: computational errors may accumulate. Revised method has a natural control mechanism: we can recompute A − 1 at any time B • Commercial and freeware solvers differ from the way the systems y T A B = c T B and A B d = a are resolved 21

Revised Simplex Method Efficient Implementations Efficiency Issues • Dual simplex with steepest descent (largest increase) • Linear Algebra: • Dynamic LU-factorization using Markowitz threshold pivoting (Suhl and Suhl, 1990) • sparse linear systems: Typically these systems take as input a vector with a very small number of nonzero entries and output a vector with only a few additional nonzeros. • Presolve, ie problem reductions: removal of redundant constraints, fixed variables, and other extraneous model elements. • dealing with degeneracy, stalling (long sequences of degenerate pivots), and cycling: • bound-shifting (Paula Harris, 1974) • Hybrid Pricing (variable selection): start with partial pricing, then switch to devex (approximate steepest-edge, Harris, 1974) • A model that might have taken a year to solve 10 years ago can now solve in less than 30 seconds (Bixby, 2002). 22