More on Duality Marco Chiarandini Department of Mathematics & - PowerPoint PPT Presentation

DM545/DM871 Linear and Integer Programming Lecture 6 More on Duality Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

Derivation Dual Simplex Outline Sensitivity Analysis 1. Derivation Lagrangian Duality 2. Dual Simplex 3. Sensitivity Analysis 2

Derivation Dual Simplex Summary Sensitivity Analysis • Derivation: 1. economic interpretation 2. bounding 3. multipliers 4. recipe 5. Lagrangian • Theory: • Symmetry • Weak duality theorem • Strong duality theorem • Complementary slackness theorem • Dual Simplex • Sensitivity Analysis, Economic interpretation 3

Derivation Dual Simplex Outline Sensitivity Analysis 1. Derivation Lagrangian Duality 2. Dual Simplex 3. Sensitivity Analysis 4

Derivation Dual Simplex Outline Sensitivity Analysis 1. Derivation Lagrangian Duality 2. Dual Simplex 3. Sensitivity Analysis 9

Derivation Dual Simplex Lagrangian Duality Sensitivity Analysis Relaxation: if a problem is hard to solve then find an easier problem resembling the original one that provides information in terms of bounds. Then, search for the strongest bounds. min 13 x 1 + 6 x 2 + 4 x 3 + 12 x 4 2 x 1 + 3 x 2 + 4 x 3 + 5 x 4 = 7 3 x 1 + + 2 x 3 + 4 x 4 = 2 x 1 , x 2 , x 3 , x 4 ≥ 0 10

Derivation Dual Simplex Lagrangian Duality Sensitivity Analysis Relaxation: if a problem is hard to solve then find an easier problem resembling the original one that provides information in terms of bounds. Then, search for the strongest bounds. min 13 x 1 + 6 x 2 + 4 x 3 + 12 x 4 2 x 1 + 3 x 2 + 4 x 3 + 5 x 4 = 7 3 x 1 + + 2 x 3 + 4 x 4 = 2 x 1 , x 2 , x 3 , x 4 ≥ 0 We wish to reduce to a problem easier to solve, ie: min c 1 x 1 + c 2 x 2 + . . . + c n x n x 1 , x 2 , . . . , x n ≥ 0 10

Derivation Dual Simplex Lagrangian Duality Sensitivity Analysis Relaxation: if a problem is hard to solve then find an easier problem resembling the original one that provides information in terms of bounds. Then, search for the strongest bounds. min 13 x 1 + 6 x 2 + 4 x 3 + 12 x 4 2 x 1 + 3 x 2 + 4 x 3 + 5 x 4 = 7 3 x 1 + + 2 x 3 + 4 x 4 = 2 x 1 , x 2 , x 3 , x 4 ≥ 0 We wish to reduce to a problem easier to solve, ie: min c 1 x 1 + c 2 x 2 + . . . + c n x n x 1 , x 2 , . . . , x n ≥ 0 solvable by inspection: 10

Derivation Dual Simplex Lagrangian Duality Sensitivity Analysis Relaxation: if a problem is hard to solve then find an easier problem resembling the original one that provides information in terms of bounds. Then, search for the strongest bounds. min 13 x 1 + 6 x 2 + 4 x 3 + 12 x 4 2 x 1 + 3 x 2 + 4 x 3 + 5 x 4 = 7 3 x 1 + + 2 x 3 + 4 x 4 = 2 x 1 , x 2 , x 3 , x 4 ≥ 0 We wish to reduce to a problem easier to solve, ie: min c 1 x 1 + c 2 x 2 + . . . + c n x n x 1 , x 2 , . . . , x n ≥ 0 solvable by inspection: if c < 0 then x = + ∞ , if c ≥ 0 then x = 0. 10

Derivation Dual Simplex Lagrangian Duality Sensitivity Analysis Relaxation: if a problem is hard to solve then find an easier problem resembling the original one that provides information in terms of bounds. Then, search for the strongest bounds. min 13 x 1 + 6 x 2 + 4 x 3 + 12 x 4 2 x 1 + 3 x 2 + 4 x 3 + 5 x 4 = 7 3 x 1 + + 2 x 3 + 4 x 4 = 2 x 1 , x 2 , x 3 , x 4 ≥ 0 We wish to reduce to a problem easier to solve, ie: min c 1 x 1 + c 2 x 2 + . . . + c n x n x 1 , x 2 , . . . , x n ≥ 0 solvable by inspection: if c < 0 then x = + ∞ , if c ≥ 0 then x = 0. measure of violation of the constraints: 7 − ( 2 x 1 + 3 x 2 + 4 x 3 + 5 x 4 ) 2 − ( 3 x 1 + + 2 x 3 + 4 x 4 ) 10

Derivation Dual Simplex Sensitivity Analysis We relax these measures in obj. function with Lagrangian multipliers y 1 , y 2 . We obtain a family of problems:   13 x 1 + 6 x 2 + 4 x 3 + 12 x 4   PR ( y 1 , y 2 ) = min + y 1 ( 7 − 2 x 1 − 3 x 2 − 4 x 3 − 5 x 4 ) x 1 , x 2 , x 3 , x 4 ≥ 0 + y 2 ( 2 − 3 x 1 − 2 x 3 − 4 x 4 )   11

Derivation Dual Simplex Sensitivity Analysis We relax these measures in obj. function with Lagrangian multipliers y 1 , y 2 . We obtain a family of problems:   13 x 1 + 6 x 2 + 4 x 3 + 12 x 4   PR ( y 1 , y 2 ) = min + y 1 ( 7 − 2 x 1 − 3 x 2 − 4 x 3 − 5 x 4 ) x 1 , x 2 , x 3 , x 4 ≥ 0 + y 2 ( 2 − 3 x 1 − 2 x 3 − 4 x 4 )   1. for all y 1 , y 2 ∈ R : opt ( PR ( y 1 , y 2 )) ≤ opt ( P ) 2. max y 1 , y 2 ∈ R { opt ( PR ( y 1 , y 2 )) } ≤ opt ( P ) PR is easy to solve. (It can be also seen as a proof of the weak duality theorem) 11

Derivation Dual Simplex Sensitivity Analysis   ( 13 − 2 y 2 − 3 y 2 ) x 1     + ( 6 − 3 y 1 ) x 2       PR ( y 1 , y 2 ) = min + ( 4 − 2 y 2 ) x 3 x 1 , x 2 , x 3 , x 4 ≥ 0 + ( 12 − 5 y 1 − 4 y 2 ) x 4         + 7 y 1 + 2 y 2   12

Derivation Dual Simplex Sensitivity Analysis   ( 13 − 2 y 2 − 3 y 2 ) x 1     + ( 6 − 3 y 1 ) x 2       PR ( y 1 , y 2 ) = min + ( 4 − 2 y 2 ) x 3 x 1 , x 2 , x 3 , x 4 ≥ 0 + ( 12 − 5 y 1 − 4 y 2 ) x 4         + 7 y 1 + 2 y 2   if coeff. of x is < 0 then bound is −∞ then LB is useless ( 13 − 2 y 2 − 3 y 2 ) ≥ 0 ( 6 − 3 y 1 ) ≥ 0 ( 4 − 2 y 2 ) ≥ 0 ( 12 − 5 y 1 − 4 y 2 ) ≥ 0 12

Derivation Dual Simplex Sensitivity Analysis   ( 13 − 2 y 2 − 3 y 2 ) x 1     + ( 6 − 3 y 1 ) x 2       PR ( y 1 , y 2 ) = min + ( 4 − 2 y 2 ) x 3 x 1 , x 2 , x 3 , x 4 ≥ 0 + ( 12 − 5 y 1 − 4 y 2 ) x 4         + 7 y 1 + 2 y 2   if coeff. of x is < 0 then bound is −∞ then LB is useless ( 13 − 2 y 2 − 3 y 2 ) ≥ 0 ( 6 − 3 y 1 ) ≥ 0 ( 4 − 2 y 2 ) ≥ 0 ( 12 − 5 y 1 − 4 y 2 ) ≥ 0 If they all hold then we are left with 7 y 1 + 2 y 2 because all go to 0. max 7 y 1 + 2 y 2 2 y 2 + 3 y 2 ≤ 13 3 y 1 ≤ 6 + 2 y 2 ≤ 4 5 y 1 + 4 y 2 ≤ 12 12

Derivation Dual Simplex General Formulation Sensitivity Analysis z = c T x c ∈ R n min A ∈ R m × n , b ∈ R m A x = b x ≥ 0 x ∈ R n { c T x + y T ( b − A x ) }} y ∈ R m { min max x ∈ R n + { ( c T − y T A ) x + y T b }} y ∈ R m { min max x ∈ R n + b T y max A T y ≤ c y ∈ R m 13

Derivation Dual Simplex Outline Sensitivity Analysis 1. Derivation Lagrangian Duality 2. Dual Simplex 3. Sensitivity Analysis 14

Derivation Dual Simplex Dual Simplex Sensitivity Analysis • Dual simplex (Lemke, 1954): apply the simplex method to the dual problem and observe what happens in the primal tableau: max { c T x | Ax ≤ b , x ≥ 0 } = min { b T y | A T y ≥ c T , y ≥ 0 } = − max {− b T y | − A T x ≤ − c T , y ≥ 0 } • We obtain a new algorithm for the primal problem: the dual simplex It corresponds to the primal simplex applied to the dual Primal simplex on primal problem: Dual simplex on primal problem: 1. pivot < 0 1. pivot > 0 2. row b i < 0 2. col c j with wrong sign (condition of feasibility) � � b i 3. row: min a ij : a ij > 0 , i = 1 , .., m �� � � � c j 3. col: min � : a ij < 0 , j = 1 , 2 , .., n + m � � a ij (least worsening solution) 15

Derivation Dual Simplex Dual Simplex Sensitivity Analysis 0. (primal) simplex on primal problem (the one studied so far) 1. Now: dual simplex on primal problem ≡ primal simplex on dual problem (implemented as dual simplex, understood as primal simplex on dual problem) Uses of 1.: • The dual simplex can work better than the primal in some cases. Eg. since running time in practice between 2 m and 3 m , then if m = 99 and n = 9 then better the dual • Infeasible start Dual based Phase I algorithm (Dual-primal algorithm) 16

Derivation Dual Simplex for Phase I Dual Simplex Sensitivity Analysis Example Primal: Dual: max − x 1 − x 2 min 4 y 1 − 8 y 2 − 7 y 3 − 2 x 1 − x 2 ≤ 4 − 2 y 1 − 2 y 2 − y 3 ≥ − 1 − 2 x 1 + 4 x 2 ≤ − 8 − y 1 + 4 y 2 + 3 y 3 ≥ − 1 − x 1 + 3 x 2 ≤ − 7 y 1 , y 2 , y 3 ≥ 0 x 1 , x 2 ≥ 0 18

Derivation Dual Simplex for Phase I Dual Simplex Sensitivity Analysis Example Primal: Dual: max − x 1 − x 2 min 4 y 1 − 8 y 2 − 7 y 3 − 2 x 1 − x 2 ≤ 4 − 2 y 1 − 2 y 2 − y 3 ≥ − 1 − 2 x 1 + 4 x 2 ≤ − 8 − y 1 + 4 y 2 + 3 y 3 ≥ − 1 − x 1 + 3 x 2 ≤ − 7 y 1 , y 2 , y 3 ≥ 0 x 1 , x 2 ≥ 0 • Initial tableau • Initial tableau (min by ≡ − max − by ) | | x1 | x2 | w1 | w2 | w3 | -z | b | | | y1 | y2 | y3 | z1 | z2 | -p | b | |---+----+----+----+----+----+----+----| |---+----+----+----+----+----+----+---| | | -2 | -1 | 1 | 0 | 0 | 0 | 4 | | | 2 | 2 | 1 | 1 | 0 | 0 | 1 | | | -2 | 4 | 0 | 1 | 0 | 0 | -8 | | | 1 | -4 | -3 | 0 | 1 | 0 | 1 | | | -1 | 3 | 0 | 0 | 1 | 0 | -7 | |---+----+----+----+----+----+----+---| |---+----+----+----+----+----+----+----| | | -4 | 8 | 7 | 0 | 0 | 1 | 0 | | | -1 | -1 | 0 | 0 | 0 | 1 | 0 | infeasible start feasible start (thanks to − x 1 − x 2 ) 18