More on Duality Marco Chiarandini Department of Mathematics & - PowerPoint PPT Presentation

DM545 Linear and Integer Programming Lecture 6 More on Duality Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

Derivation Outline Sensitivity Analysis 1. Derivation Geometric Interpretation Lagrangian Duality Dual Simplex 2. Sensitivity Analysis 2

Derivation Summary Sensitivity Analysis • Derivation: 1. economic interpretation 2. bounding 3. multipliers 4. recipe 5. Lagrangian • Theory: • Symmetry • Weak duality theorem • Strong duality theorem • Complementary slackness theorem • Dual Simplex • Sensitivity Analysis, Economic interpretation 3

Derivation Outline Sensitivity Analysis 1. Derivation Geometric Interpretation Lagrangian Duality Dual Simplex 2. Sensitivity Analysis 4

Derivation Dual Problem Sensitivity Analysis Dual variables y in one-to-one correspondence with the constraints: Primal problem: Dual Problem: w = b T y min z = c T x max A T y ≥ c A x = b y ∈ R m x ≥ 0 • Basic feasible solutions of (P) give immediate lower bounds on the optimal value z ∗ . Is there a simple way to get upper bounds? • The optimal solution must satisfy any linear combination y ∈ R m of the equality constraints. • If we can construct a linear combination of the equality constraints y T ( A x ) = y T b , for y ∈ R m , such that c T x ≤ y T ( A x ) , then y T ( A x ) = y T b is an upper bound on z ∗ . 5

Derivation Outline Sensitivity Analysis 1. Derivation Geometric Interpretation Lagrangian Duality Dual Simplex 2. Sensitivity Analysis 6

Derivation Geometric Interpretation Sensitivity Analysis x 2 max x 1 + x 2 = z 2 x 1 + x 2 ≤ 14 − x 1 + 2 x 2 ≤ 8 − x 1 + 2 x 2 ≤ 8 2 x 1 − x 2 ≤ 10 2 x 1 − x 2 ≤ 10 x 1 , x 2 ≥ 0 x 1 x 1 + x 2 2 x 1 + x 2 ≤ 14 Feasible sol x ∗ = ( 4 , 6 ) yields z ∗ = 10. To prove that it is optimal we need to verify that y ∗ = ( 3 / 5 , 1 / 5 , 0 ) is a feasible solution of D : min 14 y 1 + 8 y 2 + 10 y 3 = w the feasibility region 2 y 1 − y 2 + 2 y 3 ≥ 1 x 2 of P is a subset of the y 1 + 2 y 2 − y 3 ≥ 1 half plane x 1 + x 2 ≤ 10 y 1 , y 2 , y 3 ≥ 0 and that w ∗ = 10 3 5 · ( 2 x 1 + x 2 ≤ 14 ) 1 5 · ( − x 1 + 2 x 2 ≤ 8 ) x 1 x 1 + x 2 ≤ 10 x 1 + x 2 ≤ 10 7

Derivation Sensitivity Analysis ( 2 v − w ) x 1 + ( v + 2 w ) x 2 ≤ 14 v + 8 w set of halfplanes that contain the feasibility region of P and pass through [ 4 , 6 ] 2 v − w ≥ 1 x 2 v + 2 w ≥ 1 Example of boundary lines among those allowed: v = 1 , w = 0 = ⇒ 2 x 1 + x 2 = 14 x 1 + 3 x 2 = 22 v = 1 , w = 1 = ⇒ x 1 + 3 x 2 = 22 x 1 3 x 1 + 4 x 2 = 36 v = 2 , w = 1 = ⇒ 3 x 1 + 4 x 2 = 36 x 1 + x 2 ≤ 10 2 x 1 + x 2 = 14 8

Derivation Outline Sensitivity Analysis 1. Derivation Geometric Interpretation Lagrangian Duality Dual Simplex 2. Sensitivity Analysis 10

Derivation Lagrangian Duality Sensitivity Analysis Relaxation: if a problem is hard to solve then find an easier problem resembling the original one that provides information in terms of bounds. Then, search for the strongest bounds. min 13 x 1 + 6 x 2 + 4 x 3 + 12 x 4 2 x 1 + 3 x 2 + 4 x 3 + 5 x 4 = 7 3 x 1 + + 2 x 3 + 4 x 4 = 2 x 1 , x 2 , x 3 , x 4 ≥ 0 We wish to reduce to a problem easier to solve, ie: min c 1 x 1 + c 2 x 2 + . . . + c n x n x 1 , x 2 , . . . , x n ≥ 0 solvable by inspection: if c < 0 then x = + ∞ , if c ≥ 0 then x = 0. measure of violation of the constraints: 7 − ( 2 x 1 + 3 x 2 + 4 x 3 + 5 x 4 ) 2 − ( 3 x 1 + + 2 x 3 + 4 x 4 ) 11

Derivation Sensitivity Analysis We relax these measures in obj. function with Lagrangian multipliers y 1 , y 2 . We obtain a family of problems:   13 x 1 + 6 x 2 + 4 x 3 + 12 x 4   PR ( y 1 , y 2 ) = min + y 1 ( 7 − 2 x 1 − 3 x 2 − 4 x 3 − 5 x 4 ) x 1 , x 2 , x 3 , x 4 ≥ 0 + y 2 ( 2 − 3 x 1 − 2 x 3 − 4 x 4 )   1. for all y 1 , y 2 ∈ R : opt ( PR ( y 1 , y 2 )) ≤ opt ( P ) 2. max y 1 , y 2 ∈ R { opt ( PR ( y 1 , y 2 )) } ≤ opt ( P ) PR is easy to solve. (It can be also seen as a proof of the weak duality theorem) 12

Derivation Sensitivity Analysis   ( 13 − 2 y 2 − 3 y 2 ) x 1     + ( 6 − 3 y 1 ) x 2       PR ( y 1 , y 2 ) = min + ( 4 − 2 y 2 ) x 3 x 1 , x 2 , x 3 , x 4 ≥ 0 + ( 12 − 5 y 1 − 4 y 2 ) x 4         + 7 y 1 + 2 y 2   if coeff. of x is < 0 then bound is −∞ then LB is useless ( 13 − 2 y 2 − 3 y 2 ) ≥ 0 ( 6 − 3 y 1 ) ≥ 0 ( 4 − 2 y 2 ) ≥ 0 ( 12 − 5 y 1 − 4 y 2 ) ≥ 0 If they all hold then we are left with 7 y 1 + 2 y 2 because all go to 0. max 7 y 1 + 2 y 2 2 y 2 + 3 y 2 ≤ 13 3 y 1 ≤ 6 + 2 y 2 ≤ 4 5 y 1 + 4 y 2 ≤ 12 13

Derivation General Formulation Sensitivity Analysis z = c T x c ∈ R n min A ∈ R m × n , b ∈ R m A x = b x ∈ R n x ≥ 0 { c T x + y T ( b − A x ) }} y ∈ R m { min max x ∈ R n + { ( c T − y T A ) x + y T b }} y ∈ R m { min max x ∈ R n + b T y max A T y ≤ c y ∈ R m 14

Derivation Outline Sensitivity Analysis 1. Derivation Geometric Interpretation Lagrangian Duality Dual Simplex 2. Sensitivity Analysis 15

Derivation Dual Simplex Sensitivity Analysis • Dual simplex (Lemke, 1954): apply the simplex method to the dual problem and observe what happens in the primal tableau: max { c T x | Ax ≤ b , x ≥ 0 } = min { b T y | A T y ≥ c T , y ≥ 0 } = − max {− b T y | − A T x ≤ − c T , y ≥ 0 } • We obtain a new algorithm for the primal problem: the dual simplex It corresponds to the primal simplex applied to the dual Primal simplex on primal problem: Dual simplex on primal problem: 1. pivot < 0 1. pivot > 0 2. row b i < 0 2. col c j with wrong sign (condition of feasibility) � � b i 3. row: min a ij : a ij > 0 , i = 1 , .., m �� � � � c j 3. col: min � : a ij < 0 , j = 1 , 2 , .., n + m � � a ij (least worsening solution) 16

Derivation Dual Simplex Sensitivity Analysis 0. (primal) simplex on primal problem (the one studied so far) 1. Now: dual simplex on primal problem ≡ primal simplex on dual problem (implemented as dual simplex, understood as primal simplex on dual problem) Uses of 1.: • The dual simplex can work better than the primal in some cases. Eg. since running time in practice between 2 m and 3 m , then if m = 99 and n = 9 then better the dual • Infeasible start Dual based Phase I algorithm (Dual-primal algorithm) 17

Dual Simplex for Phase I Derivation Sensitivity Analysis Example Primal: Dual: max min 4 y 1 − 8 y 2 − 7 y 3 − x 1 − x 2 − 2 x 1 − 4 − 2 y 1 − 2 y 2 − y 3 ≥ − 1 x 2 ≤ − 2 x 1 + 4 x 2 ≤ − 8 − y 1 + 4 y 2 + 3 y 3 ≥ − 1 − x 1 + 3 x 2 ≤ − 7 y 1 , y 2 , y 3 ≥ 0 x 1 , x 2 ≥ 0 • Initial tableau • Initial tableau (min by ≡ − max − by ) | | x1 | x2 | w1 | w2 | w3 | -z | b | | | y1 | y2 | y3 | z1 | z2 | -p | b | |---+----+----+----+----+----+----+----| |---+----+----+----+----+----+----+---| | | -2 | -1 | 1 | 0 | 0 | 0 | 4 | | | 2 | 2 | 1 | 1 | 0 | 0 | 1 | | | -2 | 4 | 0 | 1 | 0 | 0 | -8 | | | 1 | -4 | -3 | 0 | 1 | 0 | 1 | | | -1 | 3 | 0 | 0 | 1 | 0 | -7 | |---+----+----+----+----+----+----+---| |---+----+----+----+----+----+----+----| | | -4 | 8 | 7 | 0 | 0 | 1 | 0 | | | -1 | -1 | 0 | 0 | 0 | 1 | 0 | infeasible start feasible start (thanks to − x 1 − x 2 ) • x 1 enters, w 2 leaves • y 2 enters, z 1 leaves 19

Derivation Sensitivity Analysis • x 1 enters, w 2 leaves • y 2 enters, z 1 leaves | | x1 | x2 | w1 | w2 | w3 | -z | b | | | y1 | y2 | y3 | z1 | z2 | -p | b | |---+----+----+----+------+----+----+----| |---+----+----+-----+-----+----+----+-----| | | 0 | -5 | 1 | -1 | 0 | 0 | 12 | | | 1 | 1 | 0.5 | 0.5 | 0 | 0 | 0.5 | | | 1 | -2 | 0 | -0.5 | 0 | 0 | 4 | | | 5 | 0 | -1 | 2 | 1 | 0 | 3 | | | 0 | 1 | 0 | -0.5 | 1 | 0 | -3 | |---+----+----+-----+-----+----+----+-----| |---+----+----+----+------+----+----+----| | | -4 | 0 | 3 | -12 | 0 | 1 | -4 | | | 0 | -3 | 0 | -0.5 | 0 | 1 | 4 | • y 3 enters, y 2 leaves • w 2 enters, w 3 leaves (note that we kept c j < 0, ie, optimality) | | y1 | y2 | y3 | z1 | z2 | -p | b | |---+-----+----+----+----+----+----+----| | | x1 | x2 | w1 | w2 | w3 | -z | b | | | 2 | 2 | 1 | 1 | 0 | 0 | 1 | |---+----+----+----+----+----+----+----| | | 7 | 2 | 0 | 3 | 1 | 0 | 3 | | | 0 | -7 | 1 | 0 | -2 | 0 | 18 | |---+-----+----+----+----+----+----+----| | | 1 | -3 | 0 | 0 | -1 | 0 | 7 | | | -18 | -6 | 0 | -7 | 0 | 1 | -7 | | | 0 | -2 | 0 | 1 | -2 | 0 | 6 | |---+----+----+----+----+----+----+----| | | 0 | -4 | 0 | 0 | -1 | 1 | 7 | 20

Derivation Summary Sensitivity Analysis • Derivation: 1. bounding 2. multipliers 3. recipe 4. Lagrangian • Theory: • Symmetry • Weak duality theorem • Strong duality theorem • Complementary slackness theorem • Dual Simplex • Sensitivity Analysis, Economic interpretation 21

Derivation Outline Sensitivity Analysis 1. Derivation Geometric Interpretation Lagrangian Duality Dual Simplex 2. Sensitivity Analysis 22