The Dual Simplex Method Combinatorial Problem Solving (CPS) Javier - PowerPoint PPT Presentation

The Dual Simplex Method Combinatorial Problem Solving (CPS) Javier Larrosa Albert Oliveras Enric Rodr´ ıguez-Carbonell April 24, 2020

Basic Idea Abuse of terminology: ■ Henceforth sometimes by “optimal” we will mean “satisfying the optimality conditions” If not explicit, the context will disambiguate The algorithm as explained so far is known as primal simplex: ■ starting with feasible basis, find optimal basis (= satisfying optimality conds.) while keeping feasibility There is an alternative algorithm known as dual simplex: ■ starting with optimal basis (= satisfying optimality conds.), find feasible basis while keeping optimality 2 / 35

Basic Idea  min − x − y  min − x − y   2 x + y ≥ 3     2 x + y − s 1 = 3     2 x + y ≤ 6   = ⇒ 2 x + y + s 2 = 6 x + 2 y ≤ 6 x + 2 y + s 3 = 6     x ≥ 0     x, y, s 1 , s 2 , s 3 ≥ 0    y ≥ 0   min − 6 + y + s 3  Basis ( x, s 1 , s 2 ) is optimal  x = 6 − 2 y − s 3  (= satisfies optimality conditions) s 1 = 9 − 3 y − 2 s 3   s 2 = − 6 + 3 y + 2 s 3  but is not feasible! 3 / 35

Basic Idea y 2 x + y ≤ 6 min − x − y 2 x + y ≥ 3 x + 2 y ≤ 6 x ≥ 0 y ≥ 0 x (6 , 0) 4 / 35

Basic Idea Let us make a violating basic variable non-negative ... ■ Increase s 2 by making it non-basic: then it will be 0 ◆ ... while preserving optimality (= optimality conditions are satisfied) ■ If y replaces s 2 in the basis, ◆ then y = 1 3 ( s 2 + 6 − 2 s 3 ) , − x − y = − 4 + 1 3 ( s 2 + s 3 ) If s 3 replaces s 2 in the basis, ◆ then s 3 = 1 2 ( s 2 + 6 − 3 y ) , − x − y = − 3 + 1 2 ( s 2 − y ) 5 / 35

Basic Idea Let us make a violating basic variable non-negative ... ■ Increase s 2 by making it non-basic: then it will be 0 ◆ ... while preserving optimality (= optimality conditions are satisfied) ■ If y replaces s 2 in the basis, ◆ then y = 1 3 ( s 2 + 6 − 2 s 3 ) , − x − y = − 4 + 1 3 ( s 2 + s 3 ) If s 3 replaces s 2 in the basis, ◆ then s 3 = 1 2 ( s 2 + 6 − 3 y ) , − x − y = − 3 + 1 2 ( s 2 − y ) To preserve optimality, y must replace s 2 ◆ 5 / 35

Basic Idea min − 4 + 1 3 s 2 + 1   min − 6 + y + s 3 3 s 3   x = 2 − 2 3 s 2 + 1   x = 6 − 2 y − s 3 3 s 3   = ⇒ y = 2 + 1 3 s 2 − 2 s 1 = 9 − 3 y − 2 s 3 3 s 3     s 2 = − 6 + 3 y + 2 s 3 s 1 = 3 − s 2   6 / 35

Basic Idea min − 4 + 1 3 s 2 + 1   min − 6 + y + s 3 3 s 3   x = 2 − 2 3 s 2 + 1   x = 6 − 2 y − s 3 3 s 3   = ⇒ y = 2 + 1 3 s 2 − 2 s 1 = 9 − 3 y − 2 s 3 3 s 3     s 2 = − 6 + 3 y + 2 s 3 s 1 = 3 − s 2   Current basis is feasible and optimal! ■ 6 / 35

Basic Idea y 2 x + y ≤ 6 min − x − y (2 , 2) 2 x + y ≥ 3 x + 2 y ≤ 6 x ≥ 0 y ≥ 0 x (6 , 0) 7 / 35

Outline of the Dual Simplex 1. Initialization: Pick an optimal basis. 2. Dual Pricing: If all basic values are ≥ 0 , then return OPTIMAL. Else pick a basic variable with value < 0 . 3. Dual Ratio test: Find non-basic variable for swapping that preserves optimality, i.e., non-negativity constraints on reduced costs. If it does not exist, then return INFEASIBLE. Else swap chosen non-basic variable with violating basic variable. 4. Update: Update the tableau and go to 2. 8 / 35

Duality To understand better how the dual simplex works: theory of duality ■ We can get lower bounds on LP optimum value ■ by adding constraints in a convenient way   min − x − y min − x − y     2 x + y ≥ 3 2 x + y ≥ 3       − x − 2 y ≥ − 6     2 x + y ≤ 6 − 2 x − y ≥ − 6     ≥ 0 y       x + 2 y ≤ 6 − x − 2 y ≥ − 6 ⇒ x ≥ 0 x ≥ 0         − x − y ≥ − 6   y ≥ 0 y ≥ 0               9 / 35

Duality In general we can get lower bounds on LP optimum value ■ by linearly combining constraints with convenient multipliers 1 · ( 2 x + y ≥ 3 ) 2 · ( − 2 x − y ≥ − 6 )  min − x − y 1 · ( ≥ 0 ) x   2 x + y ≥ 3      − 2 x − y ≥ − 6      − x − 2 y ≥ − 6 2 x + y ≥ 3 x ≥ 0    − 4 x − 2 y ≥ − 12   y ≥ 0    ≥ 0 x     − x − y ≥ − 9 There may be different choices, each giving a different lower bound ■ 10 / 35

Duality µ 1 · ( 2 x + y ≥ 3 ) µ 2 · ( − 2 x − y ≥ − 6 ) µ 3 · ( − x − 2 y ≥ − 6 ) In general: ■ µ 4 · ( ≥ 0 ) x  min − x − y µ 5 · ( ≥ 0 ) y   2 x + y ≥ 3      − 2 x − y ≥ − 6     ≥ 2 µ 1 x + µ 1 y 3 µ 1  − x − 2 y ≥ − 6 − 2 µ 2 x − µ 2 y ≥ − 6 µ 2 x ≥ 0    − µ 3 x − 2 µ 3 y ≥ − 6 µ 3   y ≥ 0    ≥ 0 µ 4 x     ≥ 0 µ 5 y (2 µ 1 − 2 µ 2 − µ 3 + µ 4 ) x + ( µ 1 − µ 2 − 2 µ 3 + µ 5 ) y ≥ 3 µ 1 − 6 µ 2 − 6 µ 3 If 2 µ 1 − 2 µ 2 − µ 3 + µ 4 = − 1 , µ 1 − µ 2 − 2 µ 3 + µ 5 = − 1 , ■ µ 1 ≥ 0 , µ 2 ≥ 0 , µ 3 ≥ 0 , µ 4 ≥ 0 , µ 5 ≥ 0 , then 3 µ 1 − 6 µ 2 − 6 µ 3 is a lower bound 11 / 35

Duality We can skip the multipliers of the non-negativity constraints ■ µ 1 · ( 2 x + y ≥ 3 ) We have: ■ µ 2 · ( − 2 x − y ≥ − 6 )  min − x − y µ 3 · ( − x − 2 y ≥ − 6 )   2 x + y ≥ 3      − 2 x − y ≥ − 6   ≥ 2 µ 1 x + µ 1 y 3 µ 1 − x − 2 y ≥ − 6  − 2 µ 2 x − µ 2 y ≥ − 6 µ 2  x ≥ 0     − µ 3 x − 2 µ 3 y ≥ − 6 µ 3  y ≥ 0   (2 µ 1 − 2 µ 2 − µ 3 ) x + ( µ 1 − µ 2 − 2 µ 3 ) y ≥ 3 µ 1 − 6 µ 2 − 6 µ 3 Imagine 2 µ 1 − 2 µ 2 − µ 3 ≤ − 1 . ■ In the coefficient of x we can “complete” 2 µ 1 − 2 µ 2 − µ 3 to reach − 1 by adding a suitable multiple of x ≥ 0 (the multiplier will be the slack) If 2 µ 1 − 2 µ 2 − µ 3 ≤ − 1 , µ 1 − µ 2 − 2 µ 3 ≤ − 1 , ■ µ 1 ≥ 0 , µ 2 ≥ 0 , µ 3 ≥ 0 , then 3 µ 1 − 6 µ 2 − 6 µ 3 is a lower bound 12 / 35

Duality Best possible lower bound with this “trick” can be found by solving ■  3 µ 1 − 6 µ 2 − 6 µ 3 max   2 µ 1 − 2 µ 2 − µ 3 ≤ − 1      µ 1 − µ 2 − 2 µ 3 ≤ − 1 µ 1 , µ 2 , µ 3 ≥ 0        How far will it be from the optimum? ■ 13 / 35

Duality Best possible lower bound with this “trick” can be found by solving ■  3 µ 1 − 6 µ 2 − 6 µ 3 max   2 µ 1 − 2 µ 2 − µ 3 ≤ − 1      µ 1 − µ 2 − 2 µ 3 ≤ − 1 µ 1 , µ 2 , µ 3 ≥ 0        How far will it be from the optimum? ■ A best solution is given by ( µ 1 , µ 2 , µ 3 ) = (0 , 1 3 , 1 3 ) ■ 0 · ( ≥ 2 x + y 3 ) 1 3 · ( − 2 x − y ≥ − 6 ) 1 3 · ( − x − 2 y ≥ − 6 ) Matches the optimum! − x − y ≥ − 4 13 / 35

Dual Problem If we multiply Ax ≥ b by multipliers y T ≥ 0 we get y T Ax ≥ y T b ■ If y T A ≤ c T then we get a lower bound y T b for the cost function c T x ■ Given an LP (called primal problem) ■ min c T x Ax ≥ b x ≥ 0 its dual problem is the LP max y T b max b T y y T A ≤ c T A T y ≤ c or equivalently y T ≥ 0 y ≥ 0 Primal variables associated with columns of A ■ Dual variables (multipliers) associated with rows of A ■ Objective and right-hand side vectors swap their roles ■ 14 / 35

Dual Problem Prop. The dual of the dual is the primal. ■ Proof: max b T y − min ( − b ) T y A T y ≤ c − A T y ≥ − c = ⇒ y ≥ 0 y ≥ 0 − max − c T x min c T x ( − A T ) T x ≤ − b ⇒ = Ax ≥ b x ≥ 0 x ≥ 0 We say the primal and the dual form a primal-dual pair ■ 15 / 35

Dual Problem min c T x and max b T y Prop. Ax = b form a primal-dual pair ■ A T y ≤ c x ≥ 0 Proof: min c T x min c T x Ax ≥ b Ax = b = ⇒ − Ax ≥ − b x ≥ 0 x ≥ 0 max b T y 1 − b T y 2 max b T y y := y 1 − y 2 A T y 1 − A T y 2 ≤ c = ⇒ A T y ≤ c y 1 , y 2 ≥ 0 16 / 35

Duality Theorems Th. (Weak Duality) Let ( P, D ) be a primal-dual pair ■ min c T x ( D ) max b T y ( P ) Ax = b and A T y ≤ c x ≥ 0 If x is feasible solution to P and y is feasible solution to D then b T y ≤ c T x Proof: c − A T y ≥ 0 , i.e., c T − y T A ≥ 0 , and x ≥ 0 imply c T x − y T Ax ≥ 0 . So c T x ≥ y T Ax , and c T x ≥ y T Ax = y T b = b T y 17 / 35

Duality Theorems Feasible solutions to D give lower bounds on P ■ Feasible solutions to P give upper bounds on D ■ Will the two optimum values be always equal? ■ 18 / 35