Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation Chen Jiang Hang Transportation and Mobility Laboratory July 19, 2013 Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 1 / 42
Outline Duality theory 1 Column generation 2 Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 2 / 42
Duality theory Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 3 / 42
Duality theory Basic idea Goal Primal Dual Start Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 4 / 42
Duality theory Lagrangian multipliers to duality Consider a linear programming problem (primal) c ′ x min : Ax = b s.t. x ≥ 0 Let x ∗ be an optimal solution. We introduce a relaxed problem in which the constraint Ax = b is replaced by a penalty p ′ ( b − Ax ) , where p is the lagrangian multiplier vector. We are then faced with the problem c ′ x + p ′ ( b − Ax ) min : s.t. x ≥ 0 Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 5 / 42
Duality theory Lagrangian multipliers to duality Let g ( p ) be the optimal cost for the relaxed problem, as a function of the multiplier vector p . x ≥ 0 [ c ′ x + p ′ ( b − Ax )] ≤ c ′ x ∗ + p ′ ( b − Ax ∗ ) = c ′ x ∗ g ( p ) = min Thus, each p leads to a lower bound g ( p ) for the optimal cost c ′ x ∗ . max : g ( p ) s.t. no constraints can be interpreted as a search for the tightest possible lower bound for the primal, which is usually called the dual problem. Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 6 / 42
Duality theory Lagrangian multipliers to duality x ≥ 0 [ c ′ x + p ′ ( b − Ax )] g ( p ) = min x ≥ 0 ( c ′ − p ′ A ) x p ′ b + min = Note that � 0 , if c ′ − p ′ A ≥ 0 ′ ; x ≥ 0 ( c ′ − p ′ A ) x = min −∞ , otherwise. In maximizing g ( p ) , we only need to consider those values of p for which g ( p ) is not equal to −∞ . We therefore conclude that the dual problem is the same as the another linear programming problem p ′ b max : p ′ A ≤ c ′ s.t. Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 7 / 42
Duality theory Shortcut to write a dual problem for a primal PRIMAL min max DUAL ≥ b i ≥ 0 constraints ≤ b i ≤ 0 variables = b i free ≥ 0 ≤ b i variables ≤ 0 ≥ b i constraints free = b i The dual of the dual is the primal!!! If we transform the dual into an equivalent minimization problem and the form its dual, we obtain a problem equivalent to the original problem. Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 8 / 42
Duality theory An example for practice min +2 x 2 +3 x 3 max 5 p 1 +6 p 2 +4 p 3 x 1 s.t. x 1 +3 x 2 = 5 ( p 1 ) s.t. p 1 free 2 x 1 − x 2 +3 x 3 ≥ 6 ( p 2 ) ≥ 0 p 2 ≤ 4 ( p 3 ) ≤ 0 x 3 p 3 x 1 ≥ 0 p 1 +2 p 2 ≤ 1 ≤ 0 3 p 1 − p 2 ≥ 2 x 2 x 3 free 3 p 2 + p 3 = 3 Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 9 / 42
Duality theory Another way to write a dual problem, my experience max 5 x 1 +6 x 2 +4 x 3 s.t. x 1 +2 x 2 ≤ 1 ( p 1) 3 x 1 − x 2 ≥ 2 ( p 2) 3 x 2 + x 3 = 3 ( p 3) ≥ 0 ( p 4) x 2 x 3 ≤ 0 ( p 5) Two rules to remember: 1 p ′ A = c ′ 2 if the sense of the problem is min , the lagrangian function (only consider the DUAL × (RHS-LHS)) always tries to provide a lower bound ; if the sense of the problem is max , the lagrangian function always tries to provide an upper bound . Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 10 / 42
Duality theory Duality theorem Week duality If x is a feasible solution to the primal problem and p is a feasible solution to the dual problem, then, p ′ b ≤ c ′ x Strong duality Let x and p be feasible solutions to the primal and dual problem, respectively, and suppose that p ′ b = c ′ x . Then, x and p are optimal solutions to the primal and the dual, respectively. Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 11 / 42
Duality theory Complementary slackness Complementary slackness Let x and p be feasible solutions to the primal and the dual problem, respectively. The vector x and p are optimal solutions for the two respective problems if and only if: p i ( a ′ i x − b i ) = 0 , ∀ i ( c j − p ′ A j ) x j = 0 , ∀ j Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 12 / 42
Duality theory The application of complementary slackness Suppose we have a linear programming problem with optimal solution ( x 1 , x 2 , x 3 ) = (1 , 0 , 1) : min 13 x 1 +10 x 2 +6 x 3 5 x 1 + x 2 +3 x 3 = 8 s.t. 3 x 1 + x 2 = 3 ≤ 0 x 1 , x 2 , x 3 The dual problem is: max 8 p 1 +3 p 2 s.t. 5 p 1 +3 p 2 ?13 + p 2 ?10 p 1 3 p 1 ?6 p 1 ? p 2 ? Can you obtain the optimal solution for the dual problem without solving it? Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 13 / 42
Duality theory The application of complementary slackness Using the complementary slackness: (13 − 5 p 1 − 3 p 2 ) x 1 = 0 (10 − p 1 − p 2 ) x 2 = 0 (6 − 3 p 1 ) x 3 = 0 Plug in x 1 = 1 , x 2 = 0 , x 3 = 1 , it can be solved that p 1 = 2 , p 2 = 1 Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 14 / 42
Column generation Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 15 / 42
Column generation Motivation to use column generation Suppose that the number of columns is so large that it is impossible to generate and store the entire matrix A in memory. c ′ x min : Ax = b s.t. x ≥ 0 Observation: 1 Experience with large optimization problems indicates that, usually, most of the columns never enter the basis, and we can therefore afford not to ever generate these unused columns; 2 From Simplex method’s point of view, at any given iteration, only requires the current basic columns and the column which is to enter the basis. Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 16 / 42
Column generation Motivation to use column generation At any given iteration: 4 ♣ K ♠ ? ♣ ? ♠ ? ♦ · · · ? ♥ Key question: how to evaluate the reduced costs (the points of the cards in this analogous example) associated with all of the nonbasic columns (there may be millions of them) and identify the entering column? Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 17 / 42
Column generation The idea Actually, we don’t need to figure out the list of reduced costs for all the nonbasic columns. Instead, in order to identify the entering column, it can be accomplish by solving another optimization. min c i where the minimization is over all column index i . In many instances, the above optimization problem has a special structure: a smallest c i can be found efficiently without computing every c i . Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 18 / 42
Column generation The basic steps of column generation 1 Identify a list of basic columns and initialize the set I which contains the indices of all of the columns that have been generated; 2 Solve the restricted problem to optimality by Simplex method; � � min { c i x i | A i x i = b , x ≥ 0 } i ∈ I i ∈ I 3 Based on the optimal basis matrix B , construct the reduced cost formula c i ′ = c ′ i − c ′ B B − 1 A i . Note that the column A i usually cannot be expressed explicitly; 4 Solve min c i . If the optimal value is greater or equal to 0, then terminate the algorithm. Otherwise, identify an index i ∗ with c i ∗ < 0 and dynamically generate a column A i ∗ and add i ∗ into I and proceed to Step 2. Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 19 / 42
Column generation A classic problem to use column generation The cutting stock problem: Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation 20 / 42
Recommend
More recommend
