The SimplexInner Algorithm Describe the SimplexInner algorithm: . . LP is in slack form. 1 . . Trivial solution x = τ (i.e., all nonbasic variables zero), is 2 feasible. . . objective value for this solution is v . 3 Reminder: Objective function is z = v + ∑ . . j ∈ N c j x j . 4 . . x e : nonbasic variable with positive coefficient in objective 5 function. � { } � . . Formally: e is one of the indices of j � c j > 0 , j ∈ N . 6 . . x e is the entering variable (enters set of basic variables). 7 . . If increase value x e (from current value of 0 in τ )... 8 . . ... one of basic variables is going to vanish (i.e., become zero). 9 Sariel (UIUC) CS573 7 Fall 2013 7 / 43
The SimplexInner Algorithm Describe the SimplexInner algorithm: . . LP is in slack form. 1 . . Trivial solution x = τ (i.e., all nonbasic variables zero), is 2 feasible. . . objective value for this solution is v . 3 Reminder: Objective function is z = v + ∑ . . j ∈ N c j x j . 4 . . x e : nonbasic variable with positive coefficient in objective 5 function. � { } � . . Formally: e is one of the indices of j � c j > 0 , j ∈ N . 6 . . x e is the entering variable (enters set of basic variables). 7 . . If increase value x e (from current value of 0 in τ )... 8 . . ... one of basic variables is going to vanish (i.e., become zero). 9 Sariel (UIUC) CS573 7 Fall 2013 7 / 43
The SimplexInner Algorithm Describe the SimplexInner algorithm: . . LP is in slack form. 1 . . Trivial solution x = τ (i.e., all nonbasic variables zero), is 2 feasible. . . objective value for this solution is v . 3 Reminder: Objective function is z = v + ∑ . . j ∈ N c j x j . 4 . . x e : nonbasic variable with positive coefficient in objective 5 function. � { } � . . Formally: e is one of the indices of j � c j > 0 , j ∈ N . 6 . . x e is the entering variable (enters set of basic variables). 7 . . If increase value x e (from current value of 0 in τ )... 8 . . ... one of basic variables is going to vanish (i.e., become zero). 9 Sariel (UIUC) CS573 7 Fall 2013 7 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Choosing the leaving variable . . x e : entering variable 1 . . x l : leaving variable – vanishing basic variable. 2 . . increase value of x e till x l becomes zero. 3 . . How do we now which variable is x l ? 4 . . set all nonbasic to 0 zero, except x e 5 . . x i = b i − a ie x e , for all i ∈ B . 6 . . Require: ∀ i ∈ B x i = b i − a ie x e ≥ 0 . 7 . . = ⇒ x e ≤ ( b i / a ie ) 8 . . l = arg min i b i / a ie 9 . . 10 If more than one achieves min i b i / a ie , just pick one. Sariel (UIUC) CS573 8 Fall 2013 8 / 43
Pivoting on x e ... . . Determined x e and x l . 1 . . Rewrite equation for x l in LP . 2 (Every basic variable has an equation in the LP !) x l = b l − ∑ . . j ∈ N a lj x j 3 x e = b l ∑ a lj ⇒ − x j , where a ll = 1 . a le a le j ∈ N ∪{ l } . . LP cleanup: remove all appearances on right side of LP of x e . 4 . . Substituting x e into the other equalities, using above. 5 . . Alternatively, do Gaussian elimination remove any appearance of 6 x e on right side LP (including objective). Transfer x l on the left side, to the right side. Sariel (UIUC) CS573 9 Fall 2013 9 / 43
Pivoting on x e ... . . Determined x e and x l . 1 . . Rewrite equation for x l in LP . 2 (Every basic variable has an equation in the LP !) x l = b l − ∑ . . j ∈ N a lj x j 3 x e = b l ∑ a lj ⇒ − x j , where a ll = 1 . a le a le j ∈ N ∪{ l } . . LP cleanup: remove all appearances on right side of LP of x e . 4 . . Substituting x e into the other equalities, using above. 5 . . Alternatively, do Gaussian elimination remove any appearance of 6 x e on right side LP (including objective). Transfer x l on the left side, to the right side. Sariel (UIUC) CS573 9 Fall 2013 9 / 43
Pivoting on x e ... . . Determined x e and x l . 1 . . Rewrite equation for x l in LP . 2 (Every basic variable has an equation in the LP !) x l = b l − ∑ . . j ∈ N a lj x j 3 x e = b l ∑ a lj ⇒ − x j , where a ll = 1 . a le a le j ∈ N ∪{ l } . . LP cleanup: remove all appearances on right side of LP of x e . 4 . . Substituting x e into the other equalities, using above. 5 . . Alternatively, do Gaussian elimination remove any appearance of 6 x e on right side LP (including objective). Transfer x l on the left side, to the right side. Sariel (UIUC) CS573 9 Fall 2013 9 / 43
Pivoting on x e ... . . Determined x e and x l . 1 . . Rewrite equation for x l in LP . 2 (Every basic variable has an equation in the LP !) x l = b l − ∑ . . j ∈ N a lj x j 3 x e = b l ∑ a lj ⇒ − x j , where a ll = 1 . a le a le j ∈ N ∪{ l } . . LP cleanup: remove all appearances on right side of LP of x e . 4 . . Substituting x e into the other equalities, using above. 5 . . Alternatively, do Gaussian elimination remove any appearance of 6 x e on right side LP (including objective). Transfer x l on the left side, to the right side. Sariel (UIUC) CS573 9 Fall 2013 9 / 43
Pivoting on x e ... . . Determined x e and x l . 1 . . Rewrite equation for x l in LP . 2 (Every basic variable has an equation in the LP !) x l = b l − ∑ . . j ∈ N a lj x j 3 x e = b l ∑ a lj ⇒ − x j , where a ll = 1 . a le a le j ∈ N ∪{ l } . . LP cleanup: remove all appearances on right side of LP of x e . 4 . . Substituting x e into the other equalities, using above. 5 . . Alternatively, do Gaussian elimination remove any appearance of 6 x e on right side LP (including objective). Transfer x l on the left side, to the right side. Sariel (UIUC) CS573 9 Fall 2013 9 / 43
Pivoting on x e ... . . Determined x e and x l . 1 . . Rewrite equation for x l in LP . 2 (Every basic variable has an equation in the LP !) x l = b l − ∑ . . j ∈ N a lj x j 3 x e = b l ∑ a lj ⇒ − x j , where a ll = 1 . a le a le j ∈ N ∪{ l } . . LP cleanup: remove all appearances on right side of LP of x e . 4 . . Substituting x e into the other equalities, using above. 5 . . Alternatively, do Gaussian elimination remove any appearance of 6 x e on right side LP (including objective). Transfer x l on the left side, to the right side. Sariel (UIUC) CS573 9 Fall 2013 9 / 43
Pivoting on x e ... . . Determined x e and x l . 1 . . Rewrite equation for x l in LP . 2 (Every basic variable has an equation in the LP !) x l = b l − ∑ . . j ∈ N a lj x j 3 x e = b l ∑ a lj ⇒ − x j , where a ll = 1 . a le a le j ∈ N ∪{ l } . . LP cleanup: remove all appearances on right side of LP of x e . 4 . . Substituting x e into the other equalities, using above. 5 . . Alternatively, do Gaussian elimination remove any appearance of 6 x e on right side LP (including objective). Transfer x l on the left side, to the right side. Sariel (UIUC) CS573 9 Fall 2013 9 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Pivoting continued... . . End of this process: have new equivalent LP . 1 basic variables: B ′ =( B \ { l } ) ∪ { e } . . 2 non-basic variables: N ′ =( N \ { e } ) ∪ { l } . . . 3 . . End of this pivoting stage: LP objective function value 4 increased. . . Made progress. 5 . . LP is completely defined by which variables are basic, and which 6 are non-basic. . . Pivoting never returns to a combination (of basic/non-basic 7 variable) already visited. . ...because improve objective in each pivoting step. 8 ( n + m ) ( ) n . . . n + m Can do at most ≤ · e 9 n n 10 examples where 2 n pivoting steps are needed. . . Sariel (UIUC) CS573 10 Fall 2013 10 / 43
Simplex algorithm summary... . . Each pivoting step takes polynomial time in n and m . 1 . . Running time of Simplex is exponential in the worst case. 2 . . In practice, Simplex is extremely fast. 3 Sariel (UIUC) CS573 11 Fall 2013 11 / 43
Simplex algorithm summary... . . Each pivoting step takes polynomial time in n and m . 1 . . Running time of Simplex is exponential in the worst case. 2 . . In practice, Simplex is extremely fast. 3 Sariel (UIUC) CS573 11 Fall 2013 11 / 43
Simplex algorithm summary... . . Each pivoting step takes polynomial time in n and m . 1 . . Running time of Simplex is exponential in the worst case. 2 . . In practice, Simplex is extremely fast. 3 Sariel (UIUC) CS573 11 Fall 2013 11 / 43
Degeneracies . . Simplex might get stuck if one of the b i s is zero. 1 . . More than > m hyperplanes (i.e., equalities) passes through the 2 same point. . . Result: might not be able to make any progress at all in a 3 pivoting step. . . Solution I: add tiny random noise to each coefficient. 4 Can be done symbolically. Intuitively, the degeneracy, being a local phenomena on the polytope disappears with high probability. Sariel (UIUC) CS573 12 Fall 2013 12 / 43
Degeneracies . . Simplex might get stuck if one of the b i s is zero. 1 . . More than > m hyperplanes (i.e., equalities) passes through the 2 same point. . . Result: might not be able to make any progress at all in a 3 pivoting step. . . Solution I: add tiny random noise to each coefficient. 4 Can be done symbolically. Intuitively, the degeneracy, being a local phenomena on the polytope disappears with high probability. Sariel (UIUC) CS573 12 Fall 2013 12 / 43
Degeneracies . . Simplex might get stuck if one of the b i s is zero. 1 . . More than > m hyperplanes (i.e., equalities) passes through the 2 same point. . . Result: might not be able to make any progress at all in a 3 pivoting step. . . Solution I: add tiny random noise to each coefficient. 4 Can be done symbolically. Intuitively, the degeneracy, being a local phenomena on the polytope disappears with high probability. Sariel (UIUC) CS573 12 Fall 2013 12 / 43
Degeneracies . . Simplex might get stuck if one of the b i s is zero. 1 . . More than > m hyperplanes (i.e., equalities) passes through the 2 same point. . . Result: might not be able to make any progress at all in a 3 pivoting step. . . Solution I: add tiny random noise to each coefficient. 4 Can be done symbolically. Intuitively, the degeneracy, being a local phenomena on the polytope disappears with high probability. Sariel (UIUC) CS573 12 Fall 2013 12 / 43
Degeneracies . . Simplex might get stuck if one of the b i s is zero. 1 . . More than > m hyperplanes (i.e., equalities) passes through the 2 same point. . . Result: might not be able to make any progress at all in a 3 pivoting step. . . Solution I: add tiny random noise to each coefficient. 4 Can be done symbolically. Intuitively, the degeneracy, being a local phenomena on the polytope disappears with high probability. Sariel (UIUC) CS573 12 Fall 2013 12 / 43
Degeneracies . . Simplex might get stuck if one of the b i s is zero. 1 . . More than > m hyperplanes (i.e., equalities) passes through the 2 same point. . . Result: might not be able to make any progress at all in a 3 pivoting step. . . Solution I: add tiny random noise to each coefficient. 4 Can be done symbolically. Intuitively, the degeneracy, being a local phenomena on the polytope disappears with high probability. Sariel (UIUC) CS573 12 Fall 2013 12 / 43
Degeneracies – cycling . . Might get into cycling: a sequence of pivoting operations that 1 do not improve the objective function, and the bases you get are cyclic (i.e., infinite loop). . . Solution II: Bland’s rule . 2 Always choose the lowest index variable for entering and leaving out of the possible candidates. (Not prove why this work - but it does.) Sariel (UIUC) CS573 13 Fall 2013 13 / 43
Degeneracies – cycling . . Might get into cycling: a sequence of pivoting operations that 1 do not improve the objective function, and the bases you get are cyclic (i.e., infinite loop). . . Solution II: Bland’s rule . 2 Always choose the lowest index variable for entering and leaving out of the possible candidates. (Not prove why this work - but it does.) Sariel (UIUC) CS573 13 Fall 2013 13 / 43
Correctness of LP . Definition . A solution to an LP is a basic solution if it the result of setting all . the nonbasic variables to zero. Simplex algorithm deals only with basic solutions. . Theorem . For an arbitrary linear program, the following statements are true: (A) If there is no optimal solution, the problem is either infeasible or unbounded. (B) If a feasible solution exists, then a basic feasible solution exists. (C) If an optimal solution exists, then a basic optimal solution exists. . Proof: is constructive by running the simplex algorithm. Sariel (UIUC) CS573 14 Fall 2013 14 / 43
Correctness of LP . Definition . A solution to an LP is a basic solution if it the result of setting all . the nonbasic variables to zero. Simplex algorithm deals only with basic solutions. . Theorem . For an arbitrary linear program, the following statements are true: (A) If there is no optimal solution, the problem is either infeasible or unbounded. (B) If a feasible solution exists, then a basic feasible solution exists. (C) If an optimal solution exists, then a basic optimal solution exists. . Proof: is constructive by running the simplex algorithm. Sariel (UIUC) CS573 14 Fall 2013 14 / 43
Correctness of LP . Definition . A solution to an LP is a basic solution if it the result of setting all . the nonbasic variables to zero. Simplex algorithm deals only with basic solutions. . Theorem . For an arbitrary linear program, the following statements are true: (A) If there is no optimal solution, the problem is either infeasible or unbounded. (B) If a feasible solution exists, then a basic feasible solution exists. (C) If an optimal solution exists, then a basic optimal solution exists. . Proof: is constructive by running the simplex algorithm. Sariel (UIUC) CS573 14 Fall 2013 14 / 43
Correctness of LP . Definition . A solution to an LP is a basic solution if it the result of setting all . the nonbasic variables to zero. Simplex algorithm deals only with basic solutions. . Theorem . For an arbitrary linear program, the following statements are true: (A) If there is no optimal solution, the problem is either infeasible or unbounded. (B) If a feasible solution exists, then a basic feasible solution exists. (C) If an optimal solution exists, then a basic optimal solution exists. . Proof: is constructive by running the simplex algorithm. Sariel (UIUC) CS573 14 Fall 2013 14 / 43
On the ellipsoid method and interior point methods . . Simplex has exponential running time in the worst case. 1 . . ellipsoid method is weakly polynomial. 2 It is polynomial in the number of bits of the input. . . Khachian in 1979 came up with it. Useless in practice. 3 . . In 1984, Karmakar came up with a different method, called the 4 interior-point method . . . Also weakly polynomial. Quite useful in practice. 5 . . Result in arm race between the interior-point method and the 6 simplex method. . . BIG OPEN QUESTION: Is there strongly polynomial time 7 algorithm for linear programming? Sariel (UIUC) CS573 15 Fall 2013 15 / 43
On the ellipsoid method and interior point methods . . Simplex has exponential running time in the worst case. 1 . . ellipsoid method is weakly polynomial. 2 It is polynomial in the number of bits of the input. . . Khachian in 1979 came up with it. Useless in practice. 3 . . In 1984, Karmakar came up with a different method, called the 4 interior-point method . . . Also weakly polynomial. Quite useful in practice. 5 . . Result in arm race between the interior-point method and the 6 simplex method. . . BIG OPEN QUESTION: Is there strongly polynomial time 7 algorithm for linear programming? Sariel (UIUC) CS573 15 Fall 2013 15 / 43
On the ellipsoid method and interior point methods . . Simplex has exponential running time in the worst case. 1 . . ellipsoid method is weakly polynomial. 2 It is polynomial in the number of bits of the input. . . Khachian in 1979 came up with it. Useless in practice. 3 . . In 1984, Karmakar came up with a different method, called the 4 interior-point method . . . Also weakly polynomial. Quite useful in practice. 5 . . Result in arm race between the interior-point method and the 6 simplex method. . . BIG OPEN QUESTION: Is there strongly polynomial time 7 algorithm for linear programming? Sariel (UIUC) CS573 15 Fall 2013 15 / 43
On the ellipsoid method and interior point methods . . Simplex has exponential running time in the worst case. 1 . . ellipsoid method is weakly polynomial. 2 It is polynomial in the number of bits of the input. . . Khachian in 1979 came up with it. Useless in practice. 3 . . In 1984, Karmakar came up with a different method, called the 4 interior-point method . . . Also weakly polynomial. Quite useful in practice. 5 . . Result in arm race between the interior-point method and the 6 simplex method. . . BIG OPEN QUESTION: Is there strongly polynomial time 7 algorithm for linear programming? Sariel (UIUC) CS573 15 Fall 2013 15 / 43
On the ellipsoid method and interior point methods . . Simplex has exponential running time in the worst case. 1 . . ellipsoid method is weakly polynomial. 2 It is polynomial in the number of bits of the input. . . Khachian in 1979 came up with it. Useless in practice. 3 . . In 1984, Karmakar came up with a different method, called the 4 interior-point method . . . Also weakly polynomial. Quite useful in practice. 5 . . Result in arm race between the interior-point method and the 6 simplex method. . . BIG OPEN QUESTION: Is there strongly polynomial time 7 algorithm for linear programming? Sariel (UIUC) CS573 15 Fall 2013 15 / 43
On the ellipsoid method and interior point methods . . Simplex has exponential running time in the worst case. 1 . . ellipsoid method is weakly polynomial. 2 It is polynomial in the number of bits of the input. . . Khachian in 1979 came up with it. Useless in practice. 3 . . In 1984, Karmakar came up with a different method, called the 4 interior-point method . . . Also weakly polynomial. Quite useful in practice. 5 . . Result in arm race between the interior-point method and the 6 simplex method. . . BIG OPEN QUESTION: Is there strongly polynomial time 7 algorithm for linear programming? Sariel (UIUC) CS573 15 Fall 2013 15 / 43
On the ellipsoid method and interior point methods . . Simplex has exponential running time in the worst case. 1 . . ellipsoid method is weakly polynomial. 2 It is polynomial in the number of bits of the input. . . Khachian in 1979 came up with it. Useless in practice. 3 . . In 1984, Karmakar came up with a different method, called the 4 interior-point method . . . Also weakly polynomial. Quite useful in practice. 5 . . Result in arm race between the interior-point method and the 6 simplex method. . . BIG OPEN QUESTION: Is there strongly polynomial time 7 algorithm for linear programming? Sariel (UIUC) CS573 15 Fall 2013 15 / 43
On the ellipsoid method and interior point methods . . Simplex has exponential running time in the worst case. 1 . . ellipsoid method is weakly polynomial. 2 It is polynomial in the number of bits of the input. . . Khachian in 1979 came up with it. Useless in practice. 3 . . In 1984, Karmakar came up with a different method, called the 4 interior-point method . . . Also weakly polynomial. Quite useful in practice. 5 . . Result in arm race between the interior-point method and the 6 simplex method. . . BIG OPEN QUESTION: Is there strongly polynomial time 7 algorithm for linear programming? Sariel (UIUC) CS573 15 Fall 2013 15 / 43
Duality... . . Every linear program L has a dual linear program L ′ . 1 . . Solving the dual problem is essentially equivalent to solving the 2 primal linear program original LP . . . Lets look an example.. 3 Sariel (UIUC) CS573 16 Fall 2013 16 / 43
Duality by Example max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . η : maximal possible value of target function. 1 . . Any feasible solution ⇒ a lower bound on η . 2 . . In above: x 1 = 1 , x 2 = x 3 = 0 is feasible, and implies z = 4 3 and thus η ≥ 4 . . . x 1 = x 2 = 0 , x 3 = 3 is feasible = ⇒ η ≥ z = 9 . 4 . . How close this solution is to opt? (i.e., η ) 5 . . If very close to optimal – might be good enough. Maybe stop? 6 Sariel (UIUC) CS573 17 Fall 2013 17 / 43
Duality by Example max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . η : maximal possible value of target function. 1 . . Any feasible solution ⇒ a lower bound on η . 2 . . In above: x 1 = 1 , x 2 = x 3 = 0 is feasible, and implies z = 4 3 and thus η ≥ 4 . . . x 1 = x 2 = 0 , x 3 = 3 is feasible = ⇒ η ≥ z = 9 . 4 . . How close this solution is to opt? (i.e., η ) 5 . . If very close to optimal – might be good enough. Maybe stop? 6 Sariel (UIUC) CS573 17 Fall 2013 17 / 43
Duality by Example max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . η : maximal possible value of target function. 1 . . Any feasible solution ⇒ a lower bound on η . 2 . . In above: x 1 = 1 , x 2 = x 3 = 0 is feasible, and implies z = 4 3 and thus η ≥ 4 . . . x 1 = x 2 = 0 , x 3 = 3 is feasible = ⇒ η ≥ z = 9 . 4 . . How close this solution is to opt? (i.e., η ) 5 . . If very close to optimal – might be good enough. Maybe stop? 6 Sariel (UIUC) CS573 17 Fall 2013 17 / 43
Duality by Example max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . η : maximal possible value of target function. 1 . . Any feasible solution ⇒ a lower bound on η . 2 . . In above: x 1 = 1 , x 2 = x 3 = 0 is feasible, and implies z = 4 3 and thus η ≥ 4 . . . x 1 = x 2 = 0 , x 3 = 3 is feasible = ⇒ η ≥ z = 9 . 4 . . How close this solution is to opt? (i.e., η ) 5 . . If very close to optimal – might be good enough. Maybe stop? 6 Sariel (UIUC) CS573 17 Fall 2013 17 / 43
Duality by Example max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . η : maximal possible value of target function. 1 . . Any feasible solution ⇒ a lower bound on η . 2 . . In above: x 1 = 1 , x 2 = x 3 = 0 is feasible, and implies z = 4 3 and thus η ≥ 4 . . . x 1 = x 2 = 0 , x 3 = 3 is feasible = ⇒ η ≥ z = 9 . 4 . . How close this solution is to opt? (i.e., η ) 5 . . If very close to optimal – might be good enough. Maybe stop? 6 Sariel (UIUC) CS573 17 Fall 2013 17 / 43
Duality by Example max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . η : maximal possible value of target function. 1 . . Any feasible solution ⇒ a lower bound on η . 2 . . In above: x 1 = 1 , x 2 = x 3 = 0 is feasible, and implies z = 4 3 and thus η ≥ 4 . . . x 1 = x 2 = 0 , x 3 = 3 is feasible = ⇒ η ≥ z = 9 . 4 . . How close this solution is to opt? (i.e., η ) 5 . . If very close to optimal – might be good enough. Maybe stop? 6 Sariel (UIUC) CS573 17 Fall 2013 17 / 43
Duality by Example: II max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . Add the first inequality (multiplied by 2) to the second inequality 1 (multiplied by 3): 2( x 1 + 4 x 2 ) ≤ 2(1) +3(3 x 1 − x 2 + x 3 ) ≤ 3(3) . . . The resulting inequality is 2 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 . (1) Sariel (UIUC) CS573 18 Fall 2013 18 / 43
Duality by Example: II max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . Add the first inequality (multiplied by 2) to the second inequality 1 (multiplied by 3): 2( x 1 + 4 x 2 ) ≤ 2(1) +3(3 x 1 − x 2 + x 3 ) ≤ 3(3) . . . The resulting inequality is 2 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 . (1) Sariel (UIUC) CS573 18 Fall 2013 18 / 43
Duality by Example: II max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . got 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 . 1 . . inequality must hold for any feasible solution of L . 2 . . Objective: z = 4 x 1 + x 2 + 3 x 3 and x 1 , x 2 and x 3 are all 3 non-negative. . . Inequality above has larger coefficients than objective (for 4 corresponding variables) . . For any feasible solution: 5 z = 4 x 1 + x 2 + 3 x 3 ≤ 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 , Sariel (UIUC) CS573 19 Fall 2013 19 / 43
Duality by Example: II max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . got 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 . 1 . . inequality must hold for any feasible solution of L . 2 . . Objective: z = 4 x 1 + x 2 + 3 x 3 and x 1 , x 2 and x 3 are all 3 non-negative. . . Inequality above has larger coefficients than objective (for 4 corresponding variables) . . For any feasible solution: 5 z = 4 x 1 + x 2 + 3 x 3 ≤ 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 , Sariel (UIUC) CS573 19 Fall 2013 19 / 43
Duality by Example: II max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . got 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 . 1 . . inequality must hold for any feasible solution of L . 2 . . Objective: z = 4 x 1 + x 2 + 3 x 3 and x 1 , x 2 and x 3 are all 3 non-negative. . . Inequality above has larger coefficients than objective (for 4 corresponding variables) . . For any feasible solution: 5 z = 4 x 1 + x 2 + 3 x 3 ≤ 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 , Sariel (UIUC) CS573 19 Fall 2013 19 / 43
Duality by Example: II max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . got 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 . 1 . . inequality must hold for any feasible solution of L . 2 . . Objective: z = 4 x 1 + x 2 + 3 x 3 and x 1 , x 2 and x 3 are all 3 non-negative. . . Inequality above has larger coefficients than objective (for 4 corresponding variables) . . For any feasible solution: 5 z = 4 x 1 + x 2 + 3 x 3 ≤ 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 , Sariel (UIUC) CS573 19 Fall 2013 19 / 43
Duality by Example: II max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . got 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 . 1 . . inequality must hold for any feasible solution of L . 2 . . Objective: z = 4 x 1 + x 2 + 3 x 3 and x 1 , x 2 and x 3 are all 3 non-negative. . . Inequality above has larger coefficients than objective (for 4 corresponding variables) . . For any feasible solution: 5 z = 4 x 1 + x 2 + 3 x 3 ≤ 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 , Sariel (UIUC) CS573 19 Fall 2013 19 / 43
Duality by Example: III max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . For any feasible solution: 1 z = 4 x 1 + x 2 + 3 x 3 ≤ 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 , . . Opt solution is LP L is somewhere between 9 and 11 . 2 . . Multiply first inequality by y 1 , second inequality by y 2 and add 3 them up: y 1 ( x 1 + 4 x 2 ) ≤ y 1 (1) + y 2 (3 x 1 - + ) ≤ y 2 (3) x 2 x 3 ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + ≤ y 1 + 3 y 2 . y 2 x 3 Sariel (UIUC) CS573 20 Fall 2013 20 / 43
Duality by Example: III max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . For any feasible solution: 1 z = 4 x 1 + x 2 + 3 x 3 ≤ 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 , . . Opt solution is LP L is somewhere between 9 and 11 . 2 . . Multiply first inequality by y 1 , second inequality by y 2 and add 3 them up: y 1 ( x 1 + 4 x 2 ) ≤ y 1 (1) + y 2 (3 x 1 - + ) ≤ y 2 (3) x 2 x 3 ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + ≤ y 1 + 3 y 2 . y 2 x 3 Sariel (UIUC) CS573 20 Fall 2013 20 / 43
Duality by Example: III max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . For any feasible solution: 1 z = 4 x 1 + x 2 + 3 x 3 ≤ 11 x 1 + 5 x 2 + 3 x 3 ≤ 11 , . . Opt solution is LP L is somewhere between 9 and 11 . 2 . . Multiply first inequality by y 1 , second inequality by y 2 and add 3 them up: y 1 ( x 1 + 4 x 2 ) ≤ y 1 (1) + y 2 (3 x 1 - + ) ≤ y 2 (3) x 2 x 3 ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + ≤ y 1 + 3 y 2 . y 2 x 3 Sariel (UIUC) CS573 20 Fall 2013 20 / 43
Duality by Example: IV max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 x 3 ≤ y 1 + 3 y 2 . 1 . . Compare to target function – 1 require expression bigger than target function in each variable. = ⇒ z = 4 x 1 + x 2 + 3 x 3 ≤ ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 x 3 ≤ y 1 + 3 y 2 , the last step follows by previous slide. Sariel (UIUC) CS573 21 Fall 2013 21 / 43
Duality by Example: IV max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 x 3 ≤ y 1 + 3 y 2 . 1 . . Compare to target function – 1 require expression bigger than target function in each variable. = ⇒ z = 4 x 1 + x 2 + 3 x 3 ≤ ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 x 3 ≤ y 1 + 3 y 2 , the last step follows by previous slide. Sariel (UIUC) CS573 21 Fall 2013 21 / 43
Duality by Example: IV max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 x 3 ≤ y 1 + 3 y 2 . 1 . . Compare to target function – 1 4 ≤ y 1 + 3 y 2 require expression bigger than 1 ≤ 4 y 1 − y 2 target function in each 3 ≤ y 2 , variable. = ⇒ z = 4 x 1 + x 2 + 3 x 3 ≤ ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 x 3 ≤ y 1 + 3 y 2 , the last step follows by previous slide. Sariel (UIUC) CS573 21 Fall 2013 21 / 43
Duality by Example: IV max z = 4 x 1 + x 2 + 3 x 3 s.t. x 1 + 4 x 2 ≤ 1 3 x 1 − x 2 + x 3 ≤ 3 x 1 , x 2 , x 3 ≥ 0 . ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 x 3 ≤ y 1 + 3 y 2 . 1 . . Compare to target function – 1 4 ≤ y 1 + 3 y 2 require expression bigger than 1 ≤ 4 y 1 − y 2 target function in each 3 ≤ y 2 , variable. = ⇒ z = 4 x 1 + x 2 + 3 x 3 ≤ ( y 1 + 3 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 x 3 ≤ y 1 + 3 y 2 , the last step follows by previous slide. Sariel (UIUC) CS573 21 Fall 2013 21 / 43
Duality by Example: IV Dual LP : � L Primal LP : min y 1 + 3 y 2 max z = 4 x 1 + x 2 + 3 x 3 s.t. y 1 + 3 y 2 ≥ 4 s.t. x 1 + 4 x 2 ≤ 1 4 y 1 − y 2 ≥ 1 3 x 1 − x 2 + x 3 ≤ 3 y 2 ≥ 3 x 1 , x 2 , x 3 ≥ 0 y 1 , y 2 ≥ 0 . . Best upper bound on η (max value of z ) then solve the LP � L . 1 . . � L : Dual program to L . 2 . . opt. solution of � L is an upper bound on optimal solution for L . 3 Sariel (UIUC) CS573 22 Fall 2013 22 / 43
Duality by Example: IV Dual LP : � L Primal LP : min y 1 + 3 y 2 max z = 4 x 1 + x 2 + 3 x 3 s.t. y 1 + 3 y 2 ≥ 4 s.t. x 1 + 4 x 2 ≤ 1 4 y 1 − y 2 ≥ 1 3 x 1 − x 2 + x 3 ≤ 3 y 2 ≥ 3 x 1 , x 2 , x 3 ≥ 0 y 1 , y 2 ≥ 0 . . Best upper bound on η (max value of z ) then solve the LP � L . 1 . . � L : Dual program to L . 2 . . opt. solution of � L is an upper bound on optimal solution for L . 3 Sariel (UIUC) CS573 22 Fall 2013 22 / 43
Duality by Example: IV Dual LP : � L Primal LP : min y 1 + 3 y 2 max z = 4 x 1 + x 2 + 3 x 3 s.t. y 1 + 3 y 2 ≥ 4 s.t. x 1 + 4 x 2 ≤ 1 4 y 1 − y 2 ≥ 1 3 x 1 − x 2 + x 3 ≤ 3 y 2 ≥ 3 x 1 , x 2 , x 3 ≥ 0 y 1 , y 2 ≥ 0 . . Best upper bound on η (max value of z ) then solve the LP � L . 1 . . � L : Dual program to L . 2 . . opt. solution of � L is an upper bound on optimal solution for L . 3 Sariel (UIUC) CS573 22 Fall 2013 22 / 43
Duality by Example: IV Dual LP : � L Primal LP : min y 1 + 3 y 2 max z = 4 x 1 + x 2 + 3 x 3 s.t. y 1 + 3 y 2 ≥ 4 s.t. x 1 + 4 x 2 ≤ 1 4 y 1 − y 2 ≥ 1 3 x 1 − x 2 + x 3 ≤ 3 y 2 ≥ 3 x 1 , x 2 , x 3 ≥ 0 y 1 , y 2 ≥ 0 . . Best upper bound on η (max value of z ) then solve the LP � L . 1 . . � L : Dual program to L . 2 . . opt. solution of � L is an upper bound on optimal solution for L . 3 Sariel (UIUC) CS573 22 Fall 2013 22 / 43
Primal program/Dual program m n ∑ ∑ min b i y i max c j x j i =1 j =1 m ∑ n ∑ s.t. a ij y i ≥ c j , s.t. a ij x j ≤ b i , i =1 j =1 for i = 1 , . . . , m , for j = 1 , . . . , n , x j ≥ 0 , y i ≥ 0 , for j = 1 , . . . , n . for i = 1 , . . . , m . Sariel (UIUC) CS573 23 Fall 2013 23 / 43
Primal program/Dual program c T x y T b max min y T A ≥ c T . s. t. Ax ≤ b . s. t. x ≥ 0 . y ≥ 0 . Sariel (UIUC) CS573 24 Fall 2013 24 / 43
Primal program/Dual program What happens when you take the dual of the dual? m n ∑ ∑ min max b i y i c j x j i =1 j =1 m ∑ n ∑ s.t. a ij y i ≥ c j , s.t. a ij x j ≤ b i , i =1 j =1 for i = 1 , . . . , m , for j = 1 , . . . , n , x j ≥ 0 , y i ≥ 0 , for j = 1 , . . . , n . for i = 1 , . . . , m . Sariel (UIUC) CS573 25 Fall 2013 25 / 43
Primal program / Dual program in standard form n ∑ m ∑ max c j x j max ( − b i ) y i j =1 i =1 n ∑ m ∑ s.t. a ij x j ≤ b i , s.t. ( − a ij ) y i ≤ − c j , j =1 i =1 for i = 1 , . . . , m , for j = 1 , . . . , n , x j ≥ 0 , y i ≥ 0 , for j = 1 , . . . , n . for i = 1 , . . . , m . Sariel (UIUC) CS573 26 Fall 2013 26 / 43
Dual program in standard form / Dual of dual program n ∑ m ∑ min − c j x j max ( − b i ) y i j =1 i =1 n ∑ m ∑ s.t. ( − a ij ) x j ≥ − b i , s.t. ( − a ij ) y i ≤ − c j , j =1 i =1 for i = 1 , . . . , m , for j = 1 , . . . , n , x j ≥ 0 , y i ≥ 0 , for j = 1 , . . . , n . for i = 1 , . . . , m . Sariel (UIUC) CS573 27 Fall 2013 27 / 43
Dual of dual program / Dual of dual program written in standard form n ∑ n ∑ min − c j x j max c j x j j =1 j =1 n ∑ n ∑ s.t. ( − a ij ) x j ≥ − b i , s.t. a ij x j ≤ b i , j =1 for i = 1 , . . . , m , j =1 for i = 1 , . . . , m , x j ≥ 0 , x j ≥ 0 , for j = 1 , . . . , n . for j = 1 , . . . , n . = ⇒ Dual of the dual LP is the primal LP ! Sariel (UIUC) CS573 28 Fall 2013 28 / 43
Dual of dual program / Dual of dual program written in standard form n ∑ n ∑ min − c j x j max c j x j j =1 j =1 n ∑ n ∑ s.t. ( − a ij ) x j ≥ − b i , s.t. a ij x j ≤ b i , j =1 for i = 1 , . . . , m , j =1 for i = 1 , . . . , m , x j ≥ 0 , x j ≥ 0 , for j = 1 , . . . , n . for j = 1 , . . . , n . = ⇒ Dual of the dual LP is the primal LP ! Sariel (UIUC) CS573 28 Fall 2013 28 / 43
Recommend
More recommend
