Linear Programming Illustration Courtesy: Kevin Wayne & Denis - - PowerPoint PPT Presentation

CSC373 Week 6: Linear Programming

373F19 - Nisarg Shah & Karan Singh 1

Illustration Courtesy: Kevin Wayne & Denis Pankratov

Recap

373F19 - Nisarg Shah & Karan Singh 2

• Network flow

➢ Ford-Fulkerson algorithm ➢ Ways to make the running time polynomial ➢ Correctness using max-flow, min-cut ➢ Applications:

• Edge-disjoint paths
• Multiple sources/sinks
• Circulation
• Circulation with lower bounds
• Survey design
• Image segmentation

Brewery Example

373F19 - Nisarg Shah & Karan Singh 3

• A brewery can invest its inventory of corn, hops

and malt into producing some amount of ale and some amount of beer

➢ Per unit resource requirement and profit of the two items

are as given below

Example Courtesy: Kevin Wayne

Brewery Example

373F19 - Nisarg Shah & Karan Singh 4

• Suppose it produces

𝐵 units of ale and 𝐶 units of beer

• Then we want to

solve this program:

Linear Function

373F19 - Nisarg Shah & Karan Singh 5

• 𝑔: ℝ𝑜 → ℝ is a linear function if 𝑔 𝑦 = 𝑏𝑈𝑦 for

some 𝑏 ∈ ℝ𝑜

➢ Example: 𝑔 𝑦1, 𝑦2 = 3𝑦1 − 5𝑦2 =

3 −5 𝑈 𝑦1 𝑦2

• Linear constraints:

➢ For a linear function 𝑕: ℝ𝑜 → ℝ and 𝑑 ∈ ℝ, 𝑕 𝑦 = 𝑑 ➢ Line in the plane (or a hyperplane in ℝ𝑜) ➢ Example: 5𝑦1 + 7𝑦2 = 10

Linear Function

373F19 - Nisarg Shah & Karan Singh 6

• Geometrically, 𝑏 is the normal vector of the line(or

hyperplane) represented by 𝑏𝑈𝑦 = 𝑑

Linear Inequality

373F19 - Nisarg Shah & Karan Singh 7

• 𝑏𝑈𝑦 ≤ 𝑑 represents a “half-space”

Linear Programming

373F19 - Nisarg Shah & Karan Singh 8

• Maximize/minimize a linear function subject to

linear equality/inequality constraints

Geometrically…

373F19 - Nisarg Shah & Karan Singh 9

Back to Brewery Example

373F19 - Nisarg Shah & Karan Singh 10

Back to Brewery Example

373F19 - Nisarg Shah & Karan Singh 11

• Claim: Regardless of the objective function, the
• ptimal solution must be at a vertex

Optimal Solution At A Vertex

373F19 - Nisarg Shah & Karan Singh 12

Convexity

373F19 - Nisarg Shah & Karan Singh 13

• Convex set 𝑇: If 𝑦, 𝑧 ∈ 𝑇 and 𝜇 ∈ [0,1], then 𝜇𝑦 +

1 − 𝜇 𝑧 ∈ 𝑇 too.

• Vertex: A point which cannot be written as a strict

convex combination of any two points in the set

• Observation: Feasible region of an LP is a convex set

Optimal Solution At A Vertex

373F19 - Nisarg Shah & Karan Singh 14

• Proof intuition:

➢ If 𝑦 is not a vertex, we can move towards the boundary in a direction

where the objective value does not decrease

• Take some direction 𝑒 such that you can move by at least 𝜗 in both 𝑒

and −𝑒 directions while remaining within the region

• Objective must not decrease in at least one of {𝑒, −𝑒} directions

➢ Reach a point that is “tight” for at least one more constraint ➢ Repeat until we are at a vertex

LP, Standard Formulation

373F19 - Nisarg Shah & Karan Singh 15

• Input: 𝑑, 𝑏1, 𝑏2, … , 𝑏𝑛 ∈ ℝ𝑜, 𝑐 ∈ ℝ𝑛

➢ There are 𝑜 variables and 𝑛 constraints

• Goal:

LP, Standard Matrix Form

373F19 - Nisarg Shah & Karan Singh 16

• Input: 𝑑, 𝑏1, 𝑏2, … , 𝑏𝑛 ∈ ℝ𝑜, 𝑐 ∈ ℝ𝑛

➢ There are 𝑜 variables and 𝑛 constraints

• Goal:

Convert to Standard Form

373F19 - Nisarg Shah & Karan Singh 17

• What if the LP is not in standard form?

➢ Constraints that use ≥

• 𝑏𝑈𝑦 ≥ 𝑐 ⇔ −𝑏𝑈𝑦 ≤ −𝑐

➢ Constraints that use equality

• 𝑏𝑈𝑦 = 𝑐 ⇔ 𝑏𝑈𝑦 ≤ 𝑐, 𝑏𝑈𝑦 ≥ 𝑐

➢ Objective function is a minimization

• Minimize 𝑑𝑈𝑦 ⇔ Maximize −𝑑𝑈𝑦

➢ Variable is unconstrained

• 𝑦 with no constraint ⇔ Replace 𝑦 by two variables 𝑦′and 𝑦′′,

replace every occurrence of 𝑦 with 𝑦′ − 𝑦′′, and add constraints 𝑦′ ≥ 0, 𝑦′′ ≥ 0

LP Transformation Example

373F19 - Nisarg Shah & Karan Singh 18

Optimal Solution

373F19 - Nisarg Shah & Karan Singh 19

• Does this LP always have an optimal solution?
• No! The LP can fail for two reasons
• 1. It is infeasible, i.e. 𝑦 𝐵𝑦 ≤ 𝑐} = ∅
• Example: 𝑦1 ≤ 1 and 𝑦1 ≥ 2 (or −𝑦1≤ −2) constraints
• 2. It is unbounded, i.e. you can get arbitrarily large or small
• bjective values
• Example: maximize 𝑦1 subject to 𝑦1 ≥ 0
• We know that if the LP has an optimal solution, it

must be at a vertex.

Simplex Algorithm

373F19 - Nisarg Shah & Karan Singh 20

• Simple algorithm, easy to specify geometrically
• Worst-case running time is exponential
• Excellent performance in practice

Simplex: Geometric View

373F19 - Nisarg Shah & Karan Singh 21

Algorithmic Implementation

373F19 - Nisarg Shah & Karan Singh 22

Start at a vertex of feasible polytope Move to a neighbor vertex with better

• bjective value

Terminate, declare the current solution and value as optimal Is there a neighbor vertex with better

• bjective value?

How Do We Implement This?

373F19 - Nisarg Shah & Karan Singh 23

• We’ll work with the slack form of LP

➢ Convenient for implementing simplex operations ➢ We want to maximize 𝑨 in the slack form, but for now,

forget about the maximization objective

Slack Form

373F19 - Nisarg Shah & Karan Singh 24

Slack Form

373F19 - Nisarg Shah & Karan Singh 25

Simplex: Step 1

373F19 - Nisarg Shah & Karan Singh 26

• Start at a feasible vertex

➢ How do we find a feasible vertex? ➢ For now, assume 𝑐 ≥ 0 (each 𝑐𝑗 ≥ 0)

• In this case, 𝑦 = 0 is a feasible vertex.
• In the slack form, this means setting the nonbasic variables to 0

➢ We’ll later see what to do in the general case

Simple: Step 2

373F19 - Nisarg Shah & Karan Singh 27

• What next? Let’s look at an example
• To increase the value of 𝑨:

➢ Find a nonbasic variable with a positive coefficient

• This is called an entering variable

➢ See how much you can increase its value without

violating any constraints

Simple: Step 2

373F19 - Nisarg Shah & Karan Singh 28

This is because the current values of 𝑦2 and 𝑦3 are 0, and we need 𝑦4, 𝑦5, 𝑦6 ≥ 0

Simple: Step 2

373F19 - Nisarg Shah & Karan Singh 29

Tightest obstacle

➢ Solve the tightest obstacle for the nonbasic variable

𝑦1 = 9 − 𝑦2 4 − 𝑦3 2 − 𝑦6 4

• Substitute the entering variable (called pivot) in other equations
• Now 𝑦1 becomes basic and 𝑦6 becomes non-basic
• 𝑦6 is called the leaving variable

Simplex: Step 2

373F19 - Nisarg Shah & Karan Singh 30

• After one iteration of this step:

➢ The basic feasible solution (i.e. substituting 0 for all

nonbasic variables) improves from 𝑨 = 0 to 𝑨 = 27

• Repeat!

Simplex: Step 2

373F19 - Nisarg Shah & Karan Singh 31

Simplex: Step 2

373F19 - Nisarg Shah & Karan Singh 32

Simplex: Step 2

373F19 - Nisarg Shah & Karan Singh 33

• There is no leaving variable (nonbasic variable with positive coefficient).
• What now? Nothing! We are done.
• Take the basic feasible solution (𝑦3 = 𝑦5 = 𝑦6 = 0).
• Gives the optimal value 𝑨 = 28
• In the optimal solution, 𝑦1 = 8, 𝑦2 = 4, 𝑦3 = 0

Simplex Overview

373F19 - Nisarg Shah & Karan Singh 34

Start at a vertex of feasible polytope Move to a neighbor vertex with better

• bjective value

Terminate, declare the current solution and value as optimal Is there a neighbor vertex with better

• bjective value?

Simplex Overview

373F19 - Nisarg Shah & Karan Singh 35

Assuming 𝑐 ≥ 0, start with a basic feasible solution

Move to a neighbor vertex with better

• bjective value

Terminate, declare the current solution and value as optimal Is there a neighbor vertex with better

• bjective value?

Simplex Overview

373F19 - Nisarg Shah & Karan Singh 36

Assuming 𝑐 ≥ 0, start with a basic feasible solution

Move to a neighbor vertex with better

• bjective value

Terminate, declare the current solution and value as optimal

Is there a leaving variable? (coefficient > 0 in 𝑨)

Simplex Overview

373F19 - Nisarg Shah & Karan Singh 37

Assuming 𝑐 ≥ 0, start with a basic feasible solution

Pivot on a leaving variable Terminate, declare the current solution and value as optimal

Is there a leaving variable? (coefficient > 0 in 𝑨)

Simplex Overview

373F19 - Nisarg Shah & Karan Singh 38

Assuming 𝑐 ≥ 0, start with a basic feasible solution

Pivot on a leaving variable Terminate, declare

• ptimal value

Is there a leaving variable? (coefficient > 0 in 𝑨)

Some Outstanding Issues

373F19 - Nisarg Shah & Karan Singh 39

• What if the entering variable has no upper bound?

➢ If it doesn’t appear in any constraints, or only appears in

constraints where it can go to ∞

➢ Then 𝑨 can also go to ∞, so declare that LP is unbounded

• What if pivoting doesn’t change the constant in 𝑨?

➢ Known as degeneracy, and can lead to infinite loops ➢ Can be prevented by “perturbing” 𝑐 by a small random

amount in each coordinate

➢ Or by carefully breaking ties among entering and leaving

variables, e.g., by smallest index (known as Bland’s rule)

Some Outstanding Issues

373F19 - Nisarg Shah & Karan Singh 40

• We assumed 𝑐 ≥ 0, and then started with the

vertex 𝑦 = 0

• What if this assumption does not hold?

𝑀𝑄

1

Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 ≤ 𝑐1

𝑏2

𝑈𝑦 ≤ 𝑐2

⋮ 𝑏𝑛

𝑈 𝑦 ≤ 𝑐𝑛

𝑦 ≥ 0 𝑀𝑄2 Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 + 𝑡1 = 𝑐1

𝑏2

𝑈𝑦 + 𝑡2 = 𝑐2

⋮ 𝑏𝑛

𝑈 𝑦 + 𝑡𝑛 = 𝑐𝑛

𝑦, 𝑡 ≥ 0 𝑀𝑄3 Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 + 𝑡1 = 𝑐1

−𝑏2

𝑈𝑦 − 𝑡2 = −𝑐2

⋮ −𝑏𝑛

𝑈 𝑦 − 𝑡𝑛 = −𝑐𝑛

𝑦, 𝑡 ≥ 0

Multiply every constraint with negative 𝑐𝑗 by − 1 so RHS is now positive

Some Outstanding Issues

373F19 - Nisarg Shah & Karan Singh 41

• We assumed 𝑐 ≥ 0, and then started with the

vertex 𝑦 = 0

• What if this assumption does not hold?

𝑀𝑄3 Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 + 𝑡1 = 𝑐1

−𝑏2

𝑈𝑦 − 𝑡2 = −𝑐2

⋮ −𝑏𝑛

𝑈 𝑦 − 𝑡𝑛 = −𝑐𝑛

𝑦, 𝑡 ≥ 0

Remember: the RHS is now positive

𝑀𝑄

4

Min σ𝑗 𝑨𝑗 s.t. 𝑏1

𝑈𝑦 + 𝑡1 + 𝑨1 = 𝑐1

−𝑏2

𝑈𝑦 − 𝑡2 + 𝑨2 = −𝑐2

⋮ −𝑏𝑛

𝑈 𝑦 − 𝑡𝑛 + 𝑨𝑛 = −𝑐𝑛

𝑦, 𝑡, 𝑨 ≥ 0

Some Outstanding Issues

373F19 - Nisarg Shah & Karan Singh 42

• We assumed 𝑐 ≥ 0, and then started with the

vertex 𝑦 = 0

• What if this assumption does not hold?

Remember: the RHS is now positive

𝑀𝑄

4

Min σ𝑗 𝑨𝑗 s.t. 𝑏1

𝑈𝑦 + 𝑡1 + 𝑨1 = 𝑐1

−𝑏2

𝑈𝑦 − 𝑡2 + 𝑨2 = −𝑐2

⋮ −𝑏𝑛

𝑈 𝑦 − 𝑡𝑛 + 𝑨𝑛 = −𝑐𝑛

𝑦, 𝑡, 𝑨 ≥ 0

What now?

• Solve 𝑀𝑄

4 using simplex with

the initial basic solution being 𝑦 = 𝑡 = 0, 𝑨 = 𝑐

• If its optimum value is 0,

extract a basic feasible solution 𝑦∗ from it, use it to solve 𝑀𝑄

1 using simplex

• If optimum value for 𝑀𝑄

4 is

greater than 0, then 𝑀𝑄

1 is

infeasible

Some Outstanding Issues

373F19 - Nisarg Shah & Karan Singh 43

• We assumed 𝑐 ≥ 0, and then started with the

vertex 𝑦 = 0

• What if this assumption does not hold?

𝑀𝑄

1

Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 ≤ 𝑐1

𝑏2

𝑈𝑦 ≤ 𝑐2

⋮ 𝑏𝑛

𝑈 𝑦 ≤ 𝑐𝑛

𝑦 ≥ 0 𝑀𝑄2 Min σ𝑗 𝑨𝑗 s.t. 𝑏1

𝑈𝑦 + 𝑡1 + 𝑨1 = 𝑐1

𝑏2

𝑈𝑦 + 𝑡2 + 𝑨2 = 𝑐2

⋮ 𝑏𝑛

𝑈 𝑦 + 𝑡𝑛 + 𝑨𝑛 = 𝑐𝑛

𝑦, 𝑡 ≥ 0

• Solve 𝑀𝑄2 using simplex with

the initial basic feasible solution 𝑦 = 𝑡 = 0, 𝑨 = 𝑐

• If its optimum value is 0,

extract a basic feasible solution 𝑦∗ from it, use it to solve 𝑀𝑄

1 using simplex

• If optimum value for 𝑀𝑄2 is

greater than 0, then 𝑀𝑄

1 is

infeasible

Some Outstanding Issues

373F19 - Nisarg Shah & Karan Singh 44

• Pseudocode? Proof of correctness? Running time

analysis?

• See textbook for details!

Running Time

373F19 - Nisarg Shah & Karan Singh 45

• Notes

➢ Number of vertices of a polytope can be exponential in

the number of constraints

• There are examples where simplex takes exponential time if you

choose your pivots arbitrarily

• No pivot rule known which guarantees polynomial running time

➢ There are other algorithms which run in polynomial time

• Ellipsoid method, interior point method, …
• Ellipsoid uses 𝑃(𝑛𝑜3𝑀) arithmetic operations, where 𝑀 = length of

input

• But no known strongly polynomial time algorithm
• Number of arithmetic operations = poly(m,n)

Certificate of Optimality

373F19 - Nisarg Shah & Karan Singh 46

• Suppose you design a state-of-the-art LP solver

that can solve very large problem instances

• You want to convince someone that you have this

new technology without showing them the code

➢ Idea: They can give you very large LPs and you can quickly

return the optimal solutions

➢ Question: But how would they know that your solutions

are optimal, if they don’t have the technology to solve those LPs?

Certificate of Optimality

373F19 - Nisarg Shah & Karan Singh 47

• Suppose I tell you that 𝑦1, 𝑦2 = (100,300) is
• ptimal with objective value 1900
• How can you check this?

➢ Note: Can easily substitute (𝑦1, 𝑦2), and verify that it is

feasible, and its objective value is indeed 1900

Certificate of Optimality

373F19 - Nisarg Shah & Karan Singh 48

• Any solution that satisfies these inequalities also

satisfies their positive combinations

➢ E.g. 2*first_constraint + 5*second_constraint +

3*third_constraint

➢ Try to take combinations which give you 𝑦1 + 6𝑦2 on LHS

• Claim: 𝑦1, 𝑦2 = (100,300) is
• ptimal with objective value 1900

Certificate of Optimality

373F19 - Nisarg Shah & Karan Singh 49

• first_constraint + 6*second_constraint

➢ 𝑦1 + 6𝑦2 ≤ 200 + 6 ∗ 300 = 2000 ➢ This shows that no feasible solution can beat 2000

• Claim: 𝑦1, 𝑦2 = (100,300) is
• ptimal with objective value 1900

Certificate of Optimality

373F19 - Nisarg Shah & Karan Singh 50

• 5*second_constraint + third_constraint

➢ 5𝑦2 + 𝑦1 + 𝑦2 ≤ 5 ∗ 300 + 400 = 1900 ➢ This shows that no feasible solution can beat 1900

• No need to proceed further
• We already know one solution that achieves 1900, so it must be
• ptimal!
• Claim: 𝑦1, 𝑦2 = (100,300) is
• ptimal with objective value 1900

Is There a General Algorithm?

373F19 - Nisarg Shah & Karan Singh 51

• Introduce variables 𝑧1, 𝑧2, 𝑧3 by which we will be

multiplying the three constraints

➢ Note: These need not be integers. They can be reals.

• After multiplying and adding constraints, we get:

𝑧1 + 𝑧3 𝑦1 + 𝑧2 + 𝑧3 𝑦2 ≤ 200𝑧1 + 300𝑧2 + 400𝑧3

Is There a General Algorithm?

373F19 - Nisarg Shah & Karan Singh 52

➢ We have:

𝑧1 + 𝑧3 𝑦1 + 𝑧2 + 𝑧3 𝑦2 ≤ 200𝑧1 + 300𝑧2 + 400𝑧3

➢ What do we want?

• 𝑧1, 𝑧2, 𝑧3 ≥ 0 because otherwise direction of inequality flips
• LHS to look like objective 𝑦1 + 6𝑦2
• In fact, it is sufficient for LHS to be an upper bound on objective
• So we want 𝑧1 + 𝑧3 ≥ 1 and 𝑧2 + 𝑧3 ≥ 6

Is There a General Algorithm?

373F19 - Nisarg Shah & Karan Singh 53

➢ We have:

𝑧1 + 𝑧3 𝑦1 + 𝑧2 + 𝑧3 𝑦2 ≤ 200𝑧1 + 300𝑧2 + 400𝑧3

➢ What do we want?

• 𝑧1, 𝑧2, 𝑧3 ≥ 0
• 𝑧1 + 𝑧3 ≥ 1, 𝑧2 + 𝑧3 ≥ 6
• Subject to these, we want to minimize the upper bound 200𝑧1 +

300𝑧2 + 400𝑧3

Is There a General Algorithm?

373F19 - Nisarg Shah & Karan Singh 54

➢ We have:

𝑧1 + 𝑧3 𝑦1 + 𝑧2 + 𝑧3 𝑦2 ≤ 200𝑧1 + 300𝑧2 + 400𝑧3

➢ What do we want?

• This is just another LP!
• Called the dual
• Original LP is called the primal

Is There a General Algorithm?

373F19 - Nisarg Shah & Karan Singh 55

➢ The problem of verifying optimality is another LP

• For any 𝑧1, 𝑧2, 𝑧3 that you can find, the objective value of the

dual is an upper bound on the objective value of the primal

• If you found a specific 𝑧1, 𝑧2, 𝑧3 for which this dual objective

becomes equal to the primal objective for the (𝑦1, 𝑦2) given to you, then you would know that the given 𝑦1, 𝑦2 is optimal for primal (and your (𝑧1, 𝑧2, 𝑧3) is optimal for dual)

Is There a General Algorithm?

373F19 - Nisarg Shah & Karan Singh 56

➢ The problem of verifying optimality is another LP

• Issue 1: But…but…if I can’t solve large LPs, how will I solve the dual

to verify if optimality of (𝑦1, 𝑦2) given to me?

• You don’t. Ask the other party to give you both (𝑦1, 𝑦2) and the

corresponding 𝑧1, 𝑧2, 𝑧3 for proof of optimality

• Issue 2: What if there are no (𝑧1, 𝑧2, 𝑧3) for which dual objective

matches primal objective under optimal solution (𝑦1, 𝑦2)?

• This can’t happen!

Is There a General Algorithm?

373F19 - Nisarg Shah & Karan Singh 57

Primal LP Dual LP

➢ General version, in our standard form for LPs ➢ Recap:

• 𝑑𝑈𝑦 for any feasible 𝑦 ≤ 𝑧𝑈𝑐 for any feasible 𝑧
• max

primal feasible 𝑦 𝑑𝑈𝑦 ≤

min

dual feasible 𝑧 𝑧𝑈𝑐

• If there are (𝑦∗, 𝑧∗) with 𝑑𝑈𝑦∗ = 𝑧∗ 𝑈𝑐, then both are optimal
• In fact, for optimal 𝑦∗, 𝑧∗ , we are claiming this must happen!
• Does this remind you of something? Max-flow, min-cut…

Weak Duality

373F19 - Nisarg Shah & Karan Singh 58

• From here on, we assume that primal LP is feasible and also

not unbounded

• Weak duality theorem:

➢ For any primal feasible 𝑦 and dual feasible 𝑧, 𝑑𝑈𝑦 ≤ 𝑧𝑈𝑐

• Proof:

𝑑𝑈𝑦 ≤ 𝑧𝑈𝐵 𝑦 = 𝑧𝑈 𝐵𝑦 ≤ 𝑧𝑈𝑐 Primal LP Dual LP

Strong Duality

373F19 - Nisarg Shah & Karan Singh 59

• Strong duality theorem:

➢ For any primal optimal 𝑦∗ and dual optimal 𝑧∗, 𝑑𝑈𝑦∗ = 𝑧∗ 𝑈𝑐

Primal LP Dual LP

Strong Duality Proof

373F19 - Nisarg Shah & Karan Singh 60

• Farkas’ lemma (one of many, many versions):

➢ Exactly one of the following holds:

1) There exists 𝑦 such that 𝐵𝑦 ≤ 𝑐 2) There exists 𝑧 such that 𝑧𝑈𝐵 = 0, 𝑧 ≥ 0, 𝑧𝑈𝑐 < 0

• Geometric intuition:

➢ Define image of 𝐵 = set of all possible values of 𝐵𝑦 ➢ It is known that this is a “linear subspace” (e.g. a line in a

plane, a line or plane in 3D, etc)

This slide is not in the scope of the course

Strong Duality Proof

373F19 - Nisarg Shah & Karan Singh 61

• Farkas’ lemma: Exactly one of the following holds:

1) There exists 𝑦 such that 𝐵𝑦 ≤ 𝑐 2) There exists 𝑧 such that 𝑧𝑈𝐵 = 0, 𝑧 ≥ 0, 𝑧𝑈𝑐 < 0

1) Image of 𝐵 contains a point “below” 𝑐 2) The region “below” 𝑐 doesn’t intersect image of 𝐵 this is witnessed by normal vector to the image of 𝐵

This slide is not in the scope of the course

Strong Duality

373F19 - Nisarg Shah & Karan Singh 62

• Strong duality theorem:

➢ For any primal optimal 𝑦∗ and dual optimal 𝑧∗, 𝑑𝑈𝑦∗ = 𝑧∗ 𝑈𝑐 ➢ Proof (by contradiction):

• Suppose optimal dual objective value > 𝑨∗
• Let 𝑨∗ = 𝑑𝑈𝑦∗ be the optimal primal value. By weak duality, there is no 𝑧

such that 𝑧𝑈𝐵 ≥ 𝑑𝑈 and 𝑧𝑈𝑐 ≤ 𝑨∗, i.e., there is no 𝑧 such that

Primal LP Dual LP

This slide is not in the scope of the course

Strong Duality

373F19 - Nisarg Shah & Karan Singh 63

➢ There is no 𝑧 such that ➢ By Farkas’ lemma, there is 𝑦 and 𝜇 such that ➢ Case 1: 𝜇 > 0

• Note: 𝑑𝑈𝑦 > 𝜇𝑨∗ and 𝐵𝑦 = 0 = 𝜇𝑐.
• Divide both by 𝜇 to get 𝐵

𝑦 𝜇 = 𝑐 and 𝑑𝑈 𝑦 𝜇 > 𝑨∗

• Contradicts optimality of 𝑨∗

➢ Case 2: 𝜇 = 0

• We have 𝐵𝑦 = 0 and 𝑑𝑈𝑦 > 0
• Adding 𝑦 to optimal 𝑦∗ of primal improves objective value beyond

𝑨∗ ⇒ contradiction

This slide is not in the scope of the course

373F19 - Nisarg Shah & Karan Singh 64