Linear Programming Illustration Courtesy: Kevin Wayne & Denis - - PowerPoint PPT Presentation

CSC373 Week 6: Linear Programming

373F20 - Nisarg Shah 1

Illustration Courtesy: Kevin Wayne & Denis Pankratov

Announcement

373F20 - Nisarg Shah 2

• ACM ICPC Qualification Round
• Oct 24, 3-8pm EST
• Sign up at: https://www.teach.cs.toronto.edu/~acm/
• Top 9 participants will be chosen to represent U of T at the

regional contest (broken into three teams of 3 each)

Recap

373F20 - Nisarg Shah 3

• Network flow

➢ Ford-Fulkerson algorithm

• Ways to make the running time polynomial

➢ Correctness using max-flow, min-cut ➢ Applications:

• Edge-disjoint paths
• Multiple sources/sinks
• Circulation
• Circulation with lower bounds
• Survey design
• Image segmentation
• Profit maximization

Brewery Example

373F20 - Nisarg Shah 4

• A brewery can invest its inventory of corn, hops and malt

into producing some amount of ale and some amount of beer

➢ Per unit resource requirement and profit of the two items are as

given below

Example Courtesy: Kevin Wayne

Brewery Example

373F20 - Nisarg Shah 5

• Suppose it produces 𝐵

units of ale and 𝐶 units

• f beer
• Then we want to solve

this program:

Linear Function

373F20 - Nisarg Shah 6

• 𝑔: ℝ𝑜 → ℝ is a linear function if 𝑔 𝑦 = 𝑏𝑈𝑦 for some 𝑏 ∈ ℝ𝑜

➢ Example: 𝑔 𝑦1, 𝑦2 = 3𝑦1 − 5𝑦2 =

3 −5 𝑈 𝑦1 𝑦2

• Linear objective: 𝑔
• Linear constraints:

➢ 𝑕 𝑦 = 𝑑, where 𝑕: ℝ𝑜 → ℝ is a linear function and 𝑑 ∈ ℝ ➢ Line in the plane (or a hyperplane in ℝ𝑜) ➢ Example: 5𝑦1 + 7𝑦2 = 10

Linear Function

373F20 - Nisarg Shah 7

• Geometrically, 𝑏 is the normal vector of the line(or

hyperplane) represented by 𝑏𝑈𝑦 = 𝑑

Linear Inequality

373F20 - Nisarg Shah 8

• 𝑏𝑈𝑦 ≤ 𝑑 represents a “half-space”

Linear Programming

373F20 - Nisarg Shah 9

• Maximize/minimize a linear function subject to linear

equality/inequality constraints

Geometrically…

373F20 - Nisarg Shah 10

Back to Brewery Example

373F20 - Nisarg Shah 11

Back to Brewery Example

373F20 - Nisarg Shah 12

• Claim: Regardless of the objective function, there must be a

vertex that is an optimal solution

Optimal Solution At A Vertex

373F20 - Nisarg Shah 13

Convexity

373F20 - Nisarg Shah 14

• Convex set: 𝑇 is convex if

𝑦, 𝑧 ∈ 𝑇, 𝜇 ∈ [0,1] ⇒ 𝜇𝑦 + 1 − 𝜇 𝑧 ∈ 𝑇

• Vertex: A point which cannot be written as a strict convex

combination of any two points in the set

• Observation: Feasible region of an LP is a convex set

Optimal Solution At A Vertex

373F20 - Nisarg Shah 15

• Intuitive proof of the claim:

➢ Start at some point 𝑦 in the feasible region ➢ If 𝑦 is not a vertex:

• Find a direction 𝑒 such that points within a positive distance of 𝜗 from 𝑦 in

both 𝑒 and −𝑒 directions are within the feasible region

• Objective must not decrease in at least one of the two directions
• Follow that direction until you reach a new point 𝑦 for which at least one

more constraint is “tight”

➢ Repeat until we are at a vertex

LP, Standard Formulation

373F20 - Nisarg Shah 16

• Input: 𝑑, 𝑏1, 𝑏2, … , 𝑏𝑛 ∈ ℝ𝑜, 𝑐 ∈ ℝ𝑛

➢ There are 𝑜 variables and 𝑛 constraints

• Goal:

LP, Standard Matrix Form

373F20 - Nisarg Shah 17

• Input: 𝑑, 𝑏1, 𝑏2, … , 𝑏𝑛 ∈ ℝ𝑜, 𝑐 ∈ ℝ𝑛

➢ There are 𝑜 variables and 𝑛 constraints

• Goal:

Convert to Standard Form

373F20 - Nisarg Shah 18

• What if the LP is not in standard form?

➢ Constraints that use ≥

• 𝑏𝑈𝑦 ≥ 𝑐 ⇔ −𝑏𝑈𝑦 ≤ −𝑐

➢ Constraints that use equality

• 𝑏𝑈𝑦 = 𝑐 ⇔ 𝑏𝑈𝑦 ≤ 𝑐, 𝑏𝑈𝑦 ≥ 𝑐

➢ Objective function is a minimization

• Minimize 𝑑𝑈𝑦 ⇔ Maximize −𝑑𝑈𝑦

➢ Variable is unconstrained

• 𝑦 with no constraint ⇔ Replace 𝑦 by two variables 𝑦′and 𝑦′′, replace

every occurrence of 𝑦 with 𝑦′ − 𝑦′′, and add constraints 𝑦′ ≥ 0, 𝑦′′ ≥ 0

LP Transformation Example

373F20 - Nisarg Shah 19

Optimal Solution

373F20 - Nisarg Shah 20

• Does an LP always have an optimal solution?
• No! The LP can “fail” for two reasons:
• 1. It is infeasible, i.e. 𝑦 𝐵𝑦 ≤ 𝑐} = ∅
• E.g. the set of constraints is 𝑦1 ≤ 1, −𝑦1≤ −2
• 2. It is unbounded, i.e. the objective function can be made arbitrarily

large (for maximization) or small (for minimization)

• E.g. “maximize 𝑦1 subject to 𝑦1 ≥ 0”
• But if the LP has an optimal solution, we know that there

must be a vertex which is optimal

Simplex Algorithm

373F20 - Nisarg Shah 21

• Simple algorithm, easy to specify geometrically
• Worst-case running time is exponential
• Excellent performance in practice

Simplex: Geometric View

373F20 - Nisarg Shah 22

Algorithmic Implementation

373F20 - Nisarg Shah 23

Start at a vertex of feasible polytope Move to a neighbor vertex with better

• bjective value

Terminate, declare the current solution and value as optimal Is there a neighbor vertex with better

• bjective value?

How Do We Implement This?

373F20 - Nisarg Shah 24

• We’ll work with the slack form of LP

➢ Convenient for implementing simplex operations ➢ We want to maximize 𝑨 in the slack form, but for now, forget about

the maximization objective

Slack Form

373F20 - Nisarg Shah 25

Slack Form

373F20 - Nisarg Shah 26

Simplex: Step 1

373F20 - Nisarg Shah 27

• Start at a feasible vertex

➢ How do we find a feasible vertex? ➢ For now, assume 𝑐 ≥ 0 (each 𝑐𝑗 ≥ 0)

• In this case, 𝑦 = 0 is a feasible vertex.
• In the slack form, this means setting the nonbasic variables to 0

➢ We’ll later see what to do in the general case

Simple: Step 2

373F20 - Nisarg Shah 28

• What next? Let’s look at an example
• To increase the value of 𝑨:

➢ Find a nonbasic variable with a positive coefficient

• This is called an entering variable

➢ See how much you can increase its value without violating any

constraints

Simple: Step 2

373F20 - Nisarg Shah 29

This is because the current values of 𝑦2 and 𝑦3 are 0, and we need 𝑦4, 𝑦5, 𝑦6 ≥ 0

Simple: Step 2

373F20 - Nisarg Shah 30

Tightest obstacle

➢ Solve the tightest obstacle for the nonbasic variable

𝑦1 = 9 − 𝑦2 4 − 𝑦3 2 − 𝑦6 4

• Substitute the entering variable (called pivot) in other equations
• Now 𝑦1 becomes basic and 𝑦6 becomes non-basic
• 𝑦6 is called the leaving variable

Simplex: Step 2

373F20 - Nisarg Shah 31

• After one iteration of this step:

➢ The basic feasible solution (i.e. substituting 0 for all nonbasic

variables) improves from 𝑨 = 0 to 𝑨 = 27

• Repeat!

Simplex: Step 2

373F20 - Nisarg Shah 32

Simplex: Step 2

373F20 - Nisarg Shah 33

Simplex: Step 2

373F20 - Nisarg Shah 34

• There is no leaving variable (nonbasic variable with positive coefficient).
• What now? Nothing! We are done.
• Take the basic feasible solution (𝑦3 = 𝑦5 = 𝑦6 = 0).
• Gives the optimal value 𝑨 = 28
• In the optimal solution, 𝑦1 = 8, 𝑦2 = 4, 𝑦3 = 0

Simplex Overview

373F20 - Nisarg Shah 35

Start at a vertex of feasible polytope Move to a neighbor vertex with better

• bjective value

Terminate, declare the current solution and value as optimal Is there a neighbor vertex with better

• bjective value?

Simplex Overview

373F20 - Nisarg Shah 36

Assuming 𝑐 ≥ 0, start with a basic feasible solution

Move to a neighbor vertex with better

• bjective value

Terminate, declare the current solution and value as optimal Is there a neighbor vertex with better

• bjective value?

Simplex Overview

373F20 - Nisarg Shah 37

Assuming 𝑐 ≥ 0, start with a basic feasible solution

Move to a neighbor vertex with better

• bjective value

Terminate, declare the current solution and value as optimal

Is there an entering variable with positive coefficient?

Simplex Overview

373F20 - Nisarg Shah 38

Assuming 𝑐 ≥ 0, start with a basic feasible solution

Pivot on a leaving variable Terminate, declare the current solution and value as optimal

Is there an entering variable with positive coefficient?

Simplex Overview

373F20 - Nisarg Shah 39

Assuming 𝑐 ≥ 0, start with a basic feasible solution

Pivot on a leaving variable Terminate, declare

• ptimal value

Is there an entering variable with positive coefficient?

Some Outstanding Issues

373F20 - Nisarg Shah 40

• What if the entering variable has no upper bound?

➢ If it doesn’t appear in any constraints, or only appears in constraints

where it can go to ∞

➢ Then 𝑨 can also go to ∞, so declare that LP is unbounded

• What if pivoting doesn’t change the constant in 𝑨?

➢ Known as degeneracy, and can lead to infinite loops ➢ Can be prevented by “perturbing” 𝑐 by a small random amount in

each coordinate

➢ Or by carefully breaking ties among entering and leaving variables,

e.g., by smallest index (known as Bland’s rule)

Some Outstanding Issues

373F20 - Nisarg Shah 41

• We assumed 𝑐 ≥ 0, and then started with the vertex 𝑦 = 0
• What if this assumption does not hold?

𝑀𝑄

1

Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 ≤ 𝑐1

𝑏2

𝑈𝑦 ≤ 𝑐2

⋮ 𝑏𝑛

𝑈 𝑦 ≤ 𝑐𝑛

𝑦 ≥ 0 𝑀𝑄

2

Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 + 𝑡1 = 𝑐1

𝑏2

𝑈𝑦 + 𝑡2 = 𝑐2

⋮ 𝑏𝑛

𝑈 𝑦 + 𝑡𝑛 = 𝑐𝑛

𝑦, 𝑡 ≥ 0 𝑀𝑄

3

Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 + 𝑡1 = 𝑐1

−𝑏2

𝑈𝑦 − 𝑡2 = −𝑐2

⋮ −𝑏𝑛

𝑈 𝑦 − 𝑡𝑛 = −𝑐𝑛

𝑦, 𝑡 ≥ 0

Multiply every constraint with negative 𝑐𝑗 by − 1 so RHS is now positive

Some Outstanding Issues

373F20 - Nisarg Shah 42

• We assumed 𝑐 ≥ 0, and then started with the vertex 𝑦 = 0
• What if this assumption does not hold?

𝑀𝑄

3

Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 + 𝑡1 = 𝑐1

−𝑏2

𝑈𝑦 − 𝑡2 = −𝑐2

⋮ −𝑏𝑛

𝑈 𝑦 − 𝑡𝑛 = −𝑐𝑛

𝑦, 𝑡 ≥ 0

Remember: RHS is now positive

𝑀𝑄

4

Min σ𝑗 𝑨𝑗 s.t. 𝑏1

𝑈𝑦 + 𝑡1 + 𝑨1 = 𝑐1

−𝑏2

𝑈𝑦 − 𝑡2 + 𝑨2 = −𝑐2

⋮ −𝑏𝑛

𝑈 𝑦 − 𝑡𝑛 + 𝑨𝑛 = −𝑐𝑛

𝑦, 𝑡, 𝑨 ≥ 0

Remember: we only want to find a basic feasible solution to 𝑀𝑄

1

Some Outstanding Issues

373F20 - Nisarg Shah 43

• We assumed 𝑐 ≥ 0, and then started with the vertex 𝑦 = 0
• What if this assumption does not hold?

Remember: the RHS is now positive

𝑀𝑄

4

Min σ𝑗 𝑨𝑗 s.t. 𝑏1

𝑈𝑦 + 𝑡1 + 𝑨1 = 𝑐1

−𝑏2

𝑈𝑦 − 𝑡2 + 𝑨2 = −𝑐2

⋮ −𝑏𝑛

𝑈 𝑦 − 𝑡𝑛 + 𝑨𝑛 = −𝑐𝑛

𝑦, 𝑡, 𝑨 ≥ 0

What now?

• Solve 𝑀𝑄

4 using simplex with

the initial basic solution being 𝑦 = 𝑡 = 0, 𝑨 = 𝑐

• If its optimum value is 0,

extract a basic feasible solution 𝑦∗ from it, use it to solve 𝑀𝑄

1 using simplex

• If optimum value for 𝑀𝑄

4 is

greater than 0, then 𝑀𝑄

1 is

infeasible

Some Outstanding Issues

373F20 - Nisarg Shah 44

• We assumed 𝑐 ≥ 0, and then started with the vertex 𝑦 = 0
• What if this assumption does not hold?

𝑀𝑄

1

Max 𝑑𝑈𝑦 s.t. 𝑏1

𝑈𝑦 ≤ 𝑐1

𝑏2

𝑈𝑦 ≤ 𝑐2

⋮ 𝑏𝑛

𝑈 𝑦 ≤ 𝑐𝑛

𝑦 ≥ 0 𝑀𝑄

2

Min σ𝑗 𝑨𝑗 s.t. 𝑏1

𝑈𝑦 + 𝑡1 + 𝑨1 = 𝑐1

𝑏2

𝑈𝑦 + 𝑡2 + 𝑨2 = 𝑐2

⋮ 𝑏𝑛

𝑈 𝑦 + 𝑡𝑛 + 𝑨𝑛 = 𝑐𝑛

𝑦, 𝑡 ≥ 0

• Solve 𝑀𝑄2 using simplex with

the initial basic feasible solution 𝑦 = 𝑡 = 0, 𝑨 = 𝑐

• If its optimum value is 0,

extract a basic feasible solution 𝑦∗ from it, use it to solve 𝑀𝑄

1 using simplex

• If optimum value for 𝑀𝑄2 is

greater than 0, then 𝑀𝑄

1 is

infeasible

Some Outstanding Issues

373F20 - Nisarg Shah 45

• Curious about pseudocode? Proof of correctness? Running

time analysis?

• See textbook for details, but this is NOT in syllabus!

Running Time

373F20 - Nisarg Shah 46

• Notes

➢ Number of vertices of a polytope can be exponential in the number

• f constraints
• There are examples where simplex takes exponential time if you choose

your pivots arbitrarily

• No pivot rule known which guarantees polynomial running time

➢ There are other algorithms which run in polynomial time

• Ellipsoid method, interior point method, …
• Ellipsoid uses 𝑃(𝑛𝑜3𝑀) arithmetic operations, where 𝑀 = length of input
• But no known strongly polynomial time algorithm
• Number of arithmetic operations = poly(m,n)

Certificate of Optimality

373F20 - Nisarg Shah 47

• Suppose you design a state-of-the-art LP solver that can

solve very large problem instances

• You want to convince someone that you have this new

technology without showing them the code

➢ Idea: They can give you very large LPs and you can quickly return the

• ptimal solutions

➢ Question: But how would they know that your solutions are optimal,

if they don’t have the technology to solve those LPs?

Certificate of Optimality

373F20 - Nisarg Shah 48

• Suppose I tell you that 𝑦1, 𝑦2 = (100,300) is optimal with
• bjective value 1900
• How can you check this?

➢ Note: Can easily substitute (𝑦1, 𝑦2), and verify that it is feasible, and

its objective value is indeed 1900

Certificate of Optimality

373F20 - Nisarg Shah 49

• Any solution that satisfies these inequalities also satisfies

their positive combinations

➢ E.g. 2*first_constraint + 5*second_constraint + 3*third_constraint ➢ Try to take combinations which give you 𝑦1 + 6𝑦2 on LHS

• Claim: 𝑦1, 𝑦2 = (100,300) is
• ptimal with objective value 1900

Certificate of Optimality

373F20 - Nisarg Shah 50

• first_constraint + 6*second_constraint

➢ 𝑦1 + 6𝑦2 ≤ 200 + 6 ∗ 300 = 2000 ➢ This shows that no feasible solution can beat 2000

• Claim: 𝑦1, 𝑦2 = (100,300) is
• ptimal with objective value 1900

Certificate of Optimality

373F20 - Nisarg Shah 51

• 5*second_constraint + third_constraint

➢ 5𝑦2 + 𝑦1 + 𝑦2 ≤ 5 ∗ 300 + 400 = 1900 ➢ This shows that no feasible solution can beat 1900

• No need to proceed further
• We already know one solution that achieves 1900, so it must be optimal!
• Claim: 𝑦1, 𝑦2 = (100,300) is
• ptimal with objective value 1900

Is There a General Algorithm?

373F20 - Nisarg Shah 52

• Introduce variables 𝑧1, 𝑧2, 𝑧3 by which we will be

multiplying the three constraints

➢ Note: These need not be integers. They can be reals.

• After multiplying and adding constraints, we get:

𝑧1 + 𝑧3 𝑦1 + 𝑧2 + 𝑧3 𝑦2 ≤ 200𝑧1 + 300𝑧2 + 400𝑧3

Is There a General Algorithm?

373F20 - Nisarg Shah 53

➢ We have:

𝑧1 + 𝑧3 𝑦1 + 𝑧2 + 𝑧3 𝑦2 ≤ 200𝑧1 + 300𝑧2 + 400𝑧3

➢ What do we want?

• 𝑧1, 𝑧2, 𝑧3 ≥ 0 because otherwise direction of inequality flips
• LHS to look like objective 𝑦1 + 6𝑦2
• In fact, it is sufficient for LHS to be an upper bound on objective
• So we want 𝑧1 + 𝑧3 ≥ 1 and 𝑧2 + 𝑧3 ≥ 6

Is There a General Algorithm?

373F20 - Nisarg Shah 54

➢ We have:

𝑧1 + 𝑧3 𝑦1 + 𝑧2 + 𝑧3 𝑦2 ≤ 200𝑧1 + 300𝑧2 + 400𝑧3

➢ What do we want?

• 𝑧1, 𝑧2, 𝑧3 ≥ 0
• 𝑧1 + 𝑧3 ≥ 1, 𝑧2 + 𝑧3 ≥ 6
• Subject to these, we want to minimize the upper bound 200𝑧1 +

300𝑧2 + 400𝑧3

Is There a General Algorithm?

373F20 - Nisarg Shah 55

➢ We have:

𝑧1 + 𝑧3 𝑦1 + 𝑧2 + 𝑧3 𝑦2 ≤ 200𝑧1 + 300𝑧2 + 400𝑧3

➢ What do we want?

• This is just another LP!
• Called the dual
• Original LP is called the primal

Is There a General Algorithm?

373F20 - Nisarg Shah 56

➢ The problem of verifying optimality is another LP

• For any 𝑧1, 𝑧2, 𝑧3 that you can find, the objective value of the dual is an

upper bound on the objective value of the primal

• If you found a specific 𝑧1, 𝑧2, 𝑧3 for which this dual objective becomes

equal to the primal objective for the (𝑦1, 𝑦2) given to you, then you would know that the given 𝑦1, 𝑦2 is optimal for primal (and your (𝑧1, 𝑧2, 𝑧3) is optimal for dual)

Is There a General Algorithm?

373F20 - Nisarg Shah 57

➢ The problem of verifying optimality is another LP

• Issue 1: But…but…if I can’t solve large LPs, how will I solve the dual to

verify if optimality of (𝑦1, 𝑦2) given to me?

• You don’t. Ask the other party to give you both (𝑦1, 𝑦2) and the

corresponding 𝑧1, 𝑧2, 𝑧3 for proof of optimality

• Issue 2: What if there are no (𝑧1, 𝑧2, 𝑧3) for which dual objective matches

primal objective under optimal solution (𝑦1, 𝑦2)?

• As we will see, this can’t happen!

Is There a General Algorithm?

373F20 - Nisarg Shah 58

Primal LP Dual LP

➢ General version, in our standard form for LPs

Is There a General Algorithm?

373F20 - Nisarg Shah 59

Primal LP Dual LP

• 𝑑𝑈𝑦 for any feasible 𝑦 ≤ 𝑧𝑈𝑐 for any feasible 𝑧
• max

primal feasible 𝑦 𝑑𝑈𝑦 ≤

min

dual feasible 𝑧 𝑧𝑈𝑐

• If there is (𝑦∗, 𝑧∗) with 𝑑𝑈𝑦∗ = 𝑧∗ 𝑈𝑐, then both must be optimal
• In fact, for optimal 𝑦∗, 𝑧∗ , we claim that this must happen!
• Does this remind you of something? Max-flow, min-cut…

Weak Duality

373F20 - Nisarg Shah 60

• From here on, assume primal LP is feasible and bounded
• Weak duality theorem:

➢ For any primal feasible 𝑦 and dual feasible 𝑧, 𝑑𝑈𝑦 ≤ 𝑧𝑈𝑐

• Proof:

𝑑𝑈𝑦 ≤ 𝑧𝑈𝐵 𝑦 = 𝑧𝑈 𝐵𝑦 ≤ 𝑧𝑈𝑐 Primal LP Dual LP

Strong Duality

373F20 - Nisarg Shah 61

• Strong duality theorem:

➢ For any primal optimal 𝑦∗ and dual optimal 𝑧∗, 𝑑𝑈𝑦∗ = 𝑧∗ 𝑈𝑐

Primal LP Dual LP

Strong Duality Proof

373F20 - Nisarg Shah 62

• Farkas’ lemma (one of many, many versions):

➢ Exactly one of the following holds:

1) There exists 𝑦 such that 𝐵𝑦 ≤ 𝑐 2) There exists 𝑧 such that 𝑧𝑈𝐵 = 0, 𝑧 ≥ 0, 𝑧𝑈𝑐 < 0

• Geometric intuition:

➢ Define image of 𝐵 = set of all possible values of 𝐵𝑦 ➢ It is known that this is a “linear subspace” (e.g. a line in a plane, a

line or plane in 3D, etc)

This slide is not in the scope of the course

Strong Duality Proof

373F20 - Nisarg Shah 63

• Farkas’ lemma: Exactly one of the following holds:

1) There exists 𝑦 such that 𝐵𝑦 ≤ 𝑐 2) There exists 𝑧 such that 𝑧𝑈𝐵 = 0, 𝑧 ≥ 0, 𝑧𝑈𝑐 < 0 1) Image of 𝐵 contains a point “below” 𝑐 2) The region “below” 𝑐 doesn’t intersect image of 𝐵 this is witnessed by normal vector to the image of 𝐵

This slide is not in the scope of the course

Strong Duality

373F20 - Nisarg Shah 64

• Strong duality theorem:

➢ For any primal optimal 𝑦∗ and dual optimal 𝑧∗, 𝑑𝑈𝑦∗ = 𝑧∗ 𝑈𝑐 ➢ Proof (by contradiction):

• Let 𝑨∗ = 𝑑𝑈𝑦∗ be the optimal primal value.
• Suppose optimal dual objective value > 𝑨∗
• So there is no 𝑧 such that 𝑧𝑈𝐵 ≥ 𝑑𝑈 and 𝑧𝑈𝑐 ≤ 𝑨∗, i.e.,

Primal LP Dual LP

This slide is not in the scope of the course

Strong Duality

373F20 - Nisarg Shah 65

➢ There is no 𝑧 such that ➢ By Farkas’ lemma, there is 𝑦 and 𝜇 such that ➢ Case 1: 𝜇 > 0

• Note: 𝑑𝑈𝑦 > 𝜇𝑨∗ and 𝐵𝑦 = 0 = 𝜇𝑐.
• Divide both by 𝜇 to get 𝐵

𝑦 𝜇 = 𝑐 and 𝑑𝑈 𝑦 𝜇 > 𝑨∗

• Contradicts optimality of 𝑨∗

➢ Case 2: 𝜇 = 0

• We have 𝐵𝑦 = 0 and 𝑑𝑈𝑦 > 0
• Adding 𝑦 to optimal 𝑦∗ of primal improves objective value beyond 𝑨∗ ⇒

contradiction This slide is not in the scope of the course

Exercise: Formulating LPs

373F20 - Nisarg Shah 66

• A canning company operates two

canning plants (A and B).

• Three suppliers of fresh fruits:
• Shipping costs in $/tonne:
• Plant capacities and labour costs:
• Selling price: $50/tonne, no limit
• Objective: Find which plant should get how much supply

from each grower to maximize profit

Exercise: Formulating LPs

373F20 - Nisarg Shah 68

• Similarly to the brewery example from the beginning:

➢ A brewery can invest its inventory of corn, hops and malt into

producing three types of beer

➢ Per unit resource requirement and profit are as given below ➢ The brewery cannot produce positive amounts of both A and B ➢ Goal: maximize profit

Beverage Corn (kg) Hops (kg) Malt (kg) Profit ($) A 5 4 35 13 B 15 4 20 23 C 10 7 25 15 Limit 500 300 1000

Exercise: Formulating LPs

373F20 - Nisarg Shah 69

• Similarly to the brewery example from the beginning:

➢ A brewery can invest its inventory of corn, hops and malt into

producing three types of beer

➢ Per unit resource requirement and profit are as given below ➢ The brewery can only produce 𝐷 in integral quantities up to 100 ➢ Goal: maximize profit

Beverage Corn (kg) Hops (kg) Malt (kg) Profit ($) A 5 4 35 13 B 15 4 20 23 C 10 7 25 15 Limit 500 300 1000

Exercise: Formulating LPs

373F20 - Nisarg Shah 70

• Similarly to the brewery example from the beginning:

➢ A brewery can invest its inventory of corn, hops and malt into

producing three types of beer

➢ Per unit resource requirement and profit are as given below ➢ Goal: maximize profit, but if there are multiple profit-maximizing

solutions, then…

• Break ties to choose those with the largest quantity of 𝐵
• Break any further ties to choose those with the largest quantity of 𝐶

Beverage Corn (kg) Hops (kg) Malt (kg) Profit ($) A 5 4 35 13 B 15 4 20 23 C 10 7 25 15 Limit 500 300 1000

More Tricks

373F20 - Nisarg Shah 71

• Constraint: 𝑦 ≤ 3

➢ Replace with constraints 𝑦 ≤ 3 and −𝑦 ≤ 3 ➢ What if the constraint is 𝑦 ≥ 3?

• Objective: minimize 3 𝑦 + 𝑧

➢ Add a variable 𝑢 ➢ Add the constraints 𝑢 ≥ 𝑦 and 𝑢 ≥ −𝑦 (so 𝑢 ≥ |𝑦|) ➢ Change the objective to minimize 3𝑢 + 𝑧 ➢ What if the objective is to maximize 3 𝑦 + 𝑧?

• Objective: minimize max(3𝑦 + 𝑧, 𝑦 + 2𝑧)

➢ Hint: minimizing 3 𝑦 + 𝑧 in the earlier bullet was equivalent to

minimizing max(3𝑦 + 𝑧, −3𝑦 + 𝑧)

• …

373F20 - Nisarg Shah 72