The Revised Simplex Method Combinatorial Problem Solving (CPS) - - PowerPoint PPT Presentation

The Revised Simplex Method

Combinatorial Problem Solving (CPS)

Javier Larrosa Albert Oliveras Enric Rodr´ ıguez-Carbonell

April 17, 2020

Tableau Simplex Method

2 / 30

■

The simplex method we have seen so far is called tableau simplex method

■

Some observations:

◆

At each iteration we update the full tableau xB = B−1b − B−1RxR for the new basis

◆

But ...

■

For pricing only one negative reduced cost is needed

■

For ratio test, only

◆

the column of the chosen non-basic variable in the tableau, and

◆

the current basic solution are needed

Revised Simplex Method

3 / 30

■

Idea: do not keep a representation of the full tableau, only B−1

■

Advantages over the tableau version:

◆

Time and space are saved

◆

Errors due to floating-point arithmetic are easier to control

Revised Simplex Method

3 / 30

■

Idea: do not keep a representation of the full tableau, only B−1

■

Advantages over the tableau version:

◆

Time and space are saved

◆

Errors due to floating-point arithmetic are easier to control

■

We will revise the algorithm and express it in terms of B−1

■

First for LP’s of the form min z = cTx Ax = b x ≥ 0

Basic Solution

4 / 30

■

Let us see how the basic solution is expressed in terms of B−1

■

For any basis B, values of basic variables can be expressed in terms of non-basic variables: BxB + RxR = b BxB = b − RxR xB = B−1b − B−1RxR

Basic Solution

4 / 30

■

Let us see how the basic solution is expressed in terms of B−1

■

For any basis B, values of basic variables can be expressed in terms of non-basic variables: BxB + RxR = b BxB = b − RxR xB = B−1b − B−1RxR

■

By definition, the basic solution corresponds to assigning null values to all non-basic variables: xR = 0 Then xB = B−1b

■

We will denote the basic solution (projected on basic variables) with β := B−1b

Optimality Condition

5 / 30

■

Let us see now how to express the reduced costs in terms of B−1

■

Recall the equation of basic variables in terms of non-basic variables: xB = B−1b − B−1RxR

■

Cost function can be split: cT x = cT

BxB + cT RxR, where

cT

B are the costs of basic variables,

cT

R are the costs of non-basic variables

Optimality Condition

5 / 30

■

Let us see now how to express the reduced costs in terms of B−1

■

Recall the equation of basic variables in terms of non-basic variables: xB = B−1b − B−1RxR

■

Cost function can be split: cT x = cT

BxB + cT RxR, where

cT

B are the costs of basic variables,

cT

R are the costs of non-basic variables

■

We can express the cost function in terms of non-basic variables: cTx = cT

BxB + cT RxR =

cT

B(B−1b − B−1RxR) + cT RxR =

cT

BB−1b − cT BB−1RxR + cT RxR =

cT

BB−1b + (cT R − cT BB−1R)xR

Optimality Condition

6 / 30

■

We found that cT x = cT

BB−1b + (cT R − cT BB−1R)xR

■

The part that depends on non-basic variables is (cT

R − cT BB−1R)xR

■

Let aj be the column in A corresponding to variable xj ∈ xR. The coefficient of xj in (cT

R − cT BB−1R)xR

is cj − cT

BB−1aj

We will denote the reduced cost of xj with dj := cj − cT

BB−1aj

■

Optimality condition: dj ≥ 0 for all j ∈ R

Cost at Basic Solution

7 / 30

■

Let’s see how to express the value of the cost function at the basic solution

■

We found that cT x = cT

BB−1b + dT RxR, where dj = cj − cT BB−1aj

■

We will denote the value of the cost function at the basic solution with z

■

Taking xR = 0 in the above equation: z := cT

BB−1b

Cost at Basic Solution

7 / 30

■

Let’s see how to express the value of the cost function at the basic solution

■

We found that cT x = cT

BB−1b + dT RxR, where dj = cj − cT BB−1aj

■

We will denote the value of the cost function at the basic solution with z

■

Taking xR = 0 in the above equation: z := cT

BB−1b

■

To avoid repeating computations: Let us define the simplex multiplier as π := (BT )−1cB Then πT = cT

BB−1

So dj = cj − πTaj (and z = πTb)

Improving a Non-Optimal Solution

8 / 30

■

Let us assume the optimality condition is violated

■

Let xq be a non-basic variable such that its reduced cost is dq < 0

■

Current value of xq is 0. We can improve by increasing only this value while non-negativity constraints of basic variables are satisfied.

Improving a Non-Optimal Solution

8 / 30

■

Let us assume the optimality condition is violated

■

Let xq be a non-basic variable such that its reduced cost is dq < 0

■

Current value of xq is 0. We can improve by increasing only this value while non-negativity constraints of basic variables are satisfied.

■

Let t ≥ 0 be the new value for xq. Let xB(t) be the values of basic variables in terms of t Let xR(t) be the values of non-basic variables in terms of t Note that xq(t) = t, and xp(t) = 0 if p ∈ R and p = q

Improving a Non-Optimal Solution

8 / 30

■

Let us assume the optimality condition is violated

■

Let xq be a non-basic variable such that its reduced cost is dq < 0

■

Current value of xq is 0. We can improve by increasing only this value while non-negativity constraints of basic variables are satisfied.

■

Let t ≥ 0 be the new value for xq. Let xB(t) be the values of basic variables in terms of t Let xR(t) be the values of non-basic variables in terms of t Note that xq(t) = t, and xp(t) = 0 if p ∈ R and p = q So xB(t) = B−1b − B−1RxR(t) = B−1b − B−1aqt = β − tαq where β = B−1b is the basic solution and we denote the column in the tableau of xq as αq := B−1aq

Improving a Non-Optimal Solution

9 / 30

■

How much do we improve? How does the objective value change as a function of t? z(t) = cTx(t) = cT

BxB(t) + cT RxR(t) =

cT

BxB(t) + cqt =

cT

Bβ − tcT Bαq + cqt =

cT

Bβ − tcT BB−1aq + cqt =

z + tdq

■

As expected, the improvement in cost is ∆z = z(t) − z = tdq

Improving a Non-Optimal Solution

10 / 30

■

Recall that xB(t) = β − tαq

■

How can we satisfy the non-negativity constraints of basic variables?

Improving a Non-Optimal Solution

10 / 30

■

Recall that xB(t) = β − tαq

■

How can we satisfy the non-negativity constraints of basic variables?

■

Basic variables have indices B = (k1, ..., km)

■

Let i ∈ {1, ..., m}. The i-th basic variable is xki

■

Value of xki as a function of t is the i-th component of xB(t): βi − tαi

q,

where βi is the i-th component of β and αi

q is the i-th component of αq

Improving a Non-Optimal Solution

10 / 30

■

Recall that xB(t) = β − tαq

■

How can we satisfy the non-negativity constraints of basic variables?

■

Basic variables have indices B = (k1, ..., km)

■

Let i ∈ {1, ..., m}. The i-th basic variable is xki

■

Value of xki as a function of t is the i-th component of xB(t): βi − tαi

q,

where βi is the i-th component of β and αi

q is the i-th component of αq

■

We need βi − tαi

q ≥ 0

⇐ ⇒ βi ≥ tαi

q

◆

If αi

q ≤ 0 the constraint is satisfied for all t ≥ 0

◆

If αi

q > 0 we need βi αi

q ≥ t

Improving a Non-Optimal Solution

10 / 30

■

Recall that xB(t) = β − tαq

■

How can we satisfy the non-negativity constraints of basic variables?

■

Basic variables have indices B = (k1, ..., km)

■

Let i ∈ {1, ..., m}. The i-th basic variable is xki

■

Value of xki as a function of t is the i-th component of xB(t): βi − tαi

q,

where βi is the i-th component of β and αi

q is the i-th component of αq

■

We need βi − tαi

q ≥ 0

⇐ ⇒ βi ≥ tαi

q

◆

If αi

q ≤ 0 the constraint is satisfied for all t ≥ 0

◆

If αi

q > 0 we need βi αi

q ≥ t

■

The best improvement is achieved with the strongest of the upper bounds: θ := min{ βi

αi

q | αi

q > 0}

■

We say the p-th basic variable xkp is blocking or tight when θ = βp

αp

q .

Then αp

q is the pivot

Improving a Non-Optimal Solution

11 / 30

1. If θ = +∞ (there is no upper bound, i.e., no i such that 1 ≤ i ≤ m and αi

q > 0):

Value of objective function can be decreased infinitely. LP is unbounded.

Improving a Non-Optimal Solution

11 / 30

1. If θ = +∞ (there is no upper bound, i.e., no i such that 1 ≤ i ≤ m and αi

q > 0):

Value of objective function can be decreased infinitely. LP is unbounded. 2. If θ < +∞ and the p-th basic variable xkp is blocking: When setting xq = θ, the non-negativity of basic variables is respected In particular the value of xkp, i.e. the p-th component of xB(t), is βp − θαp

q = 0

We can make a basis change: xq enters the basis and xkp leaves, where B = (k1, ..., km)

Update

12 / 30

■

New basic indices: ¯ B = (k1, . . . , kp−1, q, kp+1 . . . , km) Before the p-th basic variable was xkp, now it is xq

■

New basis: ¯ B = B + (aq − akp)eT

p

where eT

p = (0, ..., 0, p

• 1 , 0, ..., 0)
• m

The p-th column of the basis (which was akp) is replaced by aq.

■

New basic solution: ¯ βp = θ, ¯ βi = βi − θαi

q if i = p

Note that before the p-th component of β corresponded to xkp, now to xq

■

New objective value: ¯ z = z + θdq

Algorithmic Description

13 / 30

1. Initialization: Find an initial feasible basis B Compute B−1, β = B−1b, z = cT

Bβ

2. Pricing: Compute πT = cT

BB−1 and dj = cj − πTaj.

If for all j ∈ R, dj ≥ 0 then return OPTIMAL Else let q be such that dq < 0. Compute αq = B−1aq 3. Ratio test: Compute I = {i | 1 ≤ i ≤ m, αi

q > 0}.

If I = ∅ then return UNBOUNDED Else compute θ = mini∈I( βi

αi

q ) and p such that θ = βp

αp

q

4. Update: ¯ B = B − {kp} ∪ {q} ¯ B = B + (aq − akp)eT

p

¯ βp = θ, ¯ βi = βi − θαi

q if i = p

¯ z = z + θdq Go to 2.

Updating Matrix Inverse

14 / 30

■

Actually what we really care about is B−1, not B We need it for computing π = cT

BB−1 and αq = B−1aq at each step

(and also β = B−1b in the initialization)

■

Recomputing B−1 at each iteration is too expensive (e.g. O(m3) arithmetic operations with Gaussian elimination!)

■

Next slides: a more efficient way of computing ¯ B−1 using B−1

Updating Matrix Inverse

15 / 30

■

Let us make a diversion into linear algebra

■

Let b1, ..., bm be the columns of an invertible matrix B Let a, α be such that a = Bα = m

i=1 αibi

Let p be such that 1 ≤ p ≤ m

■

Ba = (b1, . . . , bp−1, a, bp+1, . . . , bm). Want to compute B−1

a

Updating Matrix Inverse

15 / 30

■

Let us make a diversion into linear algebra

■

Let b1, ..., bm be the columns of an invertible matrix B Let a, α be such that a = Bα = m

i=1 αibi

Let p be such that 1 ≤ p ≤ m

■

Ba = (b1, . . . , bp−1, a, bp+1, . . . , bm). Want to compute B−1

a

■

Note αp = 0 as otherwise rank(Ba) < m. Then a = αpbp +

i=p αibi

⇒ bp =

• 1

αp

• a +

i=p

• −αi

αp

• bi

■

Let ηT =

−α1 αp

• , . . . ,
• −αp−1

αp

• , 1

αp ,

• −αp+1

αp

• , . . . ,
• −αm

αp

T . Then bp = Baη

■

Let E = (e1, . . . , ep−1, η, ep+1, . . . , em). Then BaE = B = ⇒ E−1B−1

a

= B−1 = ⇒ B−1

a

= EB−1

Updating Matrix Inverse

16 / 30

■

Application to the simplex algorithm: a = aq, α = αq, where xq is entering variable Thus to update the inverse we can reuse already computed data!

■

Using this update: B−1 is not actually represented as a square table, but as follows

■

Assume initial basis is B0 (e.g., unit matrix I). Then at the k-th iteration of the simplex algorithm the inverse matrix is B−1 = EkEk−1 · · · E2E1B−1

0 ,

where Ei is the E matrix of the i-th iteration

■

Each E matrix can be stored compactly (vector η + column index p)

■

We can represent B−1 as the list EkEk−1 · · · E2E1, B−1

0 :

Product Form of the Inverse (PFI)

■

When the list is long we reset: the inverse is computed (reinversion)

■

Other ways of representing B−1: LU factoritzation

Bounded Variables

17 / 30

■

Now we want to revise the simplex algorithm for LP’s of the form min cT x Ax = b ℓ ≤ x ≤ u (−∞ ≤ ℓk ≤ uk ≤ +∞)

■

In practice, internally variables are translated so that ℓ, u = −∞ or ℓ = 0 or u = 0 to save arithmetic operations

■

Variable xk is lower bounded if ℓk > −∞

■

Variable xk is upper bounded if uk < +∞

■

Variable xk is free if ℓk = −∞ and uk = +∞

Basic Solution

18 / 30

■

For any basis B, recall that values of basic variables can be expressed in terms of non-basic variables: xB = B−1b − B−1RxR

■

The values of a non-basic variable xj can be: If lower bounded: ℓk (we say it is at lower bound; denoted by xj ∈ L) If upper bounded: uk (we say it is at upper bound; denoted by xj ∈ U) If free: (we say it is at zero level; denoted by xj ∈ Z)

■

Basic solution: β := B−1b −

• j∈L

B−1ajℓj −

• j∈U

B−1ajuj

Optimality Condition

19 / 30

■

Recall the cost function in terms of non-basic variables: cT x = cT

BB−1b + dRxR

where dj = cj − cT

BB−1aj for all variable xj

■

If xj ∈ L: cannot improve if dj ≥ 0

■

If xj ∈ U: cannot improve if dj ≤ 0

■

If xj ∈ Z: cannot improve if dj = 0

■

Optimality condition: no improving non-basic variable

Objective Function

20 / 30

■

Recall the cost function in terms of non-basic variables: cT x = cT

BB−1b + dRxR

where dj = cj − cT

BB−1aj for all variable xj

■

Value z of cost function at current basic solution: z = cT

BB−1b +

• j∈L

djℓj +

• j∈U

djuj

Improving a Non-Optimal Solution

21 / 30

■

Let xq be a non-basic variable that can improve objective value by increasing its value. This can happen when

◆

xq is lower bounded and xq ∈ L; or

◆

xq is free (so xq ∈ Z)

■

Since increasing xq can improve objective value: dq < 0

■

Let t ≥ 0 be difference of new value xq wrt old value xB(t) = B−1b − B−1RxR(t) = B−1b − tB−1aq −

j∈L B−1ajℓj − j∈U B−1ajuj

= β − tαq where β = B−1b −

• j∈L

B−1ajℓj −

• j∈U

B−1ajuj, αq = B−1aq

Improving a Non-Optimal Solution

22 / 30

■

How does the objective value change as a function of t? z(t) = cTx(t) = cT

BxB(t) + cT RxR(t) =

cT

BxB(t) + tcq +

• j∈L

cjℓj +

• j∈U

cjuj = cT

BB−1b +

• j∈L

(cj − cT

BB−1aj)ℓj +

• j∈U

(cj − cT

BB−1aj)uj + tcq − tcT Bαq =

cT

BB−1b +

• j∈L

djℓj +

• j∈U

djuj + tcq − tcT

Bαq =

z + tcq − tcT

Bαq =

z + tcq − tcT

BB−1aq =

z + tdq

■

Hence the improvement in cost is ∆z = z(t) − z = tdq

Improving a Non-Optimal Solution

23 / 30

■

Basic variables have indices B = (k1, ..., km)

■

Let i ∈ {1, ..., m}. The i-th basic variable is xki

■

Value of xki as a function of t is the i-th component of xB(t): βi − tαi

q

■

Let λi := ℓki, µi := uki We need λi ≤ βi − tαi

q ≤ µi

◆

If αi

q > 0: βi − tαi q ≥ λi

⇒

βi−λi αi

q

≥ t

◆

If αi

q < 0: βi − tαi q ≤ µi

⇒

βi−µi αi

q

≥ t

■

But we need xq(t) ≤ uq too!

■

Best improvement achieved with θ := min( uq − ℓq, min{ βi−λi

αi

q

| αi

q > 0},

min{ βi−µi

αi

q

| αi

q < 0})

■

If θ = +∞ we have unboundedness

■

Else if θ = uq − ℓq we have a bound flip: no pivot needed!

Improving a Non-Optimal Solution

24 / 30

θ := min( uq − ℓq, min{ βi−λi

αi

q

| αi

q > 0},

min{ βi−µi

αi

q

| αi

q < 0})

■

Assume θ = +∞, θ = uq − ℓq.

■

Thus variable xq enters the basis and variable xkp leaves

■

If θ = min{ βi−λi

αi

q

| αi

q > 0} then xkp leaves the basis at lower bound

■

If θ = min{ βi−µi

αi

q

| αi

q < 0} then xkp leaves the basis at upper bound

Improving a Non-Optimal Solution

25 / 30

■

Let xq be a non-basic variable that can improve objective value by decreasing its value. This can happen when

◆

xq is upper bounded and xq ∈ U; or

◆

xq is free (so xq ∈ Z)

■

Since decreasing xq can improve objective value: dq > 0

■

Let t ≤ 0 be the difference of new value xq wrt old value

■

Again xB(t) = β − tαq

■

Again the improvement in cost is ∆z = z(t) − z = tdq

Improving a Non-Optimal Solution

26 / 30

■

Basic variables have indices B = (k1, ..., km)

■

Let i ∈ {1, ..., m}. The i-th basic variable is xki

■

Value of xki as a function of t is the i-th component of xB(t): βi − tαi

q

■

Let λi := ℓki, µi := uki We need λi ≤ βi − tαi

q ≤ µi

◆

If αi

q > 0: βi − tαi q ≤ µi

⇒

βi−µi αi

q

≤ t

◆

If αi

q < 0: βi − tαi q ≥ λi

⇒

βi−λi αi

q

≤ t

■

But we need ℓq ≤ xq(t) too!

■

Best improvement achieved with θ := max( ℓq − uq, max{ βi−λi

αi

q

| αi

q < 0},

max{ βi−µi

αi

q

| αi

q > 0})

■

If θ = −∞ we have unboundedness

■

Else if θ = ℓq − uq we have a bound flip: no pivot needed!

Improving a Non-Optimal Solution

27 / 30

θ := max( ℓq − uq, max{ βi−λi

αi

q

| αi

q < 0},

max{ βi−µi

αi

q

| αi

q > 0})

■

Assume θ = −∞, θ = ℓq − uq.

■

Thus variable xq enters basis and variable xkp leaves

■

If θ = max{ βi−λi

αi

q

| αi

q < 0}, xkp leaves basis at lower bound

■

If θ = max{ βi−µi

αi

q

| αi

q > 0}, xkp leaves basis at upper bound

Update

28 / 30

■

New objective value: ¯ z = z + θdq

■

If bound flip

◆

Flip status of xq (i.e., ¯ xq ∈ L ⇔ xq ∈ U)

◆

New basic solution: ¯ β = β − θαq

■

Else

◆

New basic indices: ¯ B = (k1, . . . , kp−1, q, kp+1 . . . , km)

◆

New basic solution: ¯ βp = xq + θ, ¯ βi = βi − θαi

q if i = p

◆

New basis inverse: ¯ B−1 = EB−1

◆

If entering variable comes from lower bound ˆ L = L − {xq} else ˆ U = U − {xq}

◆

If leaving variable leaves to lower bound ¯ L = ˆ L ∪ {xkp} else ¯ U = ˆ U ∪ {xkp}

Tableau vs. Revised Simplex

29 / 30

■

Time is saved: ✗ Tableau: all dk, all αk are computed ✓ Revised: no. of non-basic variables xk for which dk, αk are computed can be adjusted

■

Space is saved: ✗ Tableau: even if A sparse, tableau tends to get filled ✓ Revised: sparsity of A can be exploited for storage, and pivots can be chosen to represent B−1 compactly

■

Better numerical behaviour: ✗ Tableau: errors due to floating-point arithmetic accumulate at each pivoting step ✓ Revised: reinversion (PFI representation of B−1) or refactorization (LU representation of B−1) can be used for resetting

Original vs. Bounds Simplex

30 / 30

■

Time is saved: ✗ Original: no special treatment of bounds ✓ Bounds: bound flips are much cheaper than pivoting steps in simplex iterations (basis does not change)

■

Space is saved: ✗ Original: each bound constraint becomes a row ✓ Bounds: bounds are stored cheaply in arrays