[PPT] - How to Exploit a Heterogeneous Cluster of Computers (Asymptotically) PowerPoint Presentation

SLIDE 1

How to “Exploit” a Heterogeneous Cluster of Computers (Asymptotically) Optimally Arnold L. Rosenberg

Electrical & Computer Engineering Colorado State University Fort Collins, CO 80523, USA rsnbrg@colostate.edu

Joint work with Micah Adler Ying Gong

SLIDE 2

The Computational Environment

A “master” computer C0

(This is our computer.)

SLIDE 3

The Computational Environment

A “master” computer C0
A cluster C of n heterogeneous computers

C1, C2, . . . , Cn that are available for dedicated “rental” (The Ci differ in processor, memory speeds.)

SLIDE 4

The Computational Environment

A “master” computer C0
A cluster C of n heterogeneous computers

C1, C2, . . . , Cn that are available for dedicated “rental” (The Ci may be geographically dispersed.)

SLIDE 5

The Computational Environment

A “master” computer C0
A cluster C of n heterogeneous computers

C1, C2, . . . , Cn that are available for dedicated “rental”

a large “bag” of (arbitrarily but) equally complex tasks

SLIDE 6

Two Simple Worksharing Problems The Cluster-Exploitation Problem

One has access to cluster C for L time units.
One wants to accomplish as much work as possible

during that time.

SLIDE 7

Two Simple Worksharing Problems The Cluster-Exploitation Problem

One has access to cluster C for L time units.
One wants to accomplish as much work as possible

during that time. The Cluster-Rental Problem

One has W units of work to complete.
One wishes to “rent” cluster C for as short a period
f time as necessary to complete that work.

SLIDE 8

Our Contributions Within HiHCoHP — a heterogeneous, long-message analog

f the LogP architectural model — we offer:

SLIDE 9

Our Contributions Within HiHCoHP — a heterogeneous, long-message analog

f the LogP architectural model — we offer:

A Generic Worksharing Protocol:

works predictably for many variants of our model.
determines all work-allocations and all communication times.

SLIDE 10

Our Contributions Within HiHCoHP — a heterogeneous, long-message analog

f the LogP architectural model — we offer:

A Generic Worksharing Protocol:

works predictably for many variants of our model.
determines all work-allocations and all communication times.

An Asymptotically Optimal Worksharing Protocol:

solves the Cluster-Exploitation and -Rental Problems optimally

— as long as L is sufficiently long.

SLIDE 11

Our Contributions — Details Worksharing protocols:

C0 supplies work to each “rented” Ci, in some order

— in a single message for each Ci

SLIDE 12

Our Contributions — Details Worksharing protocols:

C0 supplies work to each “rented” Ci, in some order
Ci does the work — and returns its results

— in a single message from each Ci

SLIDE 13

Our Contributions — Details Worksharing protocols:

C0 supplies work to each “rented” Ci, in some order
Ci does the work — and returns its results

Asymptotically optimal worksharing protocols:

Computers start and finish computing in the same order:

— first started ⇒ first finished

Optimality is independent of computers’ starting order:

— even if each Ci is 1010 times faster than Ci+1

SLIDE 14

The Model Calibration

All units — time and packet size — are calibrated to the slowest computer’s

computation rate: – This C does one “unit” of work in one “unit” of time.

Each unit of work produces δ units of results (for simplicity).

SLIDE 15

Computation Rates ρi is the per-unit work time for computer Ci

ρ1 ≤ ρ2 ≤ · · · ≤ ρn (by convention)

[The smaller the index, the faster the computer.]

ρn = 1 (by our calibration)

SLIDE 16

The Costs of Communication, 1 Message Processing time for Ci: Transmission setup: σ time units

per communication

Transmission packaging: πi time units -per packet Reception unpackaging: πi time units -per packet

Subscripts reflect computers’ heterogeneity.

SLIDE 17

The Costs of Communication, 2 Message Transmission Time: Latency: λ time units —for first packet Bandwidth limitation: τ

def

= 1/β time units/packet —for remaining packets

β

def

= network’s end-to-end bandwidth.

SLIDE 18

( λ ( σ σ − 1) − 1) λ δ wi wi δ τ wi δ π in network in and network , π0 in prepares work for

i

setup

i

transmits work work unpacks

i

in and network ,

i

in

i

does work

i

prepares results for setup

i i i

transmits results

i

unpacks results in C C C C C C C C C C C C C C C C C C C w C

i i

ρ wi

i

π wi τ in network πi wi

The timeline as C0 shares work with Ci

SLIDE 19

A Generic Worksharing Protocol Specifying a worksharing protocol

C0 sends work to C1, C2, . . . , Cn in the startup order:

Cs1, Cs2, . . . , Csn (Note subscript-sequence s1, s2, . . . , sn)

C1, C2, . . . , Cn return results to C0 in the finishing order:

Cf1, Cf2, . . . , Cfn (Note subscript-sequence f1, f2, . . . , fn)

SLIDE 20

The timeline for three “rented” computers, C1, C2, C3:

s

Receive Receive Receive

f

2

f

3

f

3

f

3

f

3

f

3

f

2

f

1

f

3

Transmit

Compute Compute Compute Compute Compute Compute Transmit

s s s

1 2 3

(Total compute time) Prepare Prepare Prepare σ σ σ σ σ σ Prepare Prepare Prepare Transmit

s

1

s

1

s

3

s

3

s

3

s

3

s

2

s

2

s

2

s

2

f

1

f

1

λ − τ + λ − τ + λ − τ +

s

1

s

2 3

s

1

s

2

s

3

s

1

s

1

λ − τ + Transmit Transmit Transmit

f

1

λ − τ +

f

2

λ − τ +

f

1

f

1

f

2

f

2

f

2

w L π w ρ w π τ C C w0 w0 C w δ τ C w δ τ C C w ρ C δ w π w δ w π w w τ ρ w w τ π w w τ w ρ w π0 π0 ρ w w π0 ρ δ w δ w π Lifespan

NOTE: Only one message in transit at a time

SLIDE 21

Some Useful Abbreviations Quantity Meaning

τ

τ(1 + δ) 2-way network transmission rate

πi

πi + πiδ Ci’s 2-way message-packaging rate (workload + results) F (σ + λ − τ) fixed communication overhead (becomes invisible as L grows) Vi π0 + τ + πi Ci’s variable communication

verhead rate

SLIDE 22

A Generic Protocol’s Work-Allocations Given: startup order: Σ = s1, s2, . . . , sn−1, sn finishing order: Φ = f1, f2, . . . , fn−1, fn Compute: Protocol (Σ, Φ)’s work-allocations w1, w2, . . . , wn by solving the nonsingular system of equations:

       V1 + ρ1 B1,2 · · · B1,n−1 B1,n B2,1 V2 + ρ2 · · · B2,n−1 B2,n . . . . . . · · · . . . . . . Bn−1,1 Bn−1,2 · · · Vn−1 + ρn−1 Bn−1,n Bn,1 Bn,2 · · · Bn,n−1 Vn + ρn        ·        w1 w2 . . . wn−1 wn        =        L − (c1 + 2)F L − (c2 + 2)F . . . L − (cn−1 + 2)F L − (cn + 2)F       

Bi,j assesses: π0 + τ for each Cj that starts before Ci (j ∈ SBi) τδ for each Cj that finishes after Ci (j ∈ FAi) ci

def

= |SBi| + |FAi|.

SLIDE 23

Worksharing Protocols Are Self-Scheduling Theorem. Worksharing protocols are self-scheduling.

SLIDE 24

Worksharing Protocols Are Self-Scheduling Theorem. Worksharing protocols are self-scheduling. Translation: A protocol’s startup and finishing indexings determine:

all work-allocations
the times for all communications.

SLIDE 25

The Optimal FIFO Worksharing Protocol Computers stop working — hence, return results — in the same order as they start working. The defining startup and finishing orderings: For each i ∈ {1, 2, . . . , n} : si = fi = i.

SLIDE 26

The FIFO timeline for three “rented” computers, C1, C2, C3:

Receive Receive Receive Prepare Prepare Prepare Transmit

3

s

3

s

1

s

1

s

1

Transmit

Compute Compute Compute Compute Compute Compute Transmit

s s s

1 2 3

Prepare Prepare Prepare σ σ σ σ σ σ

s

1

s

1

s

3

s

3

s

2

s

2

λ − τ + λ − τ + λ − τ +

s

1

s

2

s

3

s

1

s

2

s

3

λ − τ + Transmit Transmit Transmit λ − τ + λ − τ +

s

1

s

1

s

2

s

2

s

3

s

3

s

2

s

2

s

2

s

3

s

w ρ w w ρ π δ τ w ρ δ w π w δ τ

Lifespan

w τ

L

w τ C C w τ C C w π0 π0 δ w π0 w w π w δ τ δ w π w π w w π w

SLIDE 27

The FIFO Protocol’s Work-Allocations Given: startup order, Σ = s1, s2, . . . , sn−1, sn Compute: the FIFO work-allocations ws1, ws2, . . . , wsn by solving the system of equations:

       Vs1 + ρs1 τδ · · · τδ π0 + τ Vs2 + ρs2 · · · τδ . . . . . . · · · . . . π0 + τ π0 + τ · · · τδ π0 + τ π0 + τ · · · Vsn + ρsn               ws1 ws2 . . . wsn−1 wsn        =        L − (n + 1)(σ + λ − τ) L − (n + 1)(σ + λ − τ) . . . L − (n + 1)(σ + λ − τ) L − (n + 1)(σ + λ − τ)       

SLIDE 28

The FIFO Protocol’s Work-Output Let X(FIFO,Σ)

def

=

n

i=1

1 Vi + ρi − τδ

i−1

j=1
1 − π0 + τ − τδ

Vj + ρj − τδ

Then

W (FIFO,Σ)(L) = 1 τδ + 1/X(FIFO,Σ) · (L − (n + 1)F) . W (FIFO,Σ)(L) IS INDEPENDENT OF THE STARTUP ORDER Σ!

SLIDE 29

What’s so Wonderful about the FIFO Protocol? Theorem FIFO-Optimal. The FIFO Protocol provides an asymptotically optimal solution to the Cluster Exploitation Problem.

SLIDE 30

What’s so Wonderful about the FIFO Protocol? Theorem FIFO-Optimal. The FIFO Protocol provides an asymptotically optimal solution to the Cluster Exploitation Problem. Translation. For all sufficiently long lifespans L, W (FIFO)(L) is at least as large as the work-

utput of any other protocol.

SLIDE 31

How long is “sufficiently long?” Simulation experiments that compare the FIFO Protocol against 100 random competitors lead to the following conclusions.

SLIDE 32

How long is “sufficiently long?” Simulation experiments that compare the FIFO Protocol against 100 random competitors lead to the following conclusions.

The advantages of the FIFO regimen are often discernible within lifespans

whose durations are just minutes.

SLIDE 33

How long is “sufficiently long?” Simulation experiments that compare the FIFO Protocol against 100 random competitors lead to the following conclusions.

The advantages of the FIFO regimen are often discernible within lifespans

whose durations are just minutes.

The advantages of the FIFO regimen are seen earlier on:

– larger Clusters, – Clusters of lesser degrees of heterogeneity.

SLIDE 34

How long is “sufficiently long?” Simulation experiments that compare the FIFO Protocol against 100 random competitors lead to the following conclusions.

The advantages of the FIFO regimen are often discernible within lifespans

whose durations are just minutes.

The advantages of the FIFO regimen are seen earlier on:

– larger Clusters, – Clusters of lesser degrees of heterogeneity.

The advantages of the FIFO regimen are seen earlier when tasks are finer

grained.

SLIDE 35

How long is “sufficiently long?” Simulation experiments that compare the FIFO Protocol against 100 random competitors lead to the following conclusions.

The advantages of the FIFO regimen are often discernible within lifespans

whose durations are just minutes.

The advantages of the FIFO regimen are seen earlier on:

– larger Clusters, – Clusters of lesser degrees of heterogeneity.

The advantages of the FIFO regimen are seen earlier when tasks are finer

grained.

Even with coarse tasks, FIFO “wins” within (roughly) a weekend, except on

very small clusters.

SLIDE 36

FIFO vs. Random Competitors: “Practical” Lifespans Power-Index Task Lifespan L ≤ Vector Grain n 1 min 10 min 30 min 1 hr ρi ≡ 1 0.1 sec 8 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 8 0.48 0.52 0.64 0.81 ρi = 1 − 1/(i + 1) 8 0.50 0.51 0.63 0.70 ρi = 1 − 2−i 8 0.43 0.47 0.48 0.58 ρi ≡ 1 32 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 32 0.66 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 32 0.53 0.78 1.00 1.00 ρi = 1 − 2−i 32 0.54 0.74 1.00 1.00 ρi ≡ 1 128 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 128 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 128 0.93 1.00 1.00 1.00 ρi = 1 − 2−i 128 0.88 1.00 1.00 1.00 ρi ≡ 1 1 sec 8 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 8 0.49 0.49 0.49 0.50 ρi = 1 − 1/(i + 1) 8 0.49 0.49 0.49 0.49 ρi = 1 − 2−i 8 0.58 0.58 0.58 0.58 ρi ≡ 1 32 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 32 0.54 0.55 0.57 0.59 ρi = 1 − 1/(i + 1) 32 0.53 0.53 0.53 0.54 ρi = 1 − 2−i 32 0.46 0.47 0.48 0.49 ρi ≡ 1 128 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 128 0.51 0.73 0.95 1.00 ρi = 1 − 1/(i + 1) 128 0.48 0.52 0.64 0.75 ρi = 1 − 2−i 128 0.46 0.54 0.63 0.73

SLIDE 37

FIFO vs. Random Competitors: “Realistic” Lifespans Power-Index Task Lifespan L ≤ Vector Grain n 2 hr 4 hr 8 hr 24 hr 48 hr ρi ≡ 1 0.1 sec 8 1.00 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 8 0.98 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 8 0.90 1.00 1.00 1.00 1.00 ρi = 1 − 2−i 8 0.80 0.96 1.00 1.00 1.00 ρi ≡ 1 32 1.00 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 32 1.00 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 32 1.00 1.00 1.00 1.00 1.00 ρi = 1 − 2−i 32 1.00 1.00 1.00 1.00 1.00 ρi ≡ 1 128 1.00 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 128 1.00 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 128 1.00 1.00 1.00 1.00 1.00 ρi = 1 − 2−i 128 1.00 1.00 1.00 1.00 1.00 ρi ≡ 1 1 sec 8 1.00 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 8 0.40 0.40 42 0.52 0.65 ρi = 1 − 1/(i + 1) 8 0.49 0.50 0.50 0.51 0.57 ρi = 1 − 2−i 8 0.53 0.53 0.53 0.55 0.59 ρi ≡ 1 32 1.00 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 32 0.69 0.79 0.95 1.00 1.00 ρi = 1 − 1/(i + 1) 32 0.39 0.45 0.52 0.86 1.00 ρi = 1 − 2−i 32 0.55 0.58 0.67 0.83 0.96 ρi ≡ 1 128 1.00 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 128 1.00 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 128 0.83 0.97 1.00 1.00 1.00 ρi = 1 − 2−i 128 0.78 0.99 1.00 1.00 1.00

SLIDE 38

FIFO vs. Random Competitors: “Eventually” Power-Index Task Lifespan L ≤ Vector Grain n 4 days 8 days 16 days 32 days ρi ≡ 1 0.1 sec 8 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 8 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 8 1.00 1.00 1.00 1.00 ρi = 1 − 2−i 8 1.00 1.00 1.00 1.00 ρi ≡ 1 32 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 32 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 32 1.00 1.00 1.00 1.00 ρi = 1 − 2−i 32 1.00 1.00 1.00 1.00 ρi ≡ 1 128 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 128 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 128 1.00 1.00 1.00 1.00 ρi = 1 − 2−i 128 1.00 1.00 1.00 1.00 ρi ≡ 1 1 sec 8 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 8 0.79 0.95 1.00 1.00 ρi = 1 − 1/(i + 1) 8 0.72 0.89 0.98 1.00 ρi = 1 − 2−i 8 0.61 0.76 0.95 1.00 ρi ≡ 1 32 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 32 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 32 1.00 1.00 1.00 1.00 ρi = 1 − 2−i 32 1.00 1.00 1.00 1.00 ρi ≡ 1 128 1.00 1.00 1.00 1.00 ρi = (1 + 2i−n)/2 128 1.00 1.00 1.00 1.00 ρi = 1 − 1/(i + 1) 128 1.00 1.00 1.00 1.00 ρi = 1 − 2−i 128 1.00 1.00 1.00 1.00

SLIDE 39

Proof Sketch for Theorem FIFO-Optimal

SLIDE 40

1. W (FIFO,L,Σ) is independent of Σ

Theorem FIFO-Optimal did not specify a startup order for the allegedly optimal FIFO Protocol.

SLIDE 41

1. W (FIFO,L,Σ) is independent of Σ

Theorem FIFO-Optimal did not specify a startup order for the allegedly optimal FIFO Protocol. IT DIDN’T HAVE TO!

SLIDE 42

1. W (FIFO,L,Σ) is independent of Σ

Theorem FIFO-Optimal did not specify a startup order for the allegedly optimal FIFO Protocol. It didn’t have to! Lemma. Over any lifespan L, for any two startup orders Σ1 and Σ2, W (FIFO,Σ1)(L) = W (FIFO,Σ2)(L).

SLIDE 43

1. W (FIFO,L,Σ) is independent of Σ

Theorem FIFO-Optimal did not specify a startup order for the allegedly optimal FIFO Protocol. It didn’t have to! Lemma. Over any lifespan L, for any two startup orders Σ1 and Σ2, W (FIFO,Σ1)(L) = W (FIFO,Σ2)(L). ≈≈≈≈≈≈≈≈≈ Proof Sketch. By direct calculation, X(FIFO,Σ1) = X(FIFO,Σ2). X(FIFO,Σ)

def

=

n

i=1

1 Vi + ρi − τδ

i−1

j=1
1 − π0 + τ − τδ

Vj + ρj − τδ

SLIDE 44

2. “Flexible”-FIFO is Optimal
Lemma. (A rather bizarre result.)

If we make the FIFO Protocol flexible — allow it to slow down computers at will (by increasing their ρ-values) — then the thus-empowered protocol can (asymptotically) match the work-output of any other protocol.

SLIDE 45

2. “Flexible”-FIFO is Optimal
Lemma. (A rather bizarre result.)

If we make the FIFO Protocol flexible — allow it to slow down computers at will (by increasing their ρ-values) — then the thus-empowered protocol can (asymptotically) match the work-output of any other protocol. In other words. The Flexible FIFO Protocol solves the Cluster-Exploitation Problem asymptotically

ptimally.

SLIDE 46

Proof Strategy Start with a non-FIFO protocol P.

SLIDE 47

Proof Strategy Start with a non-FIFO protocol P.

Select the earliest violation of FIFO:

Some Csk with sk > si finishes working before Csi. – (All Csℓ with sℓ < si finish before Csi.)

SLIDE 48

Proof Strategy Start with a non-FIFO protocol P.

Select the earliest violation of FIFO:

Some Csk with sk > si finishes working before Csi.

Flip the finishing orders of Csi and of the Csj that finishes working

just before Csi. – but do not decrease aggregate work-output!!

SLIDE 49

Proof Strategy Start with a non-FIFO protocol P.

Select the earliest violation of FIFO:

Some Csk with sk > si finishes working before Csi.

Flip the finishing orders of Csi and of the Csj that finishes working

just before Csi. – but do not decrease aggregate work-output!! The new protocol is “closer to” a FIFO protocol than P was.

SLIDE 50

Proof Strategy Start with a non-FIFO protocol P.

Select the earliest violation of FIFO:

Some Csk with sk > si finishes working before Csi.

Flip the finishing orders of Csi and of the Csj that finishes working

just before Csi. – but do not decrease aggregate work-output!!

Iterate ...

SLIDE 51

Proof Strategy Start with a non-FIFO protocol P.

Select the earliest violation of FIFO:

Some Csk with sk > si finishes working before Csi.

Flip the finishing orders of Csi and of the Csj that finishes working

just before Csi. – but do not decrease aggregate work-output!!

Iterate ...

HOW DO WE DO THIS?

SLIDE 52

Implementing the Strategy, 1

1. Flip the finishing times of Csi and Csj.

s

j

s

i

s

j

s

i

s

j

s

i

s

i

s

j

’ ’ δ τ F C w δ τ F w δ τ F F w δ τ C C C w

This forces us to shorten wsi and lengthen wsj.

SLIDE 53

Implementing the Strategy, 2

2. Changing wsi and wsj forces us to adjust the starting times of

Csi, Csi+1, ..., Csj.

i i

s s

i i j

s

j

s s

j

s

j

s s

’

’

PREP PREP PREP PREP

w

C w w C C C w

We slow down computers when necessary, to take up slack times. ... AND IT ALL WORKS OUT!!

SLIDE 54

3. Full-Speed FIFO is Optimal

Lemma. Over any lifespan L, W (FIFO)(L) ≥ W (Flex−FIFO)(L). Proof Sketch. For all startup orders Σ and all ρ-value vectors: W (FIFO,Σ)(L) = 1 τδ + 1/X(FIFO,Σ) · (L − (n + 1)F) , where X(FIFO,Σ)

def

=

n

i=1

1 Vsi + ρsi − τδ

i−1

j=1
1 − π0 + τ − τδ

Vsj + ρsj − τδ

.

SLIDE 55

Proof Sketch, Contd.

1. By the relation between W (FIFO)(L) and X(FIFO):

[Maximizing W (Flex−FIFO)(L)] ≡ [Maximizing X(Flex−FIFO)].

SLIDE 56

Proof Sketch, Contd.

1. By the relation between W (FIFO)(L) and X(FIFO):

[Maximizing W (Flex−FIFO)(L)] ≡ [Maximizing X(Flex−FIFO)].

2. The sum X(FIFO,Σ) is maximized when ρsn is minimized.

SLIDE 57

Proof Sketch, Contd.

1. By the relation between W (FIFO)(L) and X(FIFO):

[Maximizing W (Flex−FIFO)(L)] ≡ [Maximizing X(Flex−FIFO)].

2. The sum X(FIFO,Σ) is maximized when ρsn is minimized.
3. By Order-Independence, we can now cycle through all

starting orders

SLIDE 58

Proof Sketch, Contd.

1. By the relation between W (FIFO)(L) and X(FIFO):

[Maximizing W (Flex−FIFO)(L)] ≡ [Maximizing X(Flex−FIFO)].

2. The sum X(FIFO,Σ) is maximized when ρsn is minimized.
3. By Order-Independence, we can now cycle through all

starting orders —which makes us minimize all of the ρ-values

SLIDE 59

Proof Sketch, Contd.

1. By the relation between W (FIFO)(L) and X(FIFO):

[Maximizing W (Flex−FIFO)(L)] ≡ [Maximizing X(Flex−FIFO)].

2. The sum X(FIFO,Σ) is maximized when ρsn is minimized.
3. By Order-Independence, we can now cycle through all

starting orders —which makes us minimize all of the ρ-values —which makes us have all computers run at full speed.

SLIDE 60

Proof Sketch, Contd.

1. By the relation between W (FIFO)(L) and X(FIFO):

[Maximizing W (Flex−FIFO)(L)] ≡ [Maximizing X(Flex−FIFO)].

2. The sum X(FIFO,Σ) is maximized when ρsn is minimized.
3. By Order-Independence, we can now cycle through all