[PPT] - Gittins Policy on NBUE + DHR ( k ) Job Sizes Matthew Maurer PowerPoint Presentation

SLIDE 1

Gittins Policy on NBUE + DHR(k) Job Sizes

Matthew Maurer Performance Modeling, 2009

Matthew Maurer () Gittins Policy CS 286.2b, 2009 1 / 25

SLIDE 2

Outline

1

Gittins Policy Gittins Index Gittins Policy Application

2

NBUE + DHR(k) Distributions Gittins Reduction to FCFS + FB(θ)

Gittins Index Properties Policy Properties

Pareto Example

Matthew Maurer () Gittins Policy CS 286.2b, 2009 2 / 25

SLIDE 3

Outline

1

Gittins Policy Gittins Index Gittins Policy Application

2

NBUE + DHR(k) Distributions Gittins Reduction to FCFS + FB(θ)

Gittins Index Properties Policy Properties

Pareto Example

Matthew Maurer () Gittins Policy CS 286.2b, 2009 3 / 25

SLIDE 4

Gittins Index Motivation

K-Armed Bandit Problem Optimal Blind Scheduling

Matthew Maurer () Gittins Policy CS 286.2b, 2009 4 / 25

SLIDE 5

Gittins Index Motivation

K-Armed Bandit Problem Optimal Blind Scheduling

Matthew Maurer () Gittins Policy CS 286.2b, 2009 4 / 25

SLIDE 6

Gittins Index Candidates

Payoff?

◮ Costs not accounted for

Payoff - Investment?

◮ Doesn’t make sense – Payoff and Investment are not necessarily in

the same units

? Maximal Ratio of Payoff to Investment

Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

SLIDE 7

Gittins Index Candidates

Payoff?

◮ Costs not accounted for

Payoff - Investment?

◮ Doesn’t make sense – Payoff and Investment are not necessarily in

the same units

? Maximal Ratio of Payoff to Investment

Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

SLIDE 8

Gittins Index Candidates

Payoff?

◮ Costs not accounted for

Payoff - Investment?

◮ Doesn’t make sense – Payoff and Investment are not necessarily in

the same units

? Maximal Ratio of Payoff to Investment

Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

SLIDE 9

Gittins Index Candidates

Payoff?

◮ Costs not accounted for

Payoff - Investment?

◮ Doesn’t make sense – Payoff and Investment are not necessarily in

the same units

? Maximal Ratio of Payoff to Investment

Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

SLIDE 10

Gittins Index Candidates

Payoff?

◮ Costs not accounted for

Payoff - Investment?

◮ Doesn’t make sense – Payoff and Investment are not necessarily in

the same units

? Maximal Ratio of Payoff to Investment

Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

SLIDE 11

Gittins Index Candidates

Payoff?

◮ Costs not accounted for

Payoff - Investment?

◮ Doesn’t make sense – Payoff and Investment are not necessarily in

the same units

? Maximal Ratio of Payoff to Investment

Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

SLIDE 12

Scheduling View of Gittins Index

We parameterize the Gittins Index over

◮ a, the current age of the job ◮ T, the service quota

We can think of varying T as varying the investment. J(a, T) = E[Job Completes|T]

E[TCompletion|T]

=

R T

0 f(a+t)dt

R T

0 ¯

F(a+t)

G(a) = supT≥0 J(a, t)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

SLIDE 13

Scheduling View of Gittins Index

We parameterize the Gittins Index over

◮ a, the current age of the job ◮ T, the service quota

We can think of varying T as varying the investment. J(a, T) = E[Job Completes|T]

E[TCompletion|T]

=

R T

0 f(a+t)dt

R T

0 ¯

F(a+t)

G(a) = supT≥0 J(a, t)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

SLIDE 14

Scheduling View of Gittins Index

We parameterize the Gittins Index over

◮ a, the current age of the job ◮ T, the service quota

We can think of varying T as varying the investment. J(a, T) = E[Job Completes|T]

E[TCompletion|T]

=

R T

0 f(a+t)dt

R T

0 ¯

F(a+t)

G(a) = supT≥0 J(a, t)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

SLIDE 15

Scheduling View of Gittins Index

We parameterize the Gittins Index over

◮ a, the current age of the job ◮ T, the service quota

We can think of varying T as varying the investment. J(a, T) = E[Job Completes|T]

E[TCompletion|T]

=

R T

0 f(a+t)dt

R T

0 ¯

F(a+t)

G(a) = supT≥0 J(a, t)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

SLIDE 16

Scheduling View of Gittins Index

We parameterize the Gittins Index over

◮ a, the current age of the job ◮ T, the service quota

We can think of varying T as varying the investment. J(a, T) = E[Job Completes|T]

E[TCompletion|T]

=

R T

0 f(a+t)dt

R T

0 ¯

F(a+t)

G(a) = supT≥0 J(a, t)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

SLIDE 17

Scheduling View of Gittins Index

We parameterize the Gittins Index over

◮ a, the current age of the job ◮ T, the service quota

We can think of varying T as varying the investment. J(a, T) = E[Job Completes|T]

E[TCompletion|T]

=

R T

0 f(a+t)dt

R T

0 ¯

F(a+t)

G(a) = supT≥0 J(a, t)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

SLIDE 18

Outline

1

Gittins Policy Gittins Index Gittins Policy Application

2

NBUE + DHR(k) Distributions Gittins Reduction to FCFS + FB(θ)

Gittins Index Properties Policy Properties

Pareto Example

Matthew Maurer () Gittins Policy CS 286.2b, 2009 7 / 25

SLIDE 19

Gittins Policy Motivation

We are usually blind We usually know the distribution, and can approximate it well after some startup time if not Optimal!

Matthew Maurer () Gittins Policy CS 286.2b, 2009 8 / 25

SLIDE 20

Gittins Policy Motivation

We are usually blind We usually know the distribution, and can approximate it well after some startup time if not Optimal!

Matthew Maurer () Gittins Policy CS 286.2b, 2009 8 / 25

SLIDE 21

Gittins Policy Motivation

We are usually blind We usually know the distribution, and can approximate it well after some startup time if not Optimal!

Matthew Maurer () Gittins Policy CS 286.2b, 2009 8 / 25

SLIDE 22

Gittins Index Computation

Exact

◮ To compute G(a) exactly, we have to compute J(a, T) for some T. ◮ We need to take the analytic minimum of J(a, T) w/rspt to T.

Approximation

◮ We can approximate J(a, T) easily ◮ Optimiztion of a computationally expensive function over the real

line...

This algorithm was initially developed for discrete time cases, and it shows.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

SLIDE 23

Gittins Index Computation

Exact

◮ To compute G(a) exactly, we have to compute J(a, T) for some T. ◮ We need to take the analytic minimum of J(a, T) w/rspt to T.

Approximation

◮ We can approximate J(a, T) easily ◮ Optimiztion of a computationally expensive function over the real

line...

This algorithm was initially developed for discrete time cases, and it shows.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

SLIDE 24

Gittins Index Computation

Exact

◮ To compute G(a) exactly, we have to compute J(a, T) for some T. ◮ We need to take the analytic minimum of J(a, T) w/rspt to T.

Approximation

◮ We can approximate J(a, T) easily ◮ Optimiztion of a computationally expensive function over the real

line...

This algorithm was initially developed for discrete time cases, and it shows.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

SLIDE 25

Gittins Index Computation

Exact

◮ To compute G(a) exactly, we have to compute J(a, T) for some T. ◮ We need to take the analytic minimum of J(a, T) w/rspt to T.

Approximation

◮ We can approximate J(a, T) easily ◮ Optimiztion of a computationally expensive function over the real

line...

This algorithm was initially developed for discrete time cases, and it shows.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

SLIDE 26

Gittins Index Computation

Exact

◮ To compute G(a) exactly, we have to compute J(a, T) for some T. ◮ We need to take the analytic minimum of J(a, T) w/rspt to T.

Approximation

◮ We can approximate J(a, T) easily ◮ Optimiztion of a computationally expensive function over the real

line...

This algorithm was initially developed for discrete time cases, and it shows.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

SLIDE 27

Gittins Index Computation

Exact

◮ To compute G(a) exactly, we have to compute J(a, T) for some T. ◮ We need to take the analytic minimum of J(a, T) w/rspt to T.

Approximation

◮ We can approximate J(a, T) easily ◮ Optimiztion of a computationally expensive function over the real

line...

This algorithm was initially developed for discrete time cases, and it shows.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

SLIDE 28

Gittins Index Computation

Exact

◮ To compute G(a) exactly, we have to compute J(a, T) for some T. ◮ We need to take the analytic minimum of J(a, T) w/rspt to T.

Approximation

◮ We can approximate J(a, T) easily ◮ Optimiztion of a computationally expensive function over the real

line...

This algorithm was initially developed for discrete time cases, and it shows.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

SLIDE 29

Gittins Policy Usage

Generalized Blind Approximztion - Impractical Specific Distributions - Analytic Simplification

Matthew Maurer () Gittins Policy CS 286.2b, 2009 10 / 25

SLIDE 30

Gittins Policy Usage

Generalized Blind Approximztion - Impractical Specific Distributions - Analytic Simplification

Matthew Maurer () Gittins Policy CS 286.2b, 2009 10 / 25

SLIDE 31

Outline

1

Gittins Policy Gittins Index Gittins Policy Application

2

NBUE + DHR(k) Distributions Gittins Reduction to FCFS + FB(θ)

Gittins Index Properties Policy Properties

Pareto Example

Matthew Maurer () Gittins Policy CS 286.2b, 2009 11 / 25

SLIDE 32

Problem Statement

Blind Distribution Head NBUE Distribution Tail DHR after k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 12 / 25

SLIDE 33

Problem Statement

Blind Distribution Head NBUE Distribution Tail DHR after k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 12 / 25

SLIDE 34

Problem Statement

Blind Distribution Head NBUE Distribution Tail DHR after k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 12 / 25

SLIDE 35

Derivative Calculation

To optimize J, we calculate its derivative

δJ δT = f(a+T) R T

0 ¯

F(a+t)dt+¯ F(a+T) R T

0 f(a+t)dt

R T

0 ¯

F(a+t)dt

If we let h represent the hazard rate of the distribution, we have

δJ δT = ¯ F(a+T)(h(a+T)−J(a,T)) R T

0 ¯

F(a+t)dt

Matthew Maurer () Gittins Policy CS 286.2b, 2009 13 / 25

SLIDE 36

Derivative Calculation

To optimize J, we calculate its derivative

δJ δT = f(a+T) R T

0 ¯

F(a+t)dt+¯ F(a+T) R T

0 f(a+t)dt

R T

0 ¯

F(a+t)dt

If we let h represent the hazard rate of the distribution, we have

δJ δT = ¯ F(a+T)(h(a+T)−J(a,T)) R T

0 ¯

F(a+t)dt

Matthew Maurer () Gittins Policy CS 286.2b, 2009 13 / 25

SLIDE 37

Derivative Calculation

To optimize J, we calculate its derivative

δJ δT = f(a+T) R T

0 ¯

F(a+t)dt+¯ F(a+T) R T

0 f(a+t)dt

R T

0 ¯

F(a+t)dt

If we let h represent the hazard rate of the distribution, we have

δJ δT = ¯ F(a+T)(h(a+T)−J(a,T)) R T

0 ¯

F(a+t)dt

Matthew Maurer () Gittins Policy CS 286.2b, 2009 13 / 25

SLIDE 38

Lemmas

We introduce the notation Ta to represent the optimal T choice for a job of age a We omit the proofs for these Lemmas for time and relevance

◮ ∀a, x : a ≤ x < a + Ta, G(a) ≤ G(x) ◮ ∀a : Ta < ∞, G(a + Ta) ≤ G(a) Matthew Maurer () Gittins Policy CS 286.2b, 2009 14 / 25

SLIDE 39

Lemmas

We introduce the notation Ta to represent the optimal T choice for a job of age a We omit the proofs for these Lemmas for time and relevance

◮ ∀a, x : a ≤ x < a + Ta, G(a) ≤ G(x) ◮ ∀a : Ta < ∞, G(a + Ta) ≤ G(a) Matthew Maurer () Gittins Policy CS 286.2b, 2009 14 / 25

SLIDE 40

Lemmas

We introduce the notation Ta to represent the optimal T choice for a job of age a We omit the proofs for these Lemmas for time and relevance

◮ ∀a, x : a ≤ x < a + Ta, G(a) ≤ G(x) ◮ ∀a : Ta < ∞, G(a + Ta) ≤ G(a) Matthew Maurer () Gittins Policy CS 286.2b, 2009 14 / 25

SLIDE 41

Lemmas

We introduce the notation Ta to represent the optimal T choice for a job of age a We omit the proofs for these Lemmas for time and relevance

◮ ∀a, x : a ≤ x < a + Ta, G(a) ≤ G(x) ◮ ∀a : Ta < ∞, G(a + Ta) ≤ G(a) Matthew Maurer () Gittins Policy CS 286.2b, 2009 14 / 25

SLIDE 42

Proof Overview

T0 ≥ k ∀a : a < T0, G(a) ≥ G(0) ∀a : a > k, G(a) is decreasing ∀T0 : T0 < ∞, G(T0) ≥ G(0)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 15 / 25

SLIDE 43

Proof Overview

T0 ≥ k ∀a : a < T0, G(a) ≥ G(0) ∀a : a > k, G(a) is decreasing ∀T0 : T0 < ∞, G(T0) ≥ G(0)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 15 / 25

SLIDE 44

Proof Overview

T0 ≥ k ∀a : a < T0, G(a) ≥ G(0) ∀a : a > k, G(a) is decreasing ∀T0 : T0 < ∞, G(T0) ≥ G(0)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 15 / 25

SLIDE 45

Proof Overview

T0 ≥ k ∀a : a < T0, G(a) ≥ G(0) ∀a : a > k, G(a) is decreasing ∀T0 : T0 < ∞, G(T0) ≥ G(0)

Matthew Maurer () Gittins Policy CS 286.2b, 2009 15 / 25

SLIDE 46

Property I

Take some x : 0 < x < k As it has a NBUE head, H(x) ≥ H(0) Converting to J, J(x, ∞) ≥ J(0, ∞)

¯ F(x) R ∞

x

¯ F(t)dt ≥ 1 R ∞ ¯ F(t)dt

Running math, we get

1 R ∞ ¯ F(t)dt ≥ F(x) R x

0 ¯

F(t)dt

Back in index form, this gives G(0) ≥ J(0, x) As x is valid from 0 to k, we have T0 ≥ k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 16 / 25

SLIDE 47

Property I

Take some x : 0 < x < k As it has a NBUE head, H(x) ≥ H(0) Converting to J, J(x, ∞) ≥ J(0, ∞)

¯ F(x) R ∞

x

¯ F(t)dt ≥ 1 R ∞ ¯ F(t)dt

Running math, we get

1 R ∞ ¯ F(t)dt ≥ F(x) R x

0 ¯

F(t)dt

Back in index form, this gives G(0) ≥ J(0, x) As x is valid from 0 to k, we have T0 ≥ k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 16 / 25

SLIDE 48

Property I

Take some x : 0 < x < k As it has a NBUE head, H(x) ≥ H(0) Converting to J, J(x, ∞) ≥ J(0, ∞)

¯ F(x) R ∞

x

¯ F(t)dt ≥ 1 R ∞ ¯ F(t)dt

Running math, we get

1 R ∞ ¯ F(t)dt ≥ F(x) R x

0 ¯

F(t)dt

Back in index form, this gives G(0) ≥ J(0, x) As x is valid from 0 to k, we have T0 ≥ k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 16 / 25

SLIDE 49

Property I

Take some x : 0 < x < k As it has a NBUE head, H(x) ≥ H(0) Converting to J, J(x, ∞) ≥ J(0, ∞)

¯ F(x) R ∞

x

¯ F(t)dt ≥ 1 R ∞ ¯ F(t)dt

Running math, we get

1 R ∞ ¯ F(t)dt ≥ F(x) R x

0 ¯

F(t)dt

Back in index form, this gives G(0) ≥ J(0, x) As x is valid from 0 to k, we have T0 ≥ k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 16 / 25

SLIDE 50

Property I

Take some x : 0 < x < k As it has a NBUE head, H(x) ≥ H(0) Converting to J, J(x, ∞) ≥ J(0, ∞)

¯ F(x) R ∞

x

¯ F(t)dt ≥ 1 R ∞ ¯ F(t)dt

Running math, we get

1 R ∞ ¯ F(t)dt ≥ F(x) R x

0 ¯

F(t)dt

Back in index form, this gives G(0) ≥ J(0, x) As x is valid from 0 to k, we have T0 ≥ k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 16 / 25

SLIDE 51

Property I

Take some x : 0 < x < k As it has a NBUE head, H(x) ≥ H(0) Converting to J, J(x, ∞) ≥ J(0, ∞)

¯ F(x) R ∞

x

¯ F(t)dt ≥ 1 R ∞ ¯ F(t)dt

Running math, we get

1 R ∞ ¯ F(t)dt ≥ F(x) R x

0 ¯

F(t)dt

Back in index form, this gives G(0) ≥ J(0, x) As x is valid from 0 to k, we have T0 ≥ k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 16 / 25

SLIDE 52

Property I

Take some x : 0 < x < k As it has a NBUE head, H(x) ≥ H(0) Converting to J, J(x, ∞) ≥ J(0, ∞)

¯ F(x) R ∞

x

¯ F(t)dt ≥ 1 R ∞ ¯ F(t)dt

Running math, we get

1 R ∞ ¯ F(t)dt ≥ F(x) R x

0 ¯

F(t)dt

Back in index form, this gives G(0) ≥ J(0, x) As x is valid from 0 to k, we have T0 ≥ k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 16 / 25

SLIDE 53

Property II

See the first lemma. The proof is omitted as it is a sufficiently general result.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 17 / 25

SLIDE 54

Property III

Setting our derivative to zero, we get the equation

¯ F(a+T)(h(a+T)−J(a,T)) R T

0 ¯

F(a+t)dt

= 0 Excluding infinite T, the ¯ F term will not zero, so we have h(a + T) = J(a, T) For a ≥ k, we have the DHR property, so G(a) = J(a, 0) = h(a) We have the DHR property, so G(a) is decreasing for a ≥ k.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 18 / 25

SLIDE 55

Property III

Setting our derivative to zero, we get the equation

¯ F(a+T)(h(a+T)−J(a,T)) R T

0 ¯

F(a+t)dt

= 0 Excluding infinite T, the ¯ F term will not zero, so we have h(a + T) = J(a, T) For a ≥ k, we have the DHR property, so G(a) = J(a, 0) = h(a) We have the DHR property, so G(a) is decreasing for a ≥ k.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 18 / 25

SLIDE 56

Property III

Setting our derivative to zero, we get the equation

¯ F(a+T)(h(a+T)−J(a,T)) R T

0 ¯

F(a+t)dt

= 0 Excluding infinite T, the ¯ F term will not zero, so we have h(a + T) = J(a, T) For a ≥ k, we have the DHR property, so G(a) = J(a, 0) = h(a) We have the DHR property, so G(a) is decreasing for a ≥ k.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 18 / 25

SLIDE 57

Property III

Setting our derivative to zero, we get the equation

¯ F(a+T)(h(a+T)−J(a,T)) R T

0 ¯

F(a+t)dt

= 0 Excluding infinite T, the ¯ F term will not zero, so we have h(a + T) = J(a, T) For a ≥ k, we have the DHR property, so G(a) = J(a, 0) = h(a) We have the DHR property, so G(a) is decreasing for a ≥ k.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 18 / 25

SLIDE 58

Property IV

See the second lemma. The proof is omitted as it is a sufficiently general result.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 19 / 25

SLIDE 59

Policy Derivation

We have ∀a : a < T0, G(a) ≥ G(0) and ∀T0 : T0 < ∞, G(T0) ≤ G(0) So, the Gittins Index passes its starting position at some point. We have ∀a : a > k, G(a) is decreasing So, the Gittins Index keeps going down after that. As we start NBUE, and end with this property, by optimality of Gittins FCFS + FB(T0) Additionally, we have the bound T0 > k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 20 / 25

SLIDE 60

Policy Derivation

We have ∀a : a < T0, G(a) ≥ G(0) and ∀T0 : T0 < ∞, G(T0) ≤ G(0) So, the Gittins Index passes its starting position at some point. We have ∀a : a > k, G(a) is decreasing So, the Gittins Index keeps going down after that. As we start NBUE, and end with this property, by optimality of Gittins FCFS + FB(T0) Additionally, we have the bound T0 > k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 20 / 25

SLIDE 61

Policy Derivation

We have ∀a : a < T0, G(a) ≥ G(0) and ∀T0 : T0 < ∞, G(T0) ≤ G(0) So, the Gittins Index passes its starting position at some point. We have ∀a : a > k, G(a) is decreasing So, the Gittins Index keeps going down after that. As we start NBUE, and end with this property, by optimality of Gittins FCFS + FB(T0) Additionally, we have the bound T0 > k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 20 / 25

SLIDE 62

Policy Derivation

We have ∀a : a < T0, G(a) ≥ G(0) and ∀T0 : T0 < ∞, G(T0) ≤ G(0) So, the Gittins Index passes its starting position at some point. We have ∀a : a > k, G(a) is decreasing So, the Gittins Index keeps going down after that. As we start NBUE, and end with this property, by optimality of Gittins FCFS + FB(T0) Additionally, we have the bound T0 > k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 20 / 25

SLIDE 63

Policy Derivation

We have ∀a : a < T0, G(a) ≥ G(0) and ∀T0 : T0 < ∞, G(T0) ≤ G(0) So, the Gittins Index passes its starting position at some point. We have ∀a : a > k, G(a) is decreasing So, the Gittins Index keeps going down after that. As we start NBUE, and end with this property, by optimality of Gittins FCFS + FB(T0) Additionally, we have the bound T0 > k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 20 / 25

SLIDE 64

Policy Derivation

We have ∀a : a < T0, G(a) ≥ G(0) and ∀T0 : T0 < ∞, G(T0) ≤ G(0) So, the Gittins Index passes its starting position at some point. We have ∀a : a > k, G(a) is decreasing So, the Gittins Index keeps going down after that. As we start NBUE, and end with this property, by optimality of Gittins FCFS + FB(T0) Additionally, we have the bound T0 > k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 20 / 25

SLIDE 65

Policy Derivation

We have ∀a : a < T0, G(a) ≥ G(0) and ∀T0 : T0 < ∞, G(T0) ≤ G(0) So, the Gittins Index passes its starting position at some point. We have ∀a : a > k, G(a) is decreasing So, the Gittins Index keeps going down after that. As we start NBUE, and end with this property, by optimality of Gittins FCFS + FB(T0) Additionally, we have the bound T0 > k

Matthew Maurer () Gittins Policy CS 286.2b, 2009 20 / 25

SLIDE 66

Outline

1

Gittins Policy Gittins Index Gittins Policy Application

2

NBUE + DHR(k) Distributions Gittins Reduction to FCFS + FB(θ)

Gittins Index Properties Policy Properties

Pareto Example

Matthew Maurer () Gittins Policy CS 286.2b, 2009 21 / 25

SLIDE 67

Qualification

Up through k, NBUE (starts at zero, then jumps) After k, DHR Fits the requirements for this application of Gittins

Matthew Maurer () Gittins Policy CS 286.2b, 2009 22 / 25

SLIDE 68

Qualification

Up through k, NBUE (starts at zero, then jumps) After k, DHR Fits the requirements for this application of Gittins

Matthew Maurer () Gittins Policy CS 286.2b, 2009 22 / 25

SLIDE 69

Qualification

Up through k, NBUE (starts at zero, then jumps) After k, DHR Fits the requirements for this application of Gittins

Matthew Maurer () Gittins Policy CS 286.2b, 2009 22 / 25

SLIDE 70

Gittins Index

Matthew Maurer () Gittins Policy CS 286.2b, 2009 23 / 25

SLIDE 71

Summary

When doing blind scheduling, Gittins Policy is optimal. The Gittins Policy is usually intractible. In our particular case Gittins reduces to FCFS + FB(T0) for NBUE + DHR(k).

Matthew Maurer () Gittins Policy CS 286.2b, 2009 24 / 25

SLIDE 72

Summary

When doing blind scheduling, Gittins Policy is optimal. The Gittins Policy is usually intractible. In our particular case Gittins reduces to FCFS + FB(T0) for NBUE + DHR(k).

Matthew Maurer () Gittins Policy CS 286.2b, 2009 24 / 25

SLIDE 73

Summary

When doing blind scheduling, Gittins Policy is optimal. The Gittins Policy is usually intractible. In our particular case Gittins reduces to FCFS + FB(T0) for NBUE + DHR(k).

Matthew Maurer () Gittins Policy CS 286.2b, 2009 24 / 25

SLIDE 74

For Further Reading

M. Pinedo.

Scheduling: Theory, Algorithms and Systems. Springer, 2008.

S. Aalto, U. Ayesta.

Optimal scheduling of jobs with a DHR tail in the M/G/1 queue. ValueTools, 2008.

J. Gittins.

Bandit Processes and Dynamic Allocation Indices. Royal Statistical Society, 2:148–177, 1979.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 25 / 25

SLIDE 75

For Further Reading

M. Pinedo.

Scheduling: Theory, Algorithms and Systems. Springer, 2008.

S. Aalto, U. Ayesta.

Optimal scheduling of jobs with a DHR tail in the M/G/1 queue. ValueTools, 2008.

J. Gittins.

Bandit Processes and Dynamic Allocation Indices. Royal Statistical Society, 2:148–177, 1979.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 25 / 25

SLIDE 76

For Further Reading

M. Pinedo.

Scheduling: Theory, Algorithms and Systems. Springer, 2008.

S. Aalto, U. Ayesta.

Optimal scheduling of jobs with a DHR tail in the M/G/1 queue. ValueTools, 2008.

J. Gittins.

Bandit Processes and Dynamic Allocation Indices. Royal Statistical Society, 2:148–177, 1979.

Matthew Maurer () Gittins Policy CS 286.2b, 2009 25 / 25