Gittins Policy on NBUE + DHR ( k ) Job Sizes Matthew Maurer - PowerPoint PPT Presentation

Gittins Policy on NBUE + DHR ( k ) Job Sizes Matthew Maurer Performance Modeling, 2009 Matthew Maurer () Gittins Policy CS 286.2b, 2009 1 / 25

Outline Gittins Policy 1 Gittins Index Gittins Policy Application NBUE + DHR ( k ) Distributions 2 Gittins Reduction to FCFS + FB ( θ ) Gittins Index Properties Policy Properties Pareto Example Matthew Maurer () Gittins Policy CS 286.2b, 2009 2 / 25

Outline Gittins Policy 1 Gittins Index Gittins Policy Application NBUE + DHR ( k ) Distributions 2 Gittins Reduction to FCFS + FB ( θ ) Gittins Index Properties Policy Properties Pareto Example Matthew Maurer () Gittins Policy CS 286.2b, 2009 3 / 25

Gittins Index Motivation K-Armed Bandit Problem Optimal Blind Scheduling Matthew Maurer () Gittins Policy CS 286.2b, 2009 4 / 25

Gittins Index Motivation K-Armed Bandit Problem Optimal Blind Scheduling Matthew Maurer () Gittins Policy CS 286.2b, 2009 4 / 25

Gittins Index Candidates Payoff? ◮ Costs not accounted for Payoff - Investment? ◮ Doesn’t make sense – Payoff and Investment are not necessarily in the same units ? Maximal Ratio of Payoff to Investment Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

Gittins Index Candidates Payoff? ◮ Costs not accounted for Payoff - Investment? ◮ Doesn’t make sense – Payoff and Investment are not necessarily in the same units ? Maximal Ratio of Payoff to Investment Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

Gittins Index Candidates Payoff? ◮ Costs not accounted for Payoff - Investment? ◮ Doesn’t make sense – Payoff and Investment are not necessarily in the same units ? Maximal Ratio of Payoff to Investment Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

Gittins Index Candidates Payoff? ◮ Costs not accounted for Payoff - Investment? ◮ Doesn’t make sense – Payoff and Investment are not necessarily in the same units ? Maximal Ratio of Payoff to Investment Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

Gittins Index Candidates Payoff? ◮ Costs not accounted for Payoff - Investment? ◮ Doesn’t make sense – Payoff and Investment are not necessarily in the same units ? Maximal Ratio of Payoff to Investment Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

Gittins Index Candidates Payoff? ◮ Costs not accounted for Payoff - Investment? ◮ Doesn’t make sense – Payoff and Investment are not necessarily in the same units ? Maximal Ratio of Payoff to Investment Matthew Maurer () Gittins Policy CS 286.2b, 2009 5 / 25

Scheduling View of Gittins Index We parameterize the Gittins Index over ◮ a , the current age of the job ◮ T , the service quota We can think of varying T as varying the investment. R T 0 f ( a + t ) dt J ( a , T ) = E [ Job Completes | T ] = R T E [ T Completion | T ] 0 ¯ F ( a + t ) G ( a ) = sup T ≥ 0 J ( a , t ) Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

Scheduling View of Gittins Index We parameterize the Gittins Index over ◮ a , the current age of the job ◮ T , the service quota We can think of varying T as varying the investment. R T 0 f ( a + t ) dt J ( a , T ) = E [ Job Completes | T ] = R T E [ T Completion | T ] 0 ¯ F ( a + t ) G ( a ) = sup T ≥ 0 J ( a , t ) Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

Scheduling View of Gittins Index We parameterize the Gittins Index over ◮ a , the current age of the job ◮ T , the service quota We can think of varying T as varying the investment. R T 0 f ( a + t ) dt J ( a , T ) = E [ Job Completes | T ] = R T E [ T Completion | T ] 0 ¯ F ( a + t ) G ( a ) = sup T ≥ 0 J ( a , t ) Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

Scheduling View of Gittins Index We parameterize the Gittins Index over ◮ a , the current age of the job ◮ T , the service quota We can think of varying T as varying the investment. R T 0 f ( a + t ) dt J ( a , T ) = E [ Job Completes | T ] = R T E [ T Completion | T ] 0 ¯ F ( a + t ) G ( a ) = sup T ≥ 0 J ( a , t ) Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

Scheduling View of Gittins Index We parameterize the Gittins Index over ◮ a , the current age of the job ◮ T , the service quota We can think of varying T as varying the investment. R T 0 f ( a + t ) dt J ( a , T ) = E [ Job Completes | T ] = R T E [ T Completion | T ] 0 ¯ F ( a + t ) G ( a ) = sup T ≥ 0 J ( a , t ) Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

Scheduling View of Gittins Index We parameterize the Gittins Index over ◮ a , the current age of the job ◮ T , the service quota We can think of varying T as varying the investment. R T 0 f ( a + t ) dt J ( a , T ) = E [ Job Completes | T ] = R T E [ T Completion | T ] 0 ¯ F ( a + t ) G ( a ) = sup T ≥ 0 J ( a , t ) Matthew Maurer () Gittins Policy CS 286.2b, 2009 6 / 25

Outline Gittins Policy 1 Gittins Index Gittins Policy Application NBUE + DHR ( k ) Distributions 2 Gittins Reduction to FCFS + FB ( θ ) Gittins Index Properties Policy Properties Pareto Example Matthew Maurer () Gittins Policy CS 286.2b, 2009 7 / 25

Gittins Policy Motivation We are usually blind We usually know the distribution, and can approximate it well after some startup time if not Optimal! Matthew Maurer () Gittins Policy CS 286.2b, 2009 8 / 25

Gittins Policy Motivation We are usually blind We usually know the distribution, and can approximate it well after some startup time if not Optimal! Matthew Maurer () Gittins Policy CS 286.2b, 2009 8 / 25

Gittins Policy Motivation We are usually blind We usually know the distribution, and can approximate it well after some startup time if not Optimal! Matthew Maurer () Gittins Policy CS 286.2b, 2009 8 / 25

Gittins Index Computation Exact ◮ To compute G ( a ) exactly, we have to compute J ( a , T ) for some T . ◮ We need to take the analytic minimum of J ( a , T ) w/rspt to T . Approximation ◮ We can approximate J ( a , T ) easily ◮ Optimiztion of a computationally expensive function over the real line... This algorithm was initially developed for discrete time cases, and it shows. Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

Gittins Index Computation Exact ◮ To compute G ( a ) exactly, we have to compute J ( a , T ) for some T . ◮ We need to take the analytic minimum of J ( a , T ) w/rspt to T . Approximation ◮ We can approximate J ( a , T ) easily ◮ Optimiztion of a computationally expensive function over the real line... This algorithm was initially developed for discrete time cases, and it shows. Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

Gittins Index Computation Exact ◮ To compute G ( a ) exactly, we have to compute J ( a , T ) for some T . ◮ We need to take the analytic minimum of J ( a , T ) w/rspt to T . Approximation ◮ We can approximate J ( a , T ) easily ◮ Optimiztion of a computationally expensive function over the real line... This algorithm was initially developed for discrete time cases, and it shows. Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

Gittins Index Computation Exact ◮ To compute G ( a ) exactly, we have to compute J ( a , T ) for some T . ◮ We need to take the analytic minimum of J ( a , T ) w/rspt to T . Approximation ◮ We can approximate J ( a , T ) easily ◮ Optimiztion of a computationally expensive function over the real line... This algorithm was initially developed for discrete time cases, and it shows. Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

Gittins Index Computation Exact ◮ To compute G ( a ) exactly, we have to compute J ( a , T ) for some T . ◮ We need to take the analytic minimum of J ( a , T ) w/rspt to T . Approximation ◮ We can approximate J ( a , T ) easily ◮ Optimiztion of a computationally expensive function over the real line... This algorithm was initially developed for discrete time cases, and it shows. Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

Gittins Index Computation Exact ◮ To compute G ( a ) exactly, we have to compute J ( a , T ) for some T . ◮ We need to take the analytic minimum of J ( a , T ) w/rspt to T . Approximation ◮ We can approximate J ( a , T ) easily ◮ Optimiztion of a computationally expensive function over the real line... This algorithm was initially developed for discrete time cases, and it shows. Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

Gittins Index Computation Exact ◮ To compute G ( a ) exactly, we have to compute J ( a , T ) for some T . ◮ We need to take the analytic minimum of J ( a , T ) w/rspt to T . Approximation ◮ We can approximate J ( a , T ) easily ◮ Optimiztion of a computationally expensive function over the real line... This algorithm was initially developed for discrete time cases, and it shows. Matthew Maurer () Gittins Policy CS 286.2b, 2009 9 / 25

Gittins Policy Usage Generalized Blind Approximztion - Impractical Specific Distributions - Analytic Simplification Matthew Maurer () Gittins Policy CS 286.2b, 2009 10 / 25

Gittins Policy Usage Generalized Blind Approximztion - Impractical Specific Distributions - Analytic Simplification Matthew Maurer () Gittins Policy CS 286.2b, 2009 10 / 25