Guarantees on the Probability of Good Selection David J. Eckman Shane G. Henderson Cornell University Cornell University Operations Research & Info Eng Operations Research & Info Eng ❞❥❡✽✽❅❝♦r♥❡❧❧✳❡❞✉ s❣❤✾❅❝♦r♥❡❧❧✳❡❞✉ Winter Simulation Conference December 10, 2018
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Selection of the Best 1 2 Frequentist PGS 3 Bayesian PGS 4 Computation Conclusion 5 S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 2/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Problem Setting • Optimize a scalar performance measure over a finite number of alternatives. • An alternative’s performance is observed with simulation noise. Examples: Alternatives Performance Measure hospital bed allocations expected blocking costs ambulance base locations expected call response time MDP policy expected discounted total cost Two alternatives − → A/B testing. More than two alternatives − → ranking and selection and exploratory MAB. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 2/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Selection of the Best in Software E.g., Simio. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 3/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Model Alternative 1 X 11 X 12 · · · i.i.d. ∼ F 1 with mean θ 1 Alternative 2 X 21 X 22 · · · i.i.d. ∼ F 2 with mean θ 2 . . . . ... . . . . . . . . Alternative k X k 1 X k 2 · · · i.i.d. ∼ F k with mean θ k Observations across alternatives are independent, unless CRN are used. Marginal distributions F i : • Ranking and selection (R&S): normal (via batching + CLT) • Multi-armed bandits: bounded support or sub-Gaussian with known variance bound The vector θ = ( θ 1 , θ 2 , . . . , θ k ) represents the (unknown) problem instance. • Assume that larger θ i is better. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 4/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Selection Events Let D be the index of the selected alternative. • Correct Selection: “Select one of the best alternatives.” CS := { θ D = θ [ k ] } . • Good Selection: “Select a δ -good alternative.” GS := { θ D > θ [ k ] − δ } . where θ [1] ≤ θ [2] ≤ · · · ≤ θ [ k ] are the ordered mean performances. Here δ represents the decision-maker’s tolerance toward making a suboptimal selection. “Close enough is good enough.” S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 5/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Fixed-Confidence Guarantees Guarantee that a certain selection event occurs with high probability: P ( GS ) (or P ( CS )) ≥ 1 − α, where 1 − α is specified by the decision-maker. Guarantee on PGS (PAC Selection) W.p. 1 − α , Alternative D is within δ of the best . � �� � � �� � � �� � Approximately Correct Probably S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 6/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Expected Opportunity Cost Another popular criteria is the expected opportunity cost (EOC)—a.k.a. linear loss. E [ L OC ] = E [ θ [ k ] − θ D ] . EOC can give a loose upper bound on PGS via Markov’s inequality: P ( GS ) = 1 − P ( θ [ k ] − θ D ≥ δ ) ≥ 1 − E [ θ [ k ] − θ D ] = 1 − E [ L OC ] . δ δ • EOC can be harder for a decision-maker to interpret or quantify. • EOC is commonly studied under a Bayesian framework. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 7/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Selection of the Best 1 2 Frequentist PGS 3 Bayesian PGS 4 Computation Conclusion 5 S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 8/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Indifference-Zone Formulation Bechhofer (1954) developed the idea of an indifference zone (IZ). For an IZ parameter δ > 0 : • Preference Zone : PZ ( δ ) = { θ : θ [ k ] − θ [ k − 1] ≥ δ } “The best alternative is at least δ better than all the others.” • Indifference Zone : IZ ( δ ) = { θ : θ [ k ] − θ [ k − 1] < δ } “There are close competitors to the best alternative.” The parameter δ is described as the smallest difference in performance worth detecting. • ...but that’s not its role in the IZ formulation. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 8/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Space of Configurations S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 9/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Goals of R&S Procedures Two Frequentist Guarantees Specify confidence level 1 − α ∈ (1 /k, 1) and δ > 0 and guarantee P θ ( CS ) ≥ 1 − α for all θ ∈ PZ ( δ ) , ( Goal PCS-PZ ) P θ ( GS ) ≥ 1 − α for all θ . ( Goal PGS ) Goal PGS = ⇒ Goal PCS-PZ . Goal PCS-PZ is the standard in the frequentist R&S community. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 10/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Goal PCS-PZ vs Goal PGS Issues with Goal PCS-PZ • Says nothing about a procedure’s performance in IZ ( δ ) . • Configurations in PZ ( δ ) may be unlikely in practice: • when there are a large number of alternatives, or • when alternatives found by a search. • Choice of δ restricts the problem. • May require making Bayesian assumptions about θ . Goal PGS has none of these issues! S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 11/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Proving Goal PGS Several ways to prove Goal PGS: 1. Lifting Goal PCS-PZ 2. Concentration inequalities 3. Multiple comparisons S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 12/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Equivalence of Goals PCS-PZ and PGS “When does Goal PCS-PZ = ⇒ Goal PGS?” Intuition: More good alternatives = ⇒ more likely to pick a good alternative. Scattered results since Fabian (1962), but none in the past 20 years. Show that some R&S procedures delivering Goal PCS-PZ also deliver Goal PGS. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 13/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Equivalence Results: Condition 1 Condition 1 (Guiard 1996) For all subsets A ⊂ { 1 , . . . , k } , the joint distribution of the estimators of θ i for i ∈ A does not depend on θ j for all j / ∈ A . “Changing the mean of an alternative doesn’t change the distribution of the estimators of other alternatives’ means.” Limitation: Can only be applied to procedures without screening. • Normal (i.i.d.): Bechhofer (1954), Dudewicz and Dalal (1975), Rinott (1978) • Normal (CRN): Clark and Yang (1986), Nelson and Matejcik (1995) • Bernoulli: Sobel and Huyett (1957) • Support [ a, b ] : Naive Algorithm of Even-Dar et al. (2006) S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 14/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Equivalence Results: Condition 2 Condition 2 (Hayter 1994) For all alternatives i = 1 , . . . , k , P θ ( Select Alternative i ) is nonincreasing in θ j for every j � = i . “Improving the mean of an alternative doesn’t help any other alternative get selected.” Limitation: Checking the monotonicity of P θ ( Select Alternative i ) is hard. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 15/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Equivalence Results: Condition 2 Procedure not satisfying Condition 2 1. Take n 0 samples of each alternative. 2. Eliminate all but the two alternatives with the highest means. 3. Take n 1 additional samples for the two surviving alternatives. 4. Select the surviving alternative with the highest overall mean. Consider the three-alternative case: θ 1 < θ 2 < θ 3 . • Track P θ ( Select Alternative 2) as θ 1 increases up to θ 2 . • Fix n 0 ≥ 1 and consider n 1 = 0 and n 1 = ∞ as extreme cases. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 16/36
G UARANTEES ON THE P ROBABILITY OF G OOD S ELECTION E CKMAN AND H ENDERSON Equivalence Results: Condition 3 Condition 3 For all alternatives i = 1 , . . . , k , P θ ( Select some alternative, j , for which θ j < θ i ) is nonincreasing in θ i . “Improving the mean of an alternative doesn’t help inferior alternatives get selected.” Condition 2 = ⇒ Condition 3. S ELECTION OF THE B EST F REQUENTIST PGS B AYESIAN PGS C OMPUTATION C ONCLUSION 17/36
Recommend
More recommend
