Optimal Solitaire Yahtzee Strategies Equipment 5 Dice : values 1 - PowerPoint PPT Presentation

Optimal Solitaire Yahtzee ∗ Strategies Equipment • 5 Dice : values 1 through 6 equiprobable • 1 Score Card : Tom Verhoeff Category Score Eindhoven University of Technology Aces ∗ . . . S Faculty of Math. & Computing Science Twos ∗ . . . U E Parallel Systems Threes ∗ . . . P C Fours ∗ . . . P T Fives ∗ T.Verhoeff@TUE.NL . . . E I Sixes ∗ http://wwwpa.win.tue.nl/misc/yahtzee/ . . . R O . . . N Upper Section Bonus Three of a Kind ∗ . . . S Four of a Kind ∗ . . . L E Full House ∗ . . . O C Small Straight ∗ . . . W T Large Straight ∗ . . . E I Yahtzee ∗ . . . R O *) Yahtzee is a registered trademark of Chance ∗ . . . N the Milton Bradley Company. . . . Extra Yahtzee Bonus . . . GRAND TOTAL *) Primary categories � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–1 � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–2

Playing Rules Scoring Rules Take empty score card Category Condition Score — sum 1s Aces repeat Twos — sum 2s — sum 3s Threes Roll all dice — sum 4s Fours — sum 5s Fives Keep any ∗ dice, reroll other dice — sum 6s Sixes U.S.Tot ≥ 63 35 once U. S. Bonus Keep any ∗ dice, reroll other dice ≥ 3 equals sum values Three of a Kind ≥ 4 equals sum values Four of a Kind 2+3 equals ∗ Score roll in any ∗ empty primary category 25 Full House ≥ 4 in seq. ∗ 30 Small Straight 5 in seq. ∗ 40 Large Straight until all primary categories scored 5 equals 50 Yahtzee — sum values Chance Calculate GRAND TOTAL for final score 5 equals & Extra Y. Bonus 50 at Y. 100 each Aim: Maximize final score — sum above GRAND TOTAL *) 5 y s act here as Joker , provided categories *) Player is free to choose among options y s and Yahtzee have been scored already. � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–3 � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–4

Dilemmas Random Play • First turn, first roll: 1 1 6 6 6 Without Bonuses and Jokers What to do? Expected Category Probability Score Keep 6 6 6? Aces 1 0.83 Keep all and score 25 in Full House ? 1 1.67 Twos 1 2.50 Threes 1 3.33 Fours • First turn, second roll: 1 1 3 4 6 1 4.17 Fives Sixes 1 5.00 What to do? 1656/7776 3.73 Three of a Kind 156/7776 0.35 Four of a Kind 300/7776 0.96 • First turn, third roll: 6 6 6 6 1 Full House 1200/7776 4.63 Small Straight What to do? 240/7776 1.23 Large Straight Yahtzee 6/7776 0.04 Score 24 in Sixes ? 1 17.50 Chance Score 25 in Four of a Kind ? 45.95 GRAND TOTAL � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–5 � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–6

Micro Yahtzee Game Tree Turn 1 Turn 2 • ONE die roll choose roll “choose” ✘ ✘✘ ❳❳ ✉ ❥ ❥ ✉ ❳ ✜ ✜ ✜ ✜ 1 1 • NO keeping and rerolling ✜ ✜ ✜ ✜ ✜ ✜ ✜ ✘ ✜ ✘✘ ❳❳ ✜ ✉ ❥ ✜ ❥ ✉ 2 2 ✚ ❳ ✚ ✚✚✚✚✚✚✚✚✚✚✚ ✚✚✚✚✚✚✚✚✚✚✚ ✜ ✜ ✜ ✜ ✜ ✜ • TWO primary categories: Sqr ✜ ✜ 3 ✘ 3 ✘✘ ✜ ✜ ✘✘✘✘✘✘✘✘✘✘✘ ✘ ❳❳ ✘✘✘✘✘✘✘✘✘✘✘ ✘ ✉ ❥ ✉ ❥ ✉ ❳ ✜ ✜ ✜ ✜ Dbl ✜ ✜ ❳❳❳❳❳❳❳❳❳❳❳ ✘ ❳❳❳❳❳❳❳❳❳❳❳ ✉ ✘✘✘✘✘✘✘✘✘✘✘ ✉ Category Score ❩❩❩❩❩❩❩❩❩❩❩ ❩❩❩❩❩❩❩❩❩❩❩ ❭ ❭ 4 4 ❭ ❭ ❳ ❳ ❭ ❳❳❳❳❳❳❳❳❳❳❳ ❭ value doubled ✉ ❥ Sqr ✉ ❥ Double ❭ ❭ ✜ ✜ ✚ ✚✚ ❭ ❭ 5 ✘ 5 ✘✘ ❳ ✜ ❳❳ value squared ❭ ❭ Square ✉ ❩❩ ❳ ❭ ❭ ❭ ❩ ❭ ❭ ❭ ❭ ✘ ✘✘ ❩ ❩ sum above TOTAL ❳❳ ❭ ❥ ✉ ❭ ❥ ✉ 6 ❳ 6 ❭ ❭ ❭ ❭ ❭ ❭ ❭ ❭ ✘ ❭ ✘✘ ❭ ❭ ❭ ❳❳ ❥ ✉ ❥ ✉ ❳ • How to maximize final score? Choice states are circled: ✉ ❥ • What to do if first roll is 4? # Games : 6 · 2 · 6 · 1 = 72 Score 8 in Double ? # Deterministic strategies : 2 6 · 1 6 = 64 Score 16 in Square ? � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–7 � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–8

Markov Decision Processes (sort of :-) Yahtzee as MDP • State space S = R ⊎ C The MDP is always in one state of S . • State space S = R ⊎ C R : roll states; C : choice states. • Initial state I with I ∈ S • Event sets E.s for s ∈ S • Event sets E.s for s ∈ S Roll outcomes for s ∈ R . In state s , one event from E.s occurs. Terminate if E.s = ∅ . Keep or score choices for s ∈ C . • Event probabilities p.s for s ∈ S • Event scores f.s for s ∈ S Event e ∈ E.s occurs with probability p.s.e . f.s.e = 0 for s ∈ R . � p.s.e = 1 f.s.e ≥ 0 for s ∈ C . e ∈ E.s • Event scores f.s for s ∈ S • Transition function Event e ∈ E.s scores f.s.e . R and C states alternate . Acyclic . • Transition function (juxtaposition) Event e ∈ E.s leads to next state se . � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–9 � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–10

Markov Decision Strategies Optimality Criteria • Decision strategy D defines p.s for s ∈ C Deterministic if p.s.e ∈ { 0 , 1 } • Game g after state s : • Maximize expected final score Sequence of successive events starting in s Resulting state sg : s � � = s , s ( eg ) = ( se ) g • Minimize variance in final score • Set G.s of complete games after s : • Maximize probability to beat High Score = { g | E.sg = ∅ } G.s • Maximize probability to beat opponent • Score F.s.g of game g after s : F.s. � � = 0 • Maximize minimum final score = f.s.e + F.se.g F.s.eg • Probability P.s.g of game g after s : P.s. � � = 1 = p.s.e ∗ P.se.g P.s.eg � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–11 � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–12

Optimal Strategies Recurrence Relation for E For E.s � = ∅ : • Expected final score E D by strategy D : E .s � E D = P.I.g ∗ F.I.g g ∈ G.I = { definition of E .s } � P.s.g ∗ F.s.g g ∈ G.s • Optimal strategy achieves = { g = eh with e ∈ E.s � = ∅ , h ∈ G.se } ˆ E = max E D � � P.s.eh ∗ F.s.eh D e ∈ E.s h ∈ G.se = { recurrences for P.s.eh , F.s.eh } • Conditional expectation after state s : � � p.s.e ∗ P.se.h ∗ ( f.s.e + F.se.h ) e ∈ E.s h ∈ G.se � E .s = P.s.g ∗ F.s.g = { distribution: p.s.e independent of h } g ∈ G.s � � p.s.e ∗ P.se.h ∗ ( f.s.e + F.se.h ) e ∈ E.s h ∈ G.se � • Recurrence relations : { P.x.y = 1 } = y ∈ G.x � E .s = p.s.e ∗ ( f.s.e + E .se )   � � e ∈ E.s p.s.e ∗  f.s.e + P.se.h ∗ F.se.h   e ∈ E.s h ∈ G.se � p.s.e ∗ ˆ E .se for s ∈ R   = { definition of E .se }  ˆ e ∈ E.s E .s = e ∈ E.s ( f.s.e + ˆ � max E .se ) for s ∈ C p.s.e ∗ ( f.s.e + E .se )    e ∈ E.s � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–13 � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–14

Yathzee Turn Tree Yahtzee Game Tree # nodes 1 ✉ � ✁ ❅ ❆ � ✁ ❆ ❅ ✁ ❆ � ❅ roll � ❅ • # Games : � ❅ all � ❅ � ❅ 6 5 � ❅ . . . . . . 6 5 · 7 5 · 7 5 � � 1 . 7 × 10 170 � ❅ ✉ ❥ ❥ ✉ ✉ ❥ · 13! ≈ � ✁ � ✁ ❆ ❅ ❅ ❆ � ✁ � ✁ ❆ ❅ ❆ ❅ � ✁ ✁ ❆ ❆ ❅ choose � ❅ � ❅ � ❅ keepers � ❅ � ❅ 6 5 2 5 • Probabilities : range from � ❅ . . . . . . � ❅ ✉ ✉ ✉ � ✁ � ✁ ❅ ❆ ❅ ❆ � ✁ � ✁ ❆ ❅ ❆ ❅ � ✁ ✁ ❆ ❆ ❅ 6 − 5 � 3 � 13 reroll � ❅ �� 5 . 5 × 10 − 151 � ❅ ≈ � ❅ others � ❅ � ❅ 6 5 7 5 � ❅ . . . . . . to � ❅ ❥ ✉ ❥ ✉ ✉ ❥ � ✁ � ✁ ❅ ❆ ❅ ❆ � ✁ � ✁ ❆ ❅ ❆ ❅ � ✁ ✁ ❆ ❆ ❅ � ❅ 6 − 5 � 13 choose � 3 . 8 × 10 − 50 � ❅ ≈ � ❅ keepers � ❅ � ❅ 6 5 7 5 2 5 � ❅ . . . . . . � ❅ ✉ ✉ ✉ � ✁ � ✁ ❆ ❅ ❅ ❆ � ✁ � ✁ ❆ ❅ ❆ ❅ � ✁ ✁ ❆ ❆ ❅ • # Strategies : � ❅ reroll � ❅ � ❅ others � ❅ 10 10 100 � ❅ ?? 6 5 7 5 7 5 � ❅ . . . . . . � ❅ ✉ ❥ ❥ ✉ ❥ ✉ � ✁ � ✁ ❅ ❆ ❆ ❅ � ✁ � ✁ ❆ ❅ ❆ ❅ � ✁ ✁ ❆ ❆ ❅ choose � ❅ � ❅ � ❅ category � ❅ � ❅ � ❅ . . . . . . � ❅ ✉ ✉ ✉ � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–15 � 1999–2000, Tom Verhoeff (TUE) c Yahtzee–16