Probabilistic Inference Applied to Game Heuristics Karl Cronburg - - PowerPoint PPT Presentation

Probabilistic Inference Applied to Game Heuristics

Karl Cronburg karl@cs.tufts.edu

Tufts University

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 1 / 28

Motivation

Primary Goal: Want to judge the strength & limitations of a game-play heuristic Domain experts model ‘winning‘ strategies Want to then visualize the probabilistic effects of changing the values

• f parameters in the state space

A probabilistic map of the state space where the domain expert specifies interesting variables to plot distributions over

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 2 / 28

MCMC Sampling Methods

Rejection Sampling - known model for joint distribution Random Walk

◮ Metropolis-Hastings - known model for joint distribution ◮ Gibbs Sampling - for hard-to-sample joint distribution Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 3 / 28

Probabilistic Languages

Hakaru: an EDSL in Haskell

◮ Powerful host language support (fully embedded) ◮ Somewhat lacking in domain-specific features (limited API)

BLOG: a Java-based DSL

◮ Poweful model of computation (many-worlds modeling) ◮ Lacking in domain-specific tool support and language features

Our discussion could be extended to Church, Figaro, Fun, others languages discussed in class... Focus on Hakaru for this presentation

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 4 / 28

Sampling - Hakaru

1 test :: IS.Measure Double 2 test = IS.unconditioned (normal (-3) 1) 3 test2 = IS.unconditioned (uniform 1 5) 4 5 do 6 t <- IS.sample test [] 7 let s = take 1000 $ map fst t 8 t2 <- IS.sample test2 [] 9 let s2 = take 1000 $ map fst t2 10 return (makeHistogram 30 (V.fromList (s ++ s2)) "Histogram")

Can arbitrarily compose Hakaru with Haskell code - single syntax Conditioned sampling on recursive models?

1Example taken from http://indiana.edu/ ppaml/HakaruTutorial.html Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 5 / 28

Case Study: Deckbuilding Games

We chose Dominion1to model in a probabilistic language The Process:

◮ Write sampling model based on game mechanics Dominion ◮ Determine interesting state variables to condition on ◮ Run Hakaru MCMC ◮ Plot resulting joint distributions for domain expert to explore 1by D. X. Vaccarino Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 6 / 28

Case Study: Dominion Mechanics

λ> runGreedy (0.5, 0.5) Player1: name = "Greedy1" hand = [ESTATE, GOLD, PROVINCE, PROVINCE, SILVER] inPlay = [] deck = [SILVER, PROVINCE, COPPER, SILVER, COPPER ,ESTATE, COPPER, COPPER, VILLAGE, VILLAGE ,PROVINCE, ESTATE, SILVER, COPPER] dscrd = [SILVER, SILVER, SILVER, COPPER, COPPER, PROVINCE] buys=1, actions=1, money=0 Player2: name = "Greedy2" hand = [COPPER, COPPER, COPPER, COPPER, VILLAGE] inPlay = [] deck = [SILVER, SILVER, GOLD, COPPER, COPPER, ESTATE ,GOLD, ESTATE, GOLD, ESTATE, PROVINCE, VILLAGE ,PROVINCE, GOLD] dscrd = [SILVER, COPPER, SILVER, SILVER, SILVER, PROVINCE] buys=1, actions=1, money=0 Trash: [] Supply: [(COPPER,60), (CELLAR,10), (MOAT,10), (ESTATE,8) ,(SILVER,27), (VILLAGE,6), (WOODCUTTER,10), (WORKSHOP,10) ,(MILITIA,10), (REMODEL,10), (SMITHY,10), (MARKET,10) ,(MINE,10), (DUCHY,8), (GOLD,25), (PROVINCE,0)] Turn #: 30 Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 7 / 28

Case Study: Choice of Parameters

Initially focus on binary choices in card-buying-heuristic

◮ If we have 3 treasure on a turn... ⋆ Buy VILLAGE with probability p0 ⋆ Buy CHANCELLOR with probability p1 = 1 − p0 ◮ Condition on number of turns in a game ◮ What does the distribution P(p0|turns = t) for some t look like? 1card content by D. X. Vaccarino Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 8 / 28

Case Study: Greedy vs Greedy Dominion

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 9 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 10 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 11 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 12 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 13 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 14 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 15 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 16 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 17 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 18 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 19 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 20 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 21 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 22 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 23 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 24 / 28

Case Study: Rejection Sampling

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 25 / 28

Case Study: Future

For paper:

◮ New game heuristic model(s)

Metropolis Hastings with conditioning on other heuristics

◮ # Points ◮ Average hand value on a given turn

Model-Learner on public data sets

Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 26 / 28

Technical Difficulties & Future

Haskell monad transformers - Hakaru needs more monadic support Random numbers - Hakaru needs cleaner randomness support Need graceful handling of runtime errors - preferably none Difficult to distinguish Hakaru bugs from model bugs Card shuffling bug: *** Exception: [extractTree] impossible

λ> runMetrop 25 100 [(0.9799143491179355,1.0),(0.9799143491179355,1.0),(0.9799143491179355,1.0),... Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 27 / 28

Acknowledgements

Norman stackoverflow learnyouahaskell.com

P Hakaru Code Runs Correctly | Type Checks

• Title text: This is roughly equivalent to ’number of

times I’ve picked up a seashell at the ocean’ / ’number

• f times I’ve picked up a seashell’, which in my case

is pretty close to 1, and gets much closer if we’re considering only times I didn’t put it to my ear. Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 28 / 28