Chapter 23 Planning in the Game of Bridge Dana S. Nau University of - - PowerPoint PPT Presentation

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 1

Chapter 23 Planning in the Game of Bridge

Dana S. Nau University of Maryland 5:34 PM January 24, 2012 Lecture slides for Automated Planning: Theory and Practice

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 2

Connect Four: solved Go-Moku: solved Qubic: solved Nine Men’s Morris: solved Checkers: solved Othello: better than humans Backgammon: better than all but about 10 humans Chess: competitive with the best humans

• Bridge:

about as good as mid-level humans

Computer Programs for Games of Strategy

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 3

Computer Programs for Games of Strategy

l Fundamental technique: the minimax algorithm minimax(u) = max{minimax(v) : v is a child of u} if it’s Max’s move at u = min{minimax(v) : v is a child of u} if it’s Min’s move at u l Largely “brute force” l Can prune off portions of the tree

u cutoff depth & static evaluation function u alpha-beta pruning u transposition tables u …

l But even then, it still examines thousands of game positions l For bridge, this has some problems …

10 -3 5 9 -2 -7 2 3 10 9

• 2

3 9

• 2

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 4

l Four players; 52 playing cards dealt equally among them l Bidding to determine the trump suit

u Declarer: whoever makes highest bid u Dummy: declarer’s partner

l The basic unit of play is the trick

u One player leads; the others

must follow suit if possible

u Trick won by highest card

• f the suit led, unless

someone plays a trump

u Keep playing tricks until all

cards have been played l Scoring based on how many tricks were bid and how many were taken

West North East South 6  2  8  Q  Q  J 6  5  9  7  A  K  5  3  A  9  

How Bridge Works

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 5

l Bridge is an imperfect information game

u Don’t know what cards the others have (except the dummy) u Many possible card distributions, so many possible moves

l If we encode the additional moves as additional branches in the game tree, this increases the branching factor b l Number of nodes is exponential in b

u worst case: about 6x1044 leaf nodes u average case: about 1024 leaf nodes u A chess game may take several hours u A bridge game takes about 1.5 minutes

Game Tree Search in Bridge

Not enough time to search the game tree b =2 b =3 b =4

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 6

Reducing the Size of the Game Tree

l One approach: HTN planning

u Bridge is a game of planning u The declarer plans how to play the hand u The plan combines various strategies (ruffing, finessing, etc.) u If a move doesn’t fit into a sensible strategy, it probably doesn’t need to be

considered l Write a planning procedure procedure similar to TFD (see Chapter 11)

u Modified to generate game trees instead of just paths u Describe standard bridge strategies as collections of methods u Use HTN decomposition to generate a game tree in which each move

corresponds to a different strategy, not a different card HTN-generated trees Worst case Average case Brute-force search ≈ 1024 leaf nodes ≈ 26,000 leaf nodes ≈ 305,000 leaf nodes ≈ 6x1044 leaf nodes

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 7

… … Methods for Finessing

PlayCard(P3; S, R3) PlayCard(P2; S, R2) PlayCard(P4; S, R4) FinesseFour(P4; S) PlayCard(P1; S, R1) StandardFinesseTwo(P2; S) LeadLow(P1; S) PlayCard(P4; S, R4’) StandardFinesseThree(P3; S) EasyFinesse(P2; S) BustedFinesse(P2; S) FinesseTwo(P2; S) StandardFinesse(P2; S) Finesse(P1; S) 1st opponent declarer 2nd opponent dummy task method time ordering possible moves by 1st opponent

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 8

(North— Q)

… … Instantiating the Methods

PlayCard(P3; S, R3) PlayCard(P2; S, R2) PlayCard(P4; S, R4) FinesseFour(P4; S) PlayCard(P1; S, R1) StandardFinesseTwo(P2; S) LeadLow(P1; S) PlayCard(P4; S, R4’) StandardFinesseThree(P3; S) EasyFinesse(P2; S) BustedFinesse(P2; S) FinesseTwo(P2; S) StandardFinesse(P2; S) Finesse(P1; S) Us: East declarer, West dummy Opponents: defenders, South & North Contract: East – 3NT On lead: West at trick 3 East: KJ74 West: A2 Out: QT98653 (North— 3) East— J West— 2 North— 3 South— 5 South— Q task method 1st opponent declarer 2nd opponent dummy time ordering possible moves by 1st opponent

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 9

(North— Q)

… … Generating Part of a Game Tree

PlayCard(P3; S, R3) PlayCard(P2; S, R2) PlayCard(P4; S, R4) FinesseFour(P4; S) PlayCard(P1; S, R1) StandardFinesseTwo(P2; S) LeadLow(P1; S) PlayCard(P4; S, R4’) StandardFinesseThree(P3; S) EasyFinesse(P2; S) BustedFinesse(P2; S) FinesseTwo(P2; S) StandardFinesse(P2; S) Finesse(P1; S) (North— 3) East— J West— 2 North— 3 South— 5 South— Q The red boxes are the leaf nodes

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 10

Game Tree Generated using the Methods

N— Q E— K FINESSE N— 2 E— J N— 3 W— 2 E— K S— 3 S— Q S— 5 S— 3 W— A 3 E— 4 5 +600 CASH OUT N— S— +630 +630 +600 +600 +265 +265 +600 +600 +600 +600 +270.73 0.0078 0.0078 0.9854 0.5 0.5 +630 –100 +630 +600 +600 ... later stratagems ...

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 11

l Stephen J. Smith, then a PhD student at U. of Maryland

u Wrote a procedure to plan declarer play

l Incorporated it into Bridge Baron, an existing commercial product

u This significantly improved Bridge Baron’s declarer play u Won the 1997 world championship of computer bridge

l Since then:

u Stephen Smith is now Great Game Products’ lead programmer u He has made many improvements to Bridge Baron

» Proprietary, I don’t know what they are

u Bridge Baron was a finalist in the 2003 and 2004 computer bridge

championships » I haven’t kept track since then

Implementation

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 12

Other Approaches

l Monte Carlo simulation:

u Generate many random hypotheses for how the cards might be

distributed

u Generate and search the game trees

» Average the results

u This can divide the size of the game tree by as much as 5.2x106

» (6x1044)/(5.2x106) = 1.1x1038

• still quite large

» Thus this method by itself is not enough

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 13

Other Approaches (continued)

l AJS hashing - Applegate, Jacobson, and Sleator, 1991

u Modified version of transposition tables

» Each hash-table entry represents a set of positions that are considered to be equivalent » Example: suppose we have ♠AQ532

• View the three small cards as equivalent: ♠Aqxxx

u Before searching, first look for a hash-table entry

» Reduces the branching factor of the game tree » Value calculated for one branch will be stored in the table and used as the value for similar branches l GIB (1998-99 computer bridge champion) used a combination of Monte Carlo simulation and AJS hashing l Several current bridge programs do something similar

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 14

Top contenders in computer bridge championships, 1997–2004

Year #1 #2 #3 #4 1997 Bridge Baron Q-Plus Micro Bridge Meadowlark 1998 GIB Q-Plus Micro Bridge Bridge Baron 1999 GIB WBridge5 Micro Bridge Bridge Buff 2000 Meadowlark Q-Plus Jack WBridge5 2001 Jack Micro Bridge WBridge5 Q-Plus 2002 Jack Wbridge5 Micro Bridge ? 2003 Jack Bridge Baron WBridge5 Micro Bridge 2004 Jack Bridge Baron WBridge5 Micro Bridge I haven’t kept track since 2004 For more information see http://www.jackbridge.com/ewkprt.htm