Survey of Artificial Intelligence for Card Games and Its Application - - PowerPoint PPT Presentation
Survey of Artificial Intelligence for Card Games and Its Application to the Swiss Game Jass J. Niklaus *1 , M. Alberti *1 , V. Pondenkandath 1 , R. Ingold 1 , M. Liwicki 12 *Equal contribution 1 DIVA Group, University of Fribourg, Switzerland 2
*Equal contribution
1DIVA Group, University of Fribourg, Switzerland 2EISLAB Machine Learning, Luleå University of Technology, Sweden
Great Success of Artificial Intelligence in Games in last decades AlphaGO (GO), AlphaZero (Chess), Maluuba (Pacman), …
2
Hidden information makes games hard Are very close to real-world application energy optimization in datacenters, surgical operations, business, physics, …
3
Recently several milestones have been reached OpenAI Five (Dota II), Libratus (Poker), AlphaStar (StarCraft II), …
4
Trick-taking traditional Swiss card game Hidden information Sequential Non-cooperative Finite Constant-sum Schieber variant with 4 players
5
Activity within team → Coordination Game Can convey meaning by playing cards according to common protocol Agreements like “discarding policy” Example: Top-Down Player 3 plays low Diamond Card to signal strong Diamond Suit (probably Ace or at least King) to player 1
6
Player 1 Player 3 Player 2
Competitive and Cooperative Cooperation is key to success on high level Multiplayer Game Hidden Information → Suitable testbed “New Frontier of AI Research” (Deepmind) Purely cooperative Multiplayer Game Hidden Information
7
Survey of existing methods for card games Rule Based, Evolutionary, RL, MCTS Starting point for research in hidden information or card games Discussion of methods on use case Jass
8
Leverage human knowledge Simple Used as baselines Hanabi is solved only by RB systems so far Can be seen as man-made decision trees
9
Inspired by evolutionary theory Survival of the fittest Example: Population: Blackjack strategy Fitness function: money after playing the game for N iterations Example: Population: Hearthstone deck Fitness function: score after playing versus established human-designed decks
10
11
12
Temporal Difference Learning Policy Gradient Counterfactual Regret Minimization (CFR) CFR+ Deep CFR Discounted CFR Neural Fictitious Self-Play First Order Methods
Problem analytically very hard or impossible to solve Stochastic solution: big number of random experiments Example: Approximate π, 4*probability that random point in square is within the circle
14
15
Monte Carlo Simulation Flat Monte Carlo Monte Carlo Tree Search Upper Confidence Bound for Trees Determinization Information Set Monte Carlo Tree Search Monte Carlo Sampling for Regret Minimization
MCTS and CFR most successfully applied to card games CFR only used in Poker so far Approaches NE Bad at exploiting opponents MCTS applied to plethora of complex card games No guarantees for approaching NE Good at finding good solution fast Choice depends on the goal AI that never loses, costs don’t matter → CFR AI that performs well, can exploit opponents → MCTS
Jass Server: http://jass.joeli.to Play against MCTS bot! Suggestion engine based on MCTS bot Beats the average human player (par-human) Introduction Video: bit.ly/jass_intro Experiment: bit.ly/jass_form
18
19
Rule-based is replaced by most popular methods MCTS and CFR MCTS finds good strategies fast, but is exploitable by very good opponents CFR is not exploitable but cannot exploit others Jass is a hard game and hence suitable for testbed of new technologies and methods in the field. We propose using MCTS and our preliminary results suggest it’s a good idea Our bot performs at par-human level