Dungeons and DQNs Toward Reinforcement Learning Agents that Play - - PowerPoint PPT Presentation

▶
dungeons and dqns
SMART_READER_LITE
LIVE PREVIEW

Dungeons and DQNs Toward Reinforcement Learning Agents that Play - - PowerPoint PPT Presentation

Sep 19, 2022 149 likes •332 views

Dungeons and DQNs Toward Reinforcement Learning Agents that Play Tabletop Roleplaying Games LARA J. MARTIN, SRIJAN SOOD, MARK O. RIEDL 2 Its an exciting time for AI Dungeons & DQNs Martin, Sood, & Riedl AAAI Intelligent

slide-1
SLIDE 1

Dungeons and DQNs

Toward Reinforcement Learning Agents that Play Tabletop Roleplaying Games

LARA J. MARTIN, SRIJAN SOOD, MARK O. RIEDL

slide-2
SLIDE 2

It’s an exciting time for AI

2

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

slide-3
SLIDE 3

How do we push the limits of AI?

3

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

I’m sorry, Dave. I’m afraid I can’t do that.

slide-4
SLIDE 4

Games!

4

Chess - 1997 Atari Games - 2015 Go - 2016 Doom - 2016 DOTA 2 - 2018

slide-5
SLIDE 5

What about Dungeons & Dragons?

 Players create characters to play & describe

their character’s actions

 Characters exist in a shared imaginary world  Game/Dungeon Master (GM/DM) mediates

and sets up scenarios—or campaigns 5

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

slide-6
SLIDE 6

Why Dungeons and Dragons?

 Unlimited actions (discourse)  Actions can have unexpected consequences and/or

DM can get unexpected player actions

 Actions cannot cleanly map to states (model of the

world changes as game progresses)

 Distributed game world (across players and DM)  Players receive intrinsic reward for actions (unclear win

condition)

 Collaborative

6

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

slide-7
SLIDE 7

Outline

 TRPGs compared to:

 Interactive Fiction  Experience Management  Automated Story Generation

 Our starting point:

 Genre Expectation Model + Commonsense Rules

Model

 Deep Q-Learning

7

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

slide-8
SLIDE 8

TRPGs vs the World

Medium Comparison to TRPGs Interactive Fiction (IF) Playing

  • Use puzzles to uncovers pre-

existing story

  • Often simplified grammar

Experience Management (Used in Interactive Narrative)

  • Intervenes in storyline to

keep things “on track” for quality

  • Often fixed set of actions

Automatic Story Generation

  • Generates new story
  • Uses planners to create

actions for characters for well-defined domains 8

slide-9
SLIDE 9

Outline

 TRPGs compared to:

 Interactive Fiction  Experience Management  Automated Story Generation

 Our starting point:

 Genre Expectation Model + Commonsense Rules

Model

 Deep Q-Learning

9

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

slide-10
SLIDE 10

Assumptions

 No dice rolling (i.e. no combat, etc.)  Agent is always in character  GMs aren’t refereeing

10

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

slide-11
SLIDE 11

The Proposed System (Training)

11

Policy Update Exploit Explore Environment (Rule Engine & State Updater) Seq2Seq (Genre) Reward

Updated State

  • r

<None> Updated State

  • r

<None> Event Selection & Current State

Distribution of Next Events

  • r
slide-12
SLIDE 12

World Model

  • 1. Genre Expectation Model

 Seq2Seq network generates next event in the story  Trained on relevant genre

  • 2. Commonsense Rules Model

 Things that aren’t mentioned in stories (see: Principle

  • f Minimal Departure)

 Temporal & physical rules

12

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

slide-13
SLIDE 13

The Proposed System Pipeline

13

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

Human Player’s Turn

Natural Language to Event

Event to Natural Language

Event Agent’s Turn

Update State TRPG Agent Action Selection

Selected Next Event Current State

DQN

slide-14
SLIDE 14

Back to Games!

14

Atari Games - 2015 Go - 2016 Doom - 2016 DOTA 2 - 2018

slide-15
SLIDE 15

Conclusion

TRPGs are the next AlphaGo

15

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop

slide-16
SLIDE 16

Thank you!

LARA MARTIN LJMARTIN@GATECH.EDU

16

November 13 & 14, 2018 Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop