CS 680: GAME AI WEEK 5: DRAMA MANAGEMENT 2/13/2012 Santiago Ontan - - PowerPoint PPT Presentation

CS 680: GAME AI

WEEK 5: DRAMA MANAGEMENT

2/13/2012 Santiago Ontañón santi@cs.drexel.edu https://www.cs.drexel.edu/~santi/teaching/2012/CS680/intro.html

Reminders

• Change in schedule: Week 9 (March 12) will have an

extended project help session:

• In class students are welcome to bring their laptops to discuss

projects,

• Online students are welcome to be online in the chat room for the

same purpose.

• Progress self-check indicator:
• Your progress is good is you have project 1 completed.
• Any questions from the RTS games lectures? Anything

that needs to be re-explained?

Outline

• Student Presentation:

“Adversarial Planning Through Strategy Simulation”

• Student Presentation:

“Build Order Optimization in Starcraft”

• Student Presentation:

“On-Line Case-Based Planning”

• Drama Management
• Example: Avoiding Players Getting Stuck
• Representing Game Narratives
• Player Models
• Search-Based Drama Management
• Project 2
• Project Discussion

Outline

• Student Presentation:

“Adversarial Planning Through Strategy Simulation”

• Student Presentation:

“Build Order Optimization in Starcraft”

• Student Presentation:

“On-Line Case-Based Planning”

• Drama Management
• Example: Avoiding Players Getting Stuck
• Representing Game Narratives
• Player Models
• Search-Based Drama Management
• Project 2
• Project Discussion

Drama Management

• So far, we have been concerned with:
• How can the game AI play better? (RTS games)
• How can we make it more efficient? (path-finding)
• How can it make optimal decisions? (decision making)
• Let’s turn now to a different kind of Game AI:
• How can the AI make the game experience better?

Drama Management

• Premise:
• Different players have different preferences
• Each player enjoys a different subset of the elements of a game
• Problem:
• How can a game automatically adapt to match the current player?
• Maintain the dramatic arc expected by the game designer
• Prevent the players from dealing with parts of the game not appealing to

them

• Adjust difficulty level, etc.
• Solution:
• Drama Management / Experience Management / AI Director

Drama Management

Game Engine Player

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function

Drama Management

Game Engine Drama Manager Player The Drama Manager modifies the game by executing DM Actions Game Adaptation Evaluation Function The Drama Manager

• bserves the

interaction between player and game: Player Actions

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function The DM needs a function to assess interest / difficulty / boredom / dramatic effect / etc. Player model The DM needs an adaptation strategy that decides how to change the game to maximize the evaluation function

Drama Management Examples

• Façade:
• http://www.youtube.com/watch?v=GmuLV9eMTkg
• Left 4 Dead:
• http://www.youtube.com/watch?v=VVIdHPG0wYI
• AI Director controls: spawning enemies and items, music, map (can

block or open paths)

• Not random spawns: AI Director has a set of predefined patterns

with parameter ranges, and selects the appropriate pattern with the appropriate parameters.

• Anchorhead (GraphicIFEngine):
• Demo

Purposes of Drama Management

• Maximize interest/enjoyment of the game:
• Guide players to arcs of the game more interesting for them
• Prevent players from getting bored
• Ensure dramatic intent is achieved:
• Game author wanted the game to be played in a certain way to achieve certain

dramatic effects (surprise/fear/calm/etc.)

• Achieving those effects might require different strategies for different players
• Training:
• In Training simulations, ensure the intended situations are experienced by the

player, and the expected lessons are learned.

• Adjust difficulty:
• Automatically detect the level of the player and adjust the game difficulty to the

adequate level

Outline

• Student Presentation:

“Adversarial Planning Through Strategy Simulation”

• Student Presentation:

“Build Order Optimization in Starcraft”

• Student Presentation:

“On-Line Case-Based Planning”

• Drama Management
• Example: Avoiding Players Getting Stuck
• Representing Game Narratives
• Player Models
• Search-Based Drama Management
• Project 2
• Project Discussion

Graphic Adventure

• The biggest problem in graphic adventures is that players

get constantly stuck

Example Drama Manager

• Let us illustrate how a drama manager works with a

simple example

• Game:
• Graphic Adventure
• Goal:
• Avoiding players getting frustrated when they are stuck

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function Is the player stuck? For how long? 1) Find what is the next step in the game 2) Provide some sort of hint

Evaluation Function: Is Player Stuck?

• Certain actions in graphic adventures are relevant for

advancing in the game, and some of them are not

• The player is stuck when no new “relevant action” has

been executed for a long time

• Thus, the Drama Manager needs to know which actions

are relevant for advancing in the game

Relevant Actions

• Example: the game designer could provide the DM with

an ordered list of actions:

• If the player executes the actions in such order, the game

will be completed

Take kite Take soap Take water bucket Wash car Fly kite

Detecting the Player is Stuck

• The Evaluation function could simply consist of a timer,

counting when was the last time a new action from the “relevant action list” was executed:

Take kite Take soap Take water bucket Wash car Fly kite Executed at time 6s Executed at time 22s Executed at time 24s Current time: 90s Not yet executed Not yet executed Time stuck: 66s

Game Adaptation

• The game designer could provide a series of hints for the

actions in the list.

Take kite Take soap Take water bucket Wash car Fly kite Hint 1 (subtle): character says “Oh! A kite! I love kites!” Hint 2 (obvious): character says “I wonder what could I do with that kite?” Hint 3 (obvious): an NPC points at the soap and says “I wish I had had that soap last time I washed my car!” Etc.

Game Adaptation

• When player is stuck for longer than 3 minutes, provide a

subtle hint for the next action (if available)

• When player is stuck for longer than 5 minutes, provide an
• bvious hint for the next action (if available)

Take kite Take soap Take water bucket Wash car Fly kite Executed at time 6s Executed at time 22s Executed at time 24s Next action, available clues Hint 7 (subtle) Hint 8 (obvious)

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function

Action 1 Action 2 Action 3 Action 4 Action 5 Action 6 Action n

…

Time Stuck

Hint 1 Hint 2 Hint 3 Hint 4 Hint 5 Hint m

… Player Actions Hints Time

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function

Action 1 Action 2 Action 3 Action 4 Action 5 Action 6 Action n

…

Time Stuck

Hint 1 Hint 2 Hint 3 Hint 4 Hint 5 Hint m

… Player Actions Hints Time DM also keeps track of which hints have already been given, for not repeating

Example Drama Manager

• Decides how to pace hints during game play to prevent

the player form getting stuck

• The example was very simple, but it can be enhanced

easily:

• Maximum number of hints per unit of time (to prevent too many

hints)

• Better “player stuck” detection:
• Even if the player is not executing “relevant actions”, he might be

following a systematic approach and eventually advance

• Take into account if players repeat actions of not (repeated actions are

sign of frustration)

• Detecting novice from experienced players:
• Experienced players tend to use systematic approaches (easy to detect)

Outline

• Student Presentation:

“Adversarial Planning Through Strategy Simulation”

• Student Presentation:

“Build Order Optimization in Starcraft”

• Student Presentation:

“On-Line Case-Based Planning”

• Drama Management
• Example: Avoiding Players Getting Stuck
• Representing Game Narratives
• Player Models
• Search-Based Drama Management
• Project 2
• Project Discussion

Game Narrative

• In the previous example, the Drama Manager needed a

way to infer what was the next action to be executed in the game to advance (list of “relevant actions”)

• Most Drama Managers need to reason about the plot of

the game (Game Narrative):

• To provide hints (previous example)
• To reason about which future subplot will the player like better
• To ensure the story is unfolding as the author intended
• etc.
• A lot can be learned about drama management focusing
• n simple story-based games (interactive narrative/fiction)

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function Narrative

Narrative Representation Frameworks

• Game narratives are instances of “multi-plot” narratives:
• Multiple branching points, alternative endings, etc.
• How do we represent game narratives, so the AI can

reason about them?

• Two main approaches to represent interactive narrative in

games:

• Focus on time relationships of events: plot point graphs
• Focus on content: beats

Plot Point Graphs

• Plot point:
• Event that is relevant to the game story
• Example: “the player discovers the hidden chamber”
• Plot of the game:
• Collection of plot points that constitute the game story
• Might have several “subplots” (subset of plot points that constitute a

meaningful part of the story)

• Narrative or story arc:
• Specific order in which the plot points are visited in a game execution
• Since the player has a strong influence (agency) in this order, it is hard

for the game designer to author games that tell good stories

• That is the job of the drama manager!

Plot Point Graph

• Graph where each node is a plot point, and each link is an
• rder constraint:

Leave_house Get_flask Give_bum_flask Find_magical_shop Get_amulet Give_bum_amulet Discover_book_in_sewer Get_card Read_library_book

Subset of the plot points in Anchorhead

Plot Point Graph

• Graph where each node is a plot point, and each link is an
• rder constraint:

Leave_house Get_flask Give_bum_flask Find_magical_shop Get_amulet Give_bum_amulet Discover_book_in_sewer Get_card Read_library_book

Subset of the plot points in Anchorhead Plot point Get_amulet cannot be visited before Find_magical_shop

Plot Point Graph

• Represents the complete Game Narrative.
• The subset of visited plot points defines the “narrative

state”

Leave_house Get_flask Give_bum_flask Find_magical_shop Get_amulet Give_bum_amulet Discover_book_in_sewer Get_card Read_library_book

Plot Point Graph

• In the previous example (avoiding players getting stuck),

we used a simplified form of plot point graph to represent player progress:

Game Engine Drama Manager Player Game Adaptation Evaluation Function Action 1 Action 2 Action 3 Action 4 Action 5 Action 6 Action n … Time Stuck Hint 1 Hint 2 Hint 3 Hint 4 Hint 5 Hint m … Player Actions Hints Time

Simplified plot point graph

Plot Point Graph

Complex plot point graphs Hierarchical plot points Complex ordering conditions (NOT, OR, AND, etc.) (Example plot point graph from the intro scene of a murder mystery game)

GraphicIFEngine

• The game engine provided for Project 2 has the story defined

as a graph of plotpoints:

• File “games/anchorhead/story.xml”
• Each plot point, defined as a triple:
• Preconditions: which other plot points must be satisfied
• Trigger: which player action or NPC action triggers this plot point
• Effects: events to trigger when plot point is fired
• Example: “get-flask”
• Preconditions: NOT(get-flask) AND AT(player,bar)
• Trigger:

buy(player, flask)

• Effects:

give(bartender,player,flask) say(bartender,”Here you are!”)

GraphicIFEngine

• Determining the set of relevant actions:
• Already implemented:
• IFStory.computeUserImportantActions(IFGameState gs)
• Can be used to determine if player is stuck
• Relevant actions can be automatically extracted from the

plot point graph:

• Relevant actions are those that are triggers or preconditions of any

plot point

Beat

Beat

• A Beat is a unit of story content that makes sense by

itself, e.g.: in Façade, “the characters talk about what to have for a drink”, or “Trip greets you at the door”

Beat Goal 1 Beat Goal 2 Beat Goal 3 Preconditions Postconditions Context

Beat

Beat

• A Beat is a unit of story content that makes sense by

itself, e.g.: in Façade, “the characters talk about what to have for a drink”, or “Trip greets you at the door”

Beat Goal 1 Beat Goal 2 Beat Goal 3 Preconditions Postconditions Context Preconditions: What needs to be true in order to this beat to be executable Things that will be true after this beat ends

Beat

Beat

• A Beat is a unit of story content that makes sense by

itself, e.g.: in Façade, “the characters talk about what to have for a drink”, or “Trip greets you at the door”

Beat Goal 1 Beat Goal 2 Beat Goal 3 Preconditions Postconditions Context Context defines any beat- specific responses that characters need to have in response to player actions Beat Goals define the sequence of events that need to happen in a beat. Each beat goal might have a repertoire of events that will be triggered once the goal becomes active.

Beat Example

• In the context of a “murder mystery” game
• Beat: “Police ask player about alibi” after discovered a

person beaten to death.

Beat Goal 1: Police asks Goal 2: Player justifies Goal 3: Police moves on Preconditions:

Murder discovered Police sees player with a bat Player has no alibu

Postconditions: Player has alibi Context: If player moves away, police will block him

Façade

• Façade uses Beats to represent the story
• About 200 beats
• In class presentation next week

Left 4 Dead

• The AI Director in L4D uses a system related to beats
• Each “beat” is a “wave” (of zombies), with 4 beat goals:

Goal 1: Build Up Goal 2: Sustain Peak Goal 3: Peak Fade Goal 4: Relax Director spawns enemies and threats until desired peak is reached Keeps spawning to maintain the peak for some seconds Waits for player to completely resolve the situation Waits for some fixed a mount of time, or until player advanced a certain distance

Plot points vs Beats

• Plot points focus on time ordering between events:
• Good to monitor progress of the player
• Clearly represent what is already accomplished, and what still

needs to be accomplished to complete the game

• The player has full freedom (agency) to decide what to do next
• Beats focus on specifying content without being restrictive

about the order:

• Good for allowing the Drama Manager decide what will happen

next

• Only one Beat active at a time
• The Drama Manager restricts the agency of the player, by

sequencing beats to maintain the desired story arch

Outline

• Student Presentation:

“Adversarial Planning Through Strategy Simulation”

• Student Presentation:

“Build Order Optimization in Starcraft”

• Student Presentation:

“On-Line Case-Based Planning”

• Drama Management
• Example: Avoiding Players Getting Stuck
• Representing Game Narratives
• Player Models
• Search-Based Drama Management
• Project 2
• Project Discussion

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function Narrative

Player Models

• Drama Management aims to adapt a game to specific

players

• The DM needs a player model
• Player models can capture many different things:
• Player action models: predict what the player will do next
• Player preference models: predict what the player will like
• Player emotional model: predicts the current emotional state of

the player

• Player skill model: how good is the player in dealing with certain

situations

• Etc.

In the Previous Example…

• Player Model:
• If player hasn’t advanced for more than 3 minutes -> player is

getting bored

• If player hasn’t advanced for more than 5 minutes -> player is

bored

• This is a very simplistic player emotional model:
• Tries to predict when the player will be bored

Example: Player Preference Model

• In the context of a First Person Shooter (e.g. Half-Life)
• Modeling the player preference for weapons:
• Player model could record how many times player uses each weapon
• Or even, which weapon does the player use against certain enemy

times

• Uses:
• The DM could spawn more weapons of the preferred type if the game

needs to become easier

• The DM could spawn more weapons of the non preferred type if the

player is finding the game too easy

• If the player clearly underuses a weapon X that will be needed later on

for a certain boss, the DM could force the player to learn how to use X by not spawning any of the other types of weapons (forcing the player to try X out)

Where do Player Models Come From?

• Hard coded:
• Like in the example above (predicting player boredom)
• The parameters can be extracted from user evaluations
• Like the weapon usage example above
• In RPG games (like “Elder Scrolls”), where player take choices at

the beginning (be a wizard, warrior, thief, etc.), a player model could be hardcoded for each player choice (representing populations of players that typically take that choice)

• Learned:
• Use machine learning to automatically build player models from

data

• We will cover this idea next week

Outline

• Student Presentation:

“Adversarial Planning Through Strategy Simulation”

• Student Presentation:

“Build Order Optimization in Starcraft”

• Student Presentation:

“On-Line Case-Based Planning”

• Drama Management
• Example: Avoiding Players Getting Stuck
• Representing Game Narratives
• Player Models
• Search-Based Drama Management
• Project 2
• Project Discussion

Drama Management

Game Engine Drama Manager Player Game Adaptation Evaluation Function Narrative

Game Adaptation

• Once we have determined the goal of the DM:
• Story representation
• Evaluation function (player model)
• How does the DM adapt the game to maximize the

evaluation function?

• Authored response: like in the example above about players getting

stuck

• Decision making: if the evaluation function can be seen as a utility

function, any of the techniques we covered in the RTS decision making lecture would do!

Search-based Drama Management

• Goal: maximize(minimize) the evaluation function
• Both player and DM interact with the game by executing

actions:

• Player actions: through which the player plays the game
• DM actions: through which the DM modifies the game
• Assumption: given a game state, there is a finite set of of

player and DM actions that can be executed.

Example Player and DM Actions

• Player Actions:
• Move
• Pick up objects
• Unlock a door
• etc.
• DM Actions:
• Make an NPC say something to the player (e.g. a hint)
• Make an object disappear (outside of the range of view of the

player)

• Spawn a new object (key, enemy, power-up, etc.)
• etc.

Search-based Drama Management

• Game state has two parts:
• Game Physical state: the standard game state (location of all

characters and objects, etc.)

• Game Story state: plot points or beats that have already been

visited

Game State

Search-based Drama Management

• Deciding which DM actions to

execute:

• Open a search tree (minimax-style)

where the player and the DM alternate turns

• Apply the evaluation function to

each leaf

• Instead of minimax:
• expectimax:
• DM levels use “max”
• Player levels use “average”
• Select the DM action with maximum

value

current state

state state state state state state action state action state action state action DM action DM action DM action DM action DM action

Search-based Drama Management

• Deciding which DM actions to

execute:

• Open a search tree (minimax-style)

where the player and the DM alternate turns

• Apply the evaluation function to

each leaf

• Instead of minimax:
• expectimax:
• DM levels use “max”
• Player levels use “average”
• Select the DM action with maximum

value

current state

state state state state state state action state action state action state action DM action DM action DM action DM action DM action

When propagating the values up the tree, DM action levels use the max

Search-based Drama Management

• Deciding which DM actions to

execute:

• Open a search tree (minimax-style)

where the player and the DM alternate turns

• Apply the evaluation function to

each leaf

• Instead of minimax:
• expectimax:
• DM levels use “max”
• Player levels use “average”
• Select the DM action with maximum

value

current state

state state state state state state action state action state action state action DM action DM action DM action DM action DM action

When propagating the values up the tree, player action levels use the average

Example Evaluation Function

• Goal: maintain suspense of who is the murderer until the

end.

• In terms of plot points or beats, it translates to: keep the

“discover murderer” plot point from happening for as long as possible (until very few other plot points are left to be visited)

• Easy to implement as a function, given a branch in the

search tree of the search-based drama manager

Search-based Drama Management

• Search is performed each time the player executes an

action

• If the player executes an action before the search ended,

search is aborted, and restarted after player action

• Branching factor might be very large: Monte-Carlo

techniques can help

• It has been shown to improve the overall interestingness
• f games

Experimental Evaluation

• 22 participants [4 females, 18 males]
• Age Range: [22, 37], Average age: ~24
• Typical 45 minute evaluation
• Gaming background survey
• Play Anchorhead twice (with and without DM)
• End interview questions
• Anchorhead evaluation facts
• 9 participants had no or low gaming experience
• Average game duration: 13.5 mins

Quantitative Results

2.7 2.75 2.8 2.85 2.9 2.95 3 3.05 6 26 # of Cases for Player Modeling Rating of the Interaction No DM DM

Qualitative Results

Category Player Comments

DM helped player “I would have never thought (unless told by DM) that picking up the skull would ever be useful” With DM then without DM “I realized how much the DM had helped me in the first interaction only when I played the game without the DM” Game Endings “I felt that I could influence the particular outcome in the game” Liked the Game “Game allows you to discover game elements at ones

• wn pace”

Player liked a plotpoint “I enjoyed solving the puzzle to find a magic lens”

Agency

• One of the main dangers of drama management is taking

agency away from the player

• Agency is the degree to which the player has freedom to

choose between different actions, and to which degree those actions have an effect in the game

• There must be a balance between player agency and

system agency

Outline

• Student Presentation:

“Adversarial Planning Through Strategy Simulation”

• Student Presentation:

“Build Order Optimization in Starcraft”

• Student Presentation:

“On-Line Case-Based Planning”

• Drama Management
• Example: Avoiding Players Getting Stuck
• Representing Game Narratives
• Player Models
• Search-Based Drama Management
• Project 2
• Project Discussion

Project 2: Drama Management

• Implement a Drama Manager
• n top of a game engine.
• You can use GraphicIFEngine
• The game engine comes with

a specific game defined: Anchorhead

• Feel free to modify it or

extend it if you need additional elements in your game to test your DM

Project 2: Drama Management

• Identify what do you want your Drama Manager to

accomplish (avoid player getting stuck, interestingness, etc.)

• Define the evaluation function of your DM
• Define the set of DM actions
• Define the strategy your DM will use (rule-based, search-

based, etc.)

• Experiment with your DM:
• Ask some friends to play the game with or without it, and see their
• reactions. Does the DM achieve its goal? Is the game more fun?

Project 2: Drama Management

• Example ideas:
• Try to prevent player getting stuck:
• Define Hints for some plot-points, and a drama manager that properly

provides hints at appropriate times

• Try to maintain an Aristotelian arc (next week, we’ll see how)
• Try to force the player into reaching a specific ending:
• Try to force the player into one of the 2 endings
• Better if the ending is selected based on some information about the

player

• Complex idea: use machine learning technique (explained

next week) to learn a player model, and try to maximize interest based on that player model.

Outline

• Student Presentation:

“Adversarial Planning Through Strategy Simulation”

• Student Presentation:

“Build Order Optimization in Starcraft”

• Student Presentation:

“On-Line Case-Based Planning”

• Drama Management
• Example: Avoiding Players Getting Stuck
• Representing Game Narratives
• Player Models
• Search-Based Drama Management
• Project 2
• Project Discussion

Project Discussion

• Project 1:
• Questions? Technical Problems? Assistance?
• Project 2:
• Questions? Technical Problems? Assistance?