ARTIFICIAL INTELLIGENCE Decision making: single agent Lecturer: - - PowerPoint PPT Presentation

ARTIFICIAL INTELLIGENCE

Lecturer: Silja Renooij

Decision making: single agent

Utrecht University The Netherlands

These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html

INFOB2KI 2019‐2020

2

"Five frogs were sitting on a log. One decided to jump into the

• pond. How many were left?"

Outline

• Actions vs decisions
• Decision making tools for reactive planning

– Decision trees – Finite state machines – Behavior trees

• Rule‐based expert systems

When conditions are true/false and actions succeed (or not); decide on inputs, not desires/goals

3

Action

Something which results in changes internal

• r external to agent

e.g. Precondition: in(house,fire) Action: extinguishFire Postcondition: not in(house,fire)

4

Actions

Precondition: in(house1,fire) and close(I, house1) and has(I, fireHose) and capability(I, extinguishFire) and available(water) and not in(house1,victim) and safeable(house) Action: extinguishFire(house1) Postcondition: not in(house1,fire) and soaked(house1) and charred(house1)

5

Types of Action

• World state changes

– Instantaneous; not necessarily directly visible to player/user

• ‘Cognitive’ changes (belief updates, …)

– Instantaneous; not visible, only indirectly noticeable

• In games also: animations, complex AI requests

(e.g, pathfinder)

– Duration, visible; might fail half way

6

Decisions

• Why?: object of decision (in general):

– Action, strategy, goal, classification, …

• Can we?: criteria for decision:

– World state, belief about world state, cognitive state (of others),…

• Will we?: commitment to decision:

– Priorities for outcome, expiry time, achievement time,…

7

Reactive planning

• (More recently: dynamic planning)
• Cope with dynamic and unpredictable

environments

• Compute just one next action every instant
• Sometimes use stored info on agent’s

priorities and behaviors

8

Decision trees

(handcrafted)

9

Decision trees

A B B

action1 action2 action3 action4 no yes no no yes yes

If‐then semantics: If A and B then action1 If A and not‐B then action2 If not‐A and B then action3 If not‐A and not‐B then action4 where A is short for “A=yes”, and not‐A is short for “A=no”…

10

Decision tree interpretation:

• (decide to) perform action in leaf if all conditions on

the path from root to the leaf are fulfilled

Simplifying decision trees

A B

action1 action2 action2 no yes no yes

A B

action1 action1 action2 no yes no yes

A B B

action1 action2 action2 action2 no yes no no yes yes

A B B

action1 action1 action1 action2 no yes no no yes yes

11

Decision trees

A B

action1 action2 action2 no yes no yes

A B

action1 action1 action2 no yes no yes

If A or (not‐A and B) then action1 If not‐A and not‐B then action2 Note that this is equivalent to: If A or B then action1 else action2

12

Interpretation in case multiple equivalent leaves:

• (decide to) perform action in leaf if all conditions on one of

the paths from root to the leaf are fulfilled

If A and B then action1 If not‐A or (A and not‐B) then action2 Note that this is equivalent to: If A and B then action1 else action2

Decision graphs

A B C

action1 action2 action4 no yes no no yes yes

If‐then semantics: If A and B then action1 If (A and not B) or (not A and C) then action2 If not A and not C then action4

13

More compact representation when multiple equivalent leaves:

Decision trees

A

action1 action2 action3 action4 just fed starving hungry fed

Conditions can also concern non‐binary events.

14

Example: an ant’s life If‐then semantics: If A=starving then action1 If A=hungry then action2 If A=fed then action3 If A=just fed then action4

Decision with randomness

Under attack?

Random > n ? defend patrol rest no yes no yes

15

Example: an ant’s life

Defining suitable conditions

A B

Go(north) Go(north) Go(north,west) Go(north) no yes no no yes yes

A= food(north) B=food(west) or food(south) or food(east) C=food(south) D=food(west) or food(east) E=food(west) F=food(east) C

yes no

E

no no yes Go(south) Go(north,east)

D E

yes no yes

F

no yes Go(east) Go(west) Go(random)

…

F

no yes

Good model?

16

An ant in search of food….

Using other criteria as conditions

A B

Go(north) Go(east) Go(west) Go(south) no yes no no yes yes

A=food_closest(north) B=food_closest(west) C=food_closest(south) D=food_closest(east)

C

17

Conditions

Good:

• conditions are only used in the context of another

condition

• conditions are mutually exclusive

Bad:

• all conditions apply always and all combinations

have to be checked

• several conditions can be true at the same time

18

Finite State Machines

19

State machines in games

• States: capture possible actions/behaviors of a system
• Transitions: link states, based on conditions
• The “machine” works as follows:

– start in a state – perform the behavior of that state, if any (~ do action) – transition when the condition is true – until an end state is reached

• If the number of states is finite we call the system a Finite

State Machine.

20

‘traditional’ FSM: simple example

• Machine starts ( ) in the left state; and may stop in right

terminal state ( ); action ‘wait for input’ in states

• As long as it ‘sees’ (=condition) a zero, it remains in left state
• If it ‘sees’ a one, it transitions to right state
• In right state it may stop if input from environment ceases ‐‐‐

‘accepting’ the input seen What type of bit strings does this machine accept? (strings where the OR of all bits is True)

1 0 or 1 21

Finite State Machines in games

22

For woodcutter FSM see https://www.youtube.com/watch?v=FuvaehxklOA

Game FSM

On guard run fight See small enemy See big enemy escaped Loose fight

Example: an ant’s life

23

Implementation of FSM

myState = OnGuard while myState == OnGuard: if see(small_enemy) : myState = Fight if see(big_enemy) : myState = Run guard() while myState == Fight: if loose() : myState = Run beat_enemy() while myState == Run: if escaped(): myState = OnGuard run_away() # current state # continue what you’re doing

24

FSM: handling interrupts (?)

On guard run fight See small enemy See big enemy escaped Loose fight Drink tea Drink tea Drink tea Tea finished Teatime Tea finished Tea finished Teatime Teatime

25

Not very efficient…

Hierarchical FSM

On guard run fight See small enemy See big enemy escaped Loose fight

Drink tea Tea finished Teatime

fighting

Top‐level transition:

• ‘forced’ interrupt
• return to most

recent state in ‘fighting’ FSM A (super) state can be a whole FSM: (=non‐`alarm’ level)

26

Hierarchical FSM

On guard run fight See small enemy See big enemy escaped Loose fight

Drink tea Tea finished Teatime

fighting

See no enemy

cross‐hierarchy transition

• ‘chosen’ interrupt
• return to initial state

in ‘fighting’ FSM

27

Combination: FSM‐decision tree

On guard run fight enemy small See enemy escaped Loose fight yes yes no no

28

An ant’s life: the end

Find food run fight enemy alone See enemy no See more enemies yes yes no no Get caught die Loose fight yes

29

FSM States

• States: should be disjunct cover of all

possible situations

• Activities: should be “evenly divided”
• ver states
• Activities: should be “naturally”

divided over states

• Difficulties when a character can be in

more than one state at the time…

30

Behavior Trees

31

Behavior tree

32

Behavior tree in firescenario

33

Behavior Tree

• 1. Tasks as nodes which succeed or not
• 2. Links indicate sub‐tasks
• 3. Either select one of the sub‐tasks ?

(choose first that is successful, then done; success if one sub‐task successful)

• r execute all sub‐tasks in sequence 

(bail out if fail; success only if all sub‐tasks successful)

• 4. Conditions used to choose sub‐task

34

The life of an ant

live

Normal activity

danger ?

Find food Bring food to hive multiply

? run fight

35

Summary behavior trees

• Simple form of (hierarchical) planning
• Root of tree contains all behaviors of

character

• Reactive planning: when one action fails the

next behavior is tried

• All alternatives are preplanned
• Order usually fixed, but can be influenced

through conditions

• Problems with interrupts

36

Rule‐Based Systems

• Recall: decision trees can be converted into rules
• More general rule‐based systems let you write

the rules explicitly

• System consists of:

– Knowledge:

• A rule set ‐ the rules to evaluate
• A database (working memory) ‐ stores initial and derived facts

– Reasoning:

• A matching scheme ‐ decides which rules are applicable
• A conflict resolution scheme ‐ if more than one rule is

applicable, decides how to proceed

– Explanation facilities (not discussed)

37

The knowledge base

Database with facts: health(captain,51) health(johnson,38) hold(whisker,radio) Rules: IF health(whisker,X) AND X<15 AND hold(whisker,radio) AND health(johnson,Y) AND Y>35 THEN pick_up(johnson,radio)

38

Matching and unification

Use variables in rules to make them generally applicable  unification (advanced wild‐card pattern matching) Rule: IF health(P1,X) AND X<15 AND hold(P1,radio) AND health(P2,Y) AND Y>35 THEN pick_up(P2,radio)

39

Rule types I

• 1. Rules for reasoning (book: database rewriting)

– IF enemy(X,Y) AND armed(Y,gun) AND distance(X,Y)<10 THEN in_danger(X) – IF enemy(X,Y) AND enemy(Z,Y) THEN friend(X,Z) – IF in_danger(X) AND NOT(friend(X,Z) AND distance(X,Z)<10 AND armed(Z,big‐gun)) THEN in_panic(X)

40

Reasoning rules

Reasoning rules

• are used to efficiently capture domain

knowledge

• using the database, serve for deriving new

knowledge ( database updates)

• often employed for diagnostic reasoning, i.e.

goal‐driven  inference through backward chaining

41

Backward chaining

Given a `goal’, find rules to achieve goal.

Question: friend(johnson,whisker) ?  match this with consequent of a rule:

IF enemy(X,Y) AND enemy(Z,Y) THEN friend(X,Z)

 recursively check if we can proof antecedent of rule:

enemy(johnson,Y) AND enemy(whisker,Y)

42

Rule types II

• 2. Rules for acting (book: condition‐action)

– IF enemy(X,Y) AND armed(X,gun) AND distance(X,Y)<10 THEN shoot(X,Y,gun) – IF in_panic(X) THEN run(X)

43

Action rules

Action rules

• are used to decide on actions
• uses (updated) database to match condition; if

condition true, the action part is executed

• often employed in data driven context

 inference through forward chaining

44

Rule triggering

• Match goal or fact with consequent or

antecedent (depending on forward/backward chaining)

• Any matching rule is triggered.

But: when should which triggered rule be fired?

• what if multiple rules match?
• what if multiple DB facts are matched?
• is application instant, or does it involve time?
• potentially fire any triggered rule?

45

Issue I: endless repetition

IF enemy(X,Y) AND armed(X,gun) AND distance(X,Y)<10 THEN shoot(X,Y,gun)

This rule will be triggered all the time when X is close to an

• enemy. Also after X shot him… (and killed him)!

We need a mechanism to trigger a rule only once, or only as long as it is relevant

• Solution: add a condition that is an effect of the action

– IF enemy(X,Y) AND armed(X,gun) AND distance(X,Y)<10 AND NOT(dead(Y)) THEN shoot(X,Y,gun)

46

Issue II: action duration

IF in_panic(X) THEN run(X)

After the rule is fired X starts running We check the rules again after the action (running) is finished  might be a long time!

• Solution: instead of the action, use the starting of

an action

• IF in_panic(X) THEN start(run(X))

47

Issue III: multiple DB matches

Database with facts:

health(captain,51) health(whisker,5) health(johnson,38) hold(whisker,radio)

Rule: IF health(P1,X) AND X<15 AND hold(P1,radio) AND health(P2,Y) AND Y>35 THEN pick_up(P2,radio)

• Matches: [P1=whisker, X=5, P2=captain, Y=51]

[P1=whisker, X=5, P2=johnson, Y=38]

• Solution: use first match; possibly others when first fails

48

Issue IV: multiple rules trigger

• More than one rule can be triggered

– IF enemy(X,Y) AND armed(X,gun) AND distance(X,Y)<10 THEN shoot(X,Y,gun) – IF in_panic(X) THEN run(X)

• X can be in_panic and have a gun to shoot his

enemy.

Should X run or shoot?

• Solution: mediating mechanisms (arbitration)

49

Rule arbitration

Decides which rule to fire when several are triggered. Possible mechanisms: 1.Order of rules: take first that triggers 2.(dynamic) Priorities (time consuming!) 3.Specificity: most specific condition 4.Recency: least recently used 5.…

50

Efficient triggering of rules

Combine the conditions of the rules in a network and evaluate them all at once: Rete algorithm.

holds(P1,radio) health(P1)<15 health(P2)>35 covers(P2,P1) Pick-up radio rule Change backup rule

51

Example of Rete algorithm

holds(P1,radio) health(P1)<15 health(P2)>35 covers(P2,P1) Pick-up radio rule Change backup rule

52

Bindings: P1=whisker Bindings: P1=whisker Bindings: P2=johnson P2=captain Bindings: P1=whisker P2=johnson P2=captain Bindings: P1=whisker P2=johnson P2=captain Bindings: P1=whisker P2=captain Bindings: P1=whisker P2=captain

Rule‐based System: properties

• Advantages

– Corresponds to way people often think of knowledge – Very expressive – Modular knowledge

• Easy to write and debug compared to e.g. decision trees
• More concise than e.g. FSM
• Disadvantages

– Can be memory intensive – Can be computationally intensive – Sometimes difficult to debug (rules consistent?)

53