Chapter2 Intelligent Agents 2 20070308 chap2 1 20070308 - - PDF document

SLIDE 1

1

20070308 chap2 1

Chapter2

Intelligent Agents

20070308 chap2

2

What Is An Agent ?

20070308 chap2

3

What Is An Agent ? (cont.-1)

An agent interacts with its environments through sensors and actuators.

• Perceiving through sensors
• human agent: eyes, ears, etc.
• robot agent: cameras, infrared, etc.
• software agent: receiving keystrokes, file contents,

network packets, etc.

• Acting through actuators
• human agent: hands, legs, mouth, etc.
• robot agent: arms, motors, etc.
• software agent: displaying on the screen,

sending network packets, etc.

20070308 chap2

4

• Agent function (abstract mathematical description)

that maps any given percept sequence to an action.

• Agent program (concrete implementation)

that implements the agent function, running on the physical architecture to produce f.

• A rational agent is one that does the right thing.

What Is An Agent ? (cont.-2)

20070308 chap2

5

Vacuum-cleaner world

Percepts: location and contents, e.g., [A, Dirty] Actions: Left, Right, Suck, NoOp

20070308 chap2

6

• agent function
• agent program

What is the right function? Can it be implemented in a small agent program?

Vacuum-cleaner world (cont.)

SLIDE 2

2

20070308 chap2

7

Performance Measure

The right action is the one that will cause the agent to be most successful.

• The measure should be objective.
• How does one evaluate success?
• When does one evaluate success?
• Example: to vacuum a dirty floor

the cleanness of the floor the amount of dirt cleaned up the amount of electricity consumed the amount of noise generated total time and effort spent performance over a short/long time

20070308 chap2

8

Rational Agents

What is rational depends on

• Performance measure - degree of success
• Percept sequence to date
• The agent's knowledge about the environment
• Actions that can be performed by the agent

Definition of a rational agent For each possible percept sequence, rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.

20070308 chap2

9

Rational Agents (cont.)

• rationality ≠ omniscience (全知)

e.g. crossing the road and be flattened

• rationality ≠ perfection

Rationality maximizes expected performance, while perfection maximizes actual performance.

• rationality = exploration + learning + autonomy

e.g. lowly dung beetles, female sphex wasp Information gathering/exploration -- To maximize future rewards Learn from percepts -- To extend prior knowledge Agent autonomy -- To compensate for incorrect/partial prior knowledge

20070308 chap2

10

Specifying the Task Environment

PEAS (Performance, Environment, Actuators, Sensors)

e.g. Automated Taxi

20070308 chap2

11

Specifying the Task Environment (cont.)

e.g. Medical diagnosis system

Performance measure?? Healthy patients, minimize costs Environment?? Patient, hospital, staff Actuators?? Display questions, tests, treatments, diagnoses, referrals Sensors?? Keyboard entry of symptoms, findings, patient’s answer

20070308 chap2

12

Properties of Task Environments

Single-agent?? Discrete?? Static?? Episodic?? Deterministic?? Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

SLIDE 3

3

20070308 chap2

13

Properties of Task Environments (cont.-1)

Single-agent?? Discrete?? Static?? Episodic?? Deterministic?? Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Fully vs. partially observable: an environment is full observable when the

sensors can detect all aspects that are relevant to the choice of action.

20070308 chap2

14

Properties of Task Environments (cont.-2)

Single-agent?? Discrete?? Static?? Episodic?? Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Fully vs. partially observable: an environment is full observable when the

sensors can detect all aspects that are relevant to the choice of action.

20070308 chap2

15

Properties of Task Environments (cont.-3)

Single-agent?? Discrete?? Static?? Episodic?? Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Deterministic vs. stochastic: if the next environment state is completely

determined by the current state the executed action then the environment is

• deterministic. If it is deterministic except for actions of other agents, we say the

environment is strategic.

20070308 chap2

16

Properties of Task Environments (cont.-4)

Single-agent?? Discrete?? Static?? Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Deterministic vs. stochastic: if the next environment state is completely

determined by the current state the executed action then the environment is

• deterministic. If it is deterministic except for actions of other agents, we say the

environment is strategic.

20070308 chap2

17

Properties of Task Environments (cont.-5)

Single-agent?? Discrete?? Static?? Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Episodic vs. sequential: In an episodic environment the agent’s experience

can be divided into atomic steps where the agents perceives and then performs a single action. The choice of action depends only on the episode itself .

20070308 chap2

18

Properties of Task Environments (cont.-6)

Single-agent?? Discrete?? Static?? NO NO NO NO Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Episodic vs. sequential: In an episodic environment the agent’s experience

can be divided into atomic steps where the agents perceives and then performs a single action. The choice of action depends only on the episode itself .

SLIDE 4

4

20070308 chap2

19

Properties of Task Environments (cont.-7)

Single-agent?? Discrete?? Static?? NO NO NO NO Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Static vs. dynamic: If the environment can change while the agent is choosing

an action, the environment is dynamic. Semi-dynamic if the agent’s performance changes even when the environment remains the same.

20070308 chap2

20

Properties of Task Environments (cont.-8)

Single-agent?? Discrete?? NO SEMI YES YES Static?? NO NO NO NO Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Static vs. dynamic: If the environment can change while the agent is choosing

an action, the environment is dynamic. Semi-dynamic if the agent’s performance changes even when the environment remains the same.

20070308 chap2

21

Properties of Task Environments (cont.-9)

Single-agent?? Discrete?? NO SEMI YES YES Static?? NO NO NO NO Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Discrete vs. continuous: This distinction can be applied to the state of the environment, the way time is handled and to the percepts/actions of the agent.

20070308 chap2

22

Properties of Task Environments (cont.-10)

Single-agent?? NO YES YES YES Discrete?? NO SEMI YES YES Static?? NO NO NO NO Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Discrete vs. continuous: This distinction can be applied to the state of the environment, the way time is handled and to the percepts/actions of the agent.

20070308 chap2

23

Properties of Task Environments (cont.-11)

Single-agent?? NO YES YES YES Discrete?? NO SEMI YES YES Static?? NO NO NO NO Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Single vs. multi-agent: Does the environment contain other agents who

are also maximizing some performance measure that depends on the current agent’s actions?

20070308 chap2

24

Properties of Task Environments (cont.-12)

NO NO NO YES Single-agent?? NO YES YES YES Discrete?? NO SEMI YES YES Static?? NO NO NO NO Episodic?? NO YES NO YES Deterministic?? PARTIAL PARTIAL FULL FULL Observable?? Taxi Internet shopping Backgammom

(西洋雙陸棋戲)

Solitaire

(接龍)

Single vs. multi-agent: Does the environment contain other agents who

are also maximizing some performance measure that depends on the current agent’s actions?

SLIDE 5

5

20070308 chap2

25

Properties of Task Environments (cont.-13)

• The simplest environment is

Fully observable, deterministic, episodic, static, discrete and single-agent.

• Most real situations are

Partially observable, stochastic, sequential, dynamic, continuous and multi-agent.

20070308 chap2

26

The Structure of Agents

Agent = Architecture + Program

• Agent program

implements the agent function mapping from percepts to actions input from the sensors: current percepts return: an action to the actuators

• Agent architecture

runs the program on some sort of computing device with physical sensors and actuators

20070308 chap2

27

Table Driven Agent

Why the table-driven approach to agent construction is doomed to failure?

• space requirement
• time requirement in table construction
• lack of autonomy
• slow learning of the “right” values

20070308 chap2

28

Four Basic kinds of Agent Program

• Simple reflex agents
• Model-based reflex agents
• Goal-based agents
• Utilities-based agents

All these can be turned into learning agents

20070308 chap2

29

Simple Reflex Agents

• select actions on the basis on the current percept,

ignoring the rest of the percept history.

• work only if the environment is fully observable

20070308 chap2

30

Simple Reflex Agents (cont.)

e.g. two-state vacuum environment

Reduction from 4T to 4 entries

function SIMPLE-REFLEX-AGENT(percept) returns an action static: rules, a set of condition-action rules state ← INTERPRET-INPUT(percept) rule ← RULE-MATCH(state, rule) action ← RULE-ACTION[rule] return action

SLIDE 6

6

20070308 chap2

31

Model-based Reflex Agents

i.e. Reflex agents with state

20070308 chap2

32

Model-Based Reflex Agents (cont.)

Updating internal state requires knowledge about

• Which perceptual information is significant?
• How does the world evolves?
• What are the effects of the agent's actions?

function REFLEX-AGENT-WITH-STATE(percept) returns an action static: rules, a set of condition-action rules state, a description of the current world state action, the most recent action. state ← UPDATE-STATE(state, action, percept) rule ← RULE-MATCH(state, rule) action ← RULE-ACTION[rule] return action

20070308 chap2

33

Goal-Based Agents

• The agent needs a goal to know which situations are desirable.
• The decision-making process involves reasoning about the future.
• Typically investigated in search and planning research.
• It is less efficient but more flexible, since knowledge is represented explicitly

and can be manipulated.

20070308 chap2

34

Utility-Based Agents

20070308 chap2

35

Utility-Based Agents (cont.)

• Utility function
• maps a (sequence of) state(s) onto a real number
• explicit
• Improves on goals:
• selecting between conflicting goals
• select appropriately between several goals based on likelihood
• f success.

20070308 chap2

36

Learning Agents

SLIDE 7

7

20070308 chap2

37

Learning Agents (cont.)

• Learning element
• Introducing improvements in performance element.
• Critic provides feedback on agents performance based on fixed

performance standard.

• Performance element
• Selecting actions based on percepts.
• Corresponding to the previous agent programs
• Problem generator
• Suggesting actions that will lead to new and informative experiences.
• Exploration vs. exploitation