Agents and Environments Lecture 2 How do we characterize - - PowerPoint PPT Presentation

▶
agents and environments
SMART_READER_LITE
LIVE PREVIEW

Agents and Environments Lecture 2 How do we characterize - - PowerPoint PPT Presentation

Sep 01, 2022 402 likes •723 views

Wentworth Institute of Technology COMP3770 Artificial Intelligence | Summer 2017 | Derbinsky Agents and Environments Lecture 2 How do we characterize environments? What is an agent? What characterizes rational behavior? Agents and

slide-1
SLIDE 1

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Agents and Environments

Lecture 2

How do we characterize environments? What is an agent? What characterizes rational behavior?

May 11, 2017 Agents and Environments 1

slide-2
SLIDE 2

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Agenda

  • Interaction model
  • Rationality
  • Task environments
  • Types of agents

May 11, 2017 Agents and Environments 2

slide-3
SLIDE 3

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Agent-Environment Interaction

? agent percepts sensors actions environment actuators

May 11, 2017 Agents and Environments 3

slide-4
SLIDE 4

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Agent-Environment Interaction

  • An agent is anything

that perceives its environment through sensors and acts via actuators

– In AI: non-trivial decision-making + significant computation

  • Percept refers to

sensor values at an instant; percept sequence is a complete history

May 11, 2017 Agents and Environments 4

slide-5
SLIDE 5

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Agent Behavior

  • Mathematically/externally, we consider the

agent function as a mapping between an arbitrary percept sequence and an action

  • As AI practitioners, we implement the

function via an agent program

May 11, 2017 Agents and Environments 5

f : P ∗ → A

slide-6
SLIDE 6

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Example: vacuum-cleaner World

A B

May 11, 2017 Agents and Environments 6

Percepts: [location, status] (e.g. [A, Dirty]) Actions: Left, Right, Suck, NoOp

slide-7
SLIDE 7

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Example vacuum-cleaner Agent

May 11, 2017 Agents and Environments 7

What is the right function?

slide-8
SLIDE 8

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Evaluating Behavior

To evaluate agent behavior, we consider a performance measure Notes:

  • Evaluates environment states, not agent percepts

(more on observability later) or states (i.e. no fooling ourselves)

  • One of many, not always easy to specify

– Should be based upon desired outcomes, not expected agent design/operation

May 11, 2017 Agents and Environments 8

f : S∗

E → V

slide-9
SLIDE 9

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Example Performance Measures

  • One point per square cleaned

– Penalize per move – Penalize for > k dirty squares …

May 11, 2017 Agents and Environments 9

A B

slide-10
SLIDE 10

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Defining Rationality

For each possible percept sequence, a rational agent should… select an action that is expected to maximize its performance measure, given…

  • 1. the percept sequence, and
  • 2. a priori (i.e. prior) knowledge.

May 11, 2017 Agents and Environments 10

slide-11
SLIDE 11

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Exercise

Provide a reasoned argument as to whether an agent executing the program below is rational given the following assumptions:

– One point for each clean square at each time step

  • ver 1000 time steps

– Geography is known, but initial environmental state is not; clean stays clean, cleaning always works – Perception is always accurate

May 11, 2017 Agents and Environments 11

slide-12
SLIDE 12

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Exercise

Provide a reasoned argument as to whether an agent executing the program below is rational given the following assumptions:

– One point for each clean square at each time step

  • ver 1000 time steps; minus one point per Watt

– Geography is known, but initial environmental state is not; clean stays clean, cleaning always works – Perception is always accurate

May 11, 2017 Agents and Environments 12

slide-13
SLIDE 13

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Rationality ≠ Omniscience

Knowing the actual outcome of one’s actions.

May 11, 2017 Agents and Environments 13

slide-14
SLIDE 14

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Rationality ≠ Perfection

Rationality implies information gathering, exploration, and learning

– Agents that rely upon prior knowledge vs. percepts lacks autonomy

May 11, 2017 Agents and Environments 14

DungBeetle

slide-15
SLIDE 15

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

PEAS Model

  • Before designing an agent, we should fully

specify the task environment (i.e. problem) it is to solve

  • Performance Measure
  • Environment
  • Actuators
  • Sensors

May 11, 2017 Agents and Environments 15

slide-16
SLIDE 16

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

PEAS: Example

  • Performance
  • Environment
  • Actuators
  • Sensors
  • Safe, fast, legal,

comfortable, profit!

  • Roads, traffic,

pedestrians, customers

  • Steering, acceleration,

brake, signal, horn, payment

  • Camera, sonar,

speedometer, GPS,

  • dometer,

accelerometer, engine

May 11, 2017 Agents and Environments 16

slide-17
SLIDE 17

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Properties of Task Environments (1)

  • Observability

– Partially vs. Fully

  • Agents

– Single vs. Multi (competitive/cooperative)

  • Certainty

– Stochastic vs. Deterministic

May 11, 2017 Agents and Environments 17

slide-18
SLIDE 18

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Properties of Task Environments (2)

  • Temporal independence

– Episodic vs. Sequential

  • Environmental change [during deliberation]

– Static vs. Dynamic

  • Representation [of states, time, percepts/actions]

– Discrete vs. Continuous

  • A priori environmental model

– Known vs. unknown

May 11, 2017 Agents and Environments 18

slide-19
SLIDE 19

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Environment: Example (1)

  • Fully observable
  • Single agent
  • Deterministic
  • Sequential
  • Static
  • Discrete
  • Known

May 11, 2017 Agents and Environments 19

slide-20
SLIDE 20

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Environment: Example (2)

  • Partially observable
  • Multi-agent, semi-

cooperative

  • Stochastic
  • Sequential
  • Dynamic
  • Continuous
  • Known

May 11, 2017 Agents and Environments 20

slide-21
SLIDE 21

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Agent Structure

  • agent = architecture + program
  • The key challenge for AI is to write

[smallish] programs that produce rational behavior given complex environments

  • We now examine 4 representative agent

architectures

May 11, 2017 Agents and Environments 21

slide-22
SLIDE 22

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Reflexive Action

May 11, 2017 Agents and Environments 22

slide-23
SLIDE 23

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Simple Reflex Agents

  • Select actions based

upon the current percept, ignoring history

  • Sees the world as it

is, does not consider future consequences

Agent Environment

Sensors What the world is like now What action I should do now Condition−action rules Actuators

May 11, 2017 Agents and Environments 23

slide-24
SLIDE 24

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Adding Planning

  • To handle partial
  • bservability, the agent

needs to maintain internal state

– Information it can’t presently sense

  • Updating requires

models of the world

– How the world evolves – Results of actions

May 11, 2017 Agents and Environments 24

slide-25
SLIDE 25

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Model-based Reflex Agents

  • Agent uses model +

state to expand inputs to rules

Agent Environment

Sensors What action I should do now State How the world evolves What my actions do Condition−action rules Actuators What the world is like now

May 11, 2017 Agents and Environments 25

slide-26
SLIDE 26

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Goal-based Agents

  • Incorporates both

what the world is like, and goals are to be achieved

  • More flexibility than

rules: as long as new information relates to goals, can adapt

Agent Environment

Sensors What it will be like if I do action A What action I should do now State How the world evolves What my actions do Goals Actuators What the world is like now

May 11, 2017 Agents and Environments 26

slide-27
SLIDE 27

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Utility-based Agents

  • Utility: internalized

performance measure

  • Expands binary nature
  • f goals
  • A rational agent must

behave as if it possesses a utility function whose expected value it tries to maximize

Agent Environment

Sensors What it will be like if I do action A How happy I will be in such a state What action I should do now State How the world evolves What my actions do Utility Actuators What the world is like now

May 11, 2017 Agents and Environments 27

slide-28
SLIDE 28

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Learning Agents

  • Performance element

converts percepts into actions

  • Learning element improves
  • ver time
  • Critic converts percepts into

good/bad (reward/penalty)

  • Problem generator

suggests actions to lead to “informative” experiences

Performance standard

Agent Environment

Sensors Performance element changes knowledge learning goals Problem generator feedback Learning element Critic Actuators

May 11, 2017 Agents and Environments 28

slide-29
SLIDE 29

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Summary (1)

  • Agents interact with environments through

sensors and actuators

  • The agent function describes what the

agent does in all circumstances; the agent program is an actual implementation

  • The performance measure evaluates the

environment sequence; a rational agent maximizes expected performance

May 11, 2017 Agents and Environments 29

slide-30
SLIDE 30

Wentworth Institute of Technology COMP3770 – Artificial Intelligence | Summer 2017 | Derbinsky

Summary (2)

  • PEAS descriptions define task environments

– Environments are described along numerous dimensions (observability, agents, certainty, temporal independence, environmental change, representation)

  • Agent = architecture + program

– Architectures: reflexive (with model]), goal-based, utility-based, learning

May 11, 2017 Agents and Environments 30