CM30174 + CM50206 Introduction to Intelligent Agents Semester 1, - - PDF document

Applications of Agents Agent characteristics Agent architecture Summary

CM30174 + CM50206 Introduction to Intelligent Agents Semester 1, 2009-10

Marina De Vos, Julian Padget

Introduction / version 0.4

October 5, 2010

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 1 / 35 Applications of Agents Agent characteristics Agent architecture Summary

Authors/Credits for this lecture

Material sourced from Michael Wooldridge’s book “An Introduction to Multiagent Systems”, Chapters 1 and 2 [Wooldridge, 2009]. Agentlink material supplied by Mike Luck.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 2 / 35 Applications of Agents Agent characteristics Agent architecture Summary

Content

1

Applications of Agents

2

Agent characteristics

3

Agent architecture

4

Summary

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 3 / 35

Applications of Agents Agent characteristics Agent architecture Summary

Motivation

Agents might help solve some hard problems BUT they also create new ones:

Independent action ⇒ responsibility, but whose? for what? How to engineer reliable MAS? A new challenge for SE? Software to cooperate, coordinate, negotiate, adapt, argue

Application areas? Here are some examples collected by the Agentlink network

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 4 / 35

Monday, 14 June 2010 www.dcs.kcl.ac.uk/staf/mml

Production scheduling

• ptimisation
• Simulation and optimisation of processes in a corrugated box plant
• Objective: to find the production schedule that allows stock level

reduction without compromising delivery times

• Plant modelling is complex (modelling relationships between

customer order patterns, factory capacity, machine speeds, order batching and warehouse size, etc.)

• Combines agent technology with discrete event simulation
• Used as simulation tool – it helped choose between two

customers, by determining the necessary plant capacity and the incurred costs of serving these customers

• SCA Packaging reduced inventory levels by 35% while maintaining

delivery commitments

4

Monday, 14 June 2010

Monday, 14 June 2010 www.dcs.kcl.ac.uk/staf/mml

Vessel transportation scheduling

• Ocean i-Scheduler, developed by Magenta Technology

for Tankers International

• Finds the most profitable allocation of cargoes to

vessels (oil carriers) for a fleet

• Agents model vessels and cooperate with each other to

find the optimal schedule for the entire fleet

• Schedules are adapted in real-time in response to

changes in the environment, e.g.:

• cargoes change constantly,
• tankers can fail unexpectedly,
• oil transportation costs change daily

6

Monday, 14 June 2010 Monday, 14 June 2010

www.dcs.kcl.ac.uk/staf/mml

Supply Chain Production Optimiser

• NuTech; Client: Air Liquide America
• Optimisation of production and distribution of liquefied gases
• Combines domain dependent heuristics, with genetic

algorithms and ant based optimisation:

• the genetic algorithm optimises the production schedule at each plant
• the ant algorithm optimises energy distribution routes from plant to

customer

• Solutions are adapted dynamically to take into account

fluctuations in energy prices, weather changes, client demand and desired inventory levels

• Information is fed back into the control systems that operate

the power plant

8

Monday, 14 June 2010 Monday, 14 June 2010 www.dcs.kcl.ac.uk/staf/mml

Human Variability in Computer Generated Forces

• Agent Oriented Software for the UK MoD
• Simulation of combat situations for military training
• Models the influence of moderating factors (e.g.

fatigue, cafeine intake) on soldiers’ behaviour, both at individual and team levels

• Built on Jack Intelligent Agents toolkit, makes use of

the BDI reasoning model

• Integrated with other simulation environments (CGF

systems) used by the MoD

10

Monday, 14 June 2010

Monday, 14 June 2010 www.dcs.kcl.ac.uk/staf/mml

Aerogility

• Software agents represent the Aftermarket resources -

people, assets and processes.

• For each resource we capture their purpose, business

goals and objectives.

• The interactions between the agents - Aftermarket

resources - are determined by easily changed parameters covering overall strategies, management policies and organisation configurations, as well as business processes and rules.

• The overall Aftermarket model yields SLA, KPI and
• perating metrics.

12

Monday, 14 June 2010

Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Content

1

Applications of Agents

2

Agent characteristics Are agents new or different? Agents and their environment The intentional perspective

3

Agent architecture

4

Summary

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 5 / 35

Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

What is an Agent?

An intelligent agent is a computer system capable of flexible, autonomous action in some environment: the situated agent. AGENT ENVIRONMENT act sense

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 6 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

What are Multi-Agent Systems?

An agent can be more useful in the context of others:

Can concentrate on tasks within competence Can delegate other tasks Can use ability to communicate, coordinate, negotiate How to organize?

AGENT1 AGENT2 AGENT3 ENVIRONMENT

act sense act sense act sense

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 7 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Agent Characteristics

Major: Reactive: has an on-going interaction with its environment, and responds to changes that occur in it (in time for the response to be useful). Pro-active: means generating and attempting to achieve goals Social: ability to interact with other agents (and possibly humans) via some kind of agent-communication language, and perhaps cooperate with others.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 8 / 35

Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Agent Characteristics

Minor: Mobility: The ability of an agent to move around an electronic network. Veracity: Whether an agent will knowingly communicate false information. Benevolence: Whether agents have conflicting goals, and thus whether they are inherently helpful. Rationality: Whether an agent will act in order to achieve its goals, and will not deliberately act so as to prevent its goals being achieved. Learning/adaption: Whether agents improve performance

• ver time.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 9 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Reactivity

Simple and not-so-simple agents:

thermostat washing machines engine management systems? house management system — “intelligent buildings”?

If environment never changes, success or failure are meaningless — program executes blindly The real world is not like that: change, incompleteness. Many (most?) interesting environments are dynamic Software is hard to build: planning, failures, choice A reactive system interacts continuously with environment, responds to changes – consider a robot sharing an environment with people...

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 10 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Proactivity

Reactive systems are relatively easy: stimulus → response Want agents to do things for us Want goal-directed behaviour — implies AI techniques, e.g. reasoning with rules Pro-activity: generating and achieving goals

Not driven (solely) by events Taking the initiative Recognising opportunities

Need a model of the environment to support the decision-making process:

symbolic — classical AI non-symbolic — neural networks, time series, Markov decision processes etc.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 11 / 35

Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Social Ability

The real world is a multi-agent environment: we cannot go around attempting to achieve goals without taking others into account. Some goals can only be achieved with the cooperation of

• thers. Suggests need for:

Information/models of other agents’ state Trust metrics Reputation models (e.g. FOAF)

Social ability in agents is the ability to interact with other agents (and possibly humans) via some kind of agent-communication language, and perhaps cooperate with others.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 12 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Agents are not Objects

Object:

Encapsulates some state Communicates via message passing Has methods — operations on object state Objects do what they are told

Agents:

Autonomous: decision procedure inside the agent determines whether or not to perform an action on request from another agent Smart: capable of flexible (reactive, pro-active, social) behaviour Active: a multi-agent system is multi-threaded Agents do something because they want to (benevolence) Agents do something for gain (utility)

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 13 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Agents are not Expert Systems

Expert systems typically disembodied ‘expertise’ about some (abstract) domain of discourse. Example: MYCIN knows about blood diseases in humans. Knowledge is stored as rules. A doctor enters facts, answers questions and puts questions to obtain advice. Main differences:

Agents are situated in an environment: MYCIN is not aware

• f the world

Only information obtained is from asking the user questions Agents act: MYCIN does not operate on patients.

Some real-time (typically process control) expert systems are agents. Expert systems are useful components of agents

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 14 / 35

Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Environment Characteristics 1/2

Accessible vs. inaccessible: where the agent can obtain complete, accurate, up-to-date information about the state

• f the environment. Most moderately complex

environments are inaccessible. Accessibility ⇒ simpler to build. Deterministic vs. non-deterministic: where any action has a single guaranteed effect — there is no uncertainty about the state resulting from an action. The physical world is (largely!) non-deterministic. Non-determinism ⇒ more problems. Static vs. dynamic: where only the agent’s action changes the environment. Other processes — outside the agent’s control — operate in a dynamic environment — just like the physical world.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 15 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Environment Characteristics 2/2

Episodic vs. non-episodic: where the agent performance depends on several discrete episodes, but each episode is

• independent. Simplifies development because the agent

chooses an action based only on the current episode — there is no need to consider either the past or the future. Discrete vs. continuous: where there are a fixed, finite number of actions and percepts. For example a chess game is a discrete environment, while driving a taxi is a continuous environment.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 16 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Agents as Intentional Systems 1/3

The philosopher Daniel Dennett coined the term intentional system to describe entities ‘whose behaviour can be predicted by the method of attributing belief, desires and rational acumen’. Dennett identifies different ‘grades’ of intentional system:

‘A first order intentional system has beliefs and desires (etc.) but no beliefs and desires about beliefs and desires... A second order intentional system is more sophisticated; it has beliefs and desires (and no doubt other intentional states) about beliefs and desires (and other intentional states) — both those of others and its own’

Basis for the Belief-Desire-Intention (BDI) model of agency

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 17 / 35

Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Agents as Intentional Systems 2/3

As computer systems become more complex, we need more powerful abstractions and metaphors to explain their

• peration — low level explanations become impractical.

The most important phases in computing are identified by new abstractions:

procedural abstraction abstract data types

• bjects

next: agents? or services? both! http://www.ist-alive.eu

Abstractions help in solving problems because they replace lots of detail with a single concept Agents, and agents as intentional systems, represent an abstraction both for software components and software systems

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 18 / 35 Applications of Agents Agent characteristics Agent architecture Summary Are agents new or different? Agents and their environment The intentional perspective

Agents as Intentional Systems 3/3

Characterising Agents: Provides familiar, non-technical way to understand and explain agents. Good for requirements. Nested Representations: Can specify systems that include

• ther systems. Legacy systems can be embedded

Beyond declarative programming:

Procedural programming states how a system operates — too fragile Declarative programming states what to achieve, declares relationships between objects, lets a built-in control mechanism solve the problem — more robust Agent programming states what to achieve, relies on agent control mechanisms solve problem, respects some built-in theory of agency (e.g. Cohen-Levesque model of intention)

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 19 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Content

1

Applications of Agents

2

Agent characteristics

3

Agent architecture Models of architecture and environment Kinds of agents Goals and actions

4

Summary

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 20 / 35

Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

An Abstract Architecture for Agents

Environment is a finite set E of discrete, instantaneous states: E = {e0, e1, . . .} Agents actions change the environment. Ac = {α0, α1, . . .} An agent acting in an environment generates a run, r: r : e0

α0

− → e1

α1

− → e2

α2

− → e3 . . . eu An agent senses the environment state ei ∈ E and takes action αi ∈ Ac Exercise: E = {light, dark}, Ac = {on, off} What are the runs?

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 21 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Environments

Define R as the set of all runs then RAc ⊂ R that ends with an action, and RE ⊂ R that ends with an environment state A state transformer function τ : RAc → 2E Environments are: history dependent and non-deterministic If τ(r) = ∅, there are no possible successor states to r: the run has ended An environment Env is a triple Env = E, e0, τ where E is set of environment states, e0 ∈ E is the initial state; and τ is the state transformer function.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 22 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Agents

An Agent is a function that maps runs to actions: Ag : RE → Ac Thus an agent makes a decision about what action to perform based on the history of the system witnessed to date. Let AG be the set of all agents.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 23 / 35

Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Systems

A system is a pair of an agent and an environment. A system induces a set of possible runs: R(Ag, Env). Assume R(Ag, Env) contains only runs that have ended.

that is, ∀r ∈ R : τ(r) = ∅

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 24 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Traces

Thus, a sequence (e0, α0, e1, α1, e2, . . .) represents a run of an agent Ag in environment Env = E, e0, τ if

e0 is the initial state of Env α0 = Ag(e0) and for u > 0, eu ∈ τ((e0, α0, . . . , αu−1)) and αu = Ag((e0, α0, . . . , eu))

That is, E is finally in state eu as a result of action αu

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 25 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Purely Reactive Agents

Simple but useful Some agents decide what to do without reference to their history We call such agents purely reactive action : E → Ac A thermostat is a purely reactive agent. action(e) =

• ff if e = temperature OK
• n otherwise

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 26 / 35

Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Perception

Add perception: AGENT see action ENVIRONMENT act sense The see function is the agent’s ability to observe its environment, and the action function represents the agent’s decision making process. Output of the see function is a percept:

see : E → Per maps environment states to percepts action : Per ∗ → Ac maps sequences of percepts to actions

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 27 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Agents with State 1/2

Add memory: AGENT see action next state ENVIRONMENT act sense Let I be the set of all internal states of the agent — the agent remembers the states of the environment.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 28 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Agents with State 2/2

The perception function is unchanged. Redefine action and introduce next

see : E → Per maps environment states to percepts action : I → Ac maps from internal states to actions next : I × Per → I maps an internal state and a percept to an internal state

Agent control loop

1

Agent starts in some initial internal state i0.

2

Observes environment state e, generates a percept see(e).

3

Internal state of the agent is then updated via next function, becoming next(i0, see(e)).

4

The action selected by the agent is action(next(i0, see(e))). This action is then performed.

5

Goto step 2

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 29 / 35

Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Tasks for Agents

We build agents in order to carry out tasks for us. The task must be specified by us... But we want to tell agents what to do without telling them how to do it. Why? Because we would get it wrong. Exercise: program a robot to go from A to B Approach:

Actions are solutions to atomic problems Utility functions measure achievment of goal

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 30 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Utility functions

The Tileworld example (Wooldridge, p.39) illustrates the use of utility functions:

Simulated two dimensional grid environment on which there are agents, tiles, obstacles, and holes. An agent can move in four directions, up, down, left, or right, and if it is located next to a tile, it can push it. Holes have to be filled up with tiles by the agent. An agent scores points by filling holes with tiles, with the aim of filling as many holes as possible. Tileworld changes: holes appear and disappear randomly

• ver time — dynamic environment

Utility function can be defined as: u(r) = number of holes filled in r number of holes that appeared in r

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 31 / 35 Applications of Agents Agent characteristics Agent architecture Summary Models of architecture and environment Kinds of agents Goals and actions

Achievement and Maintenance Tasks

Two most common types of tasks are achievement tasks and maintenance tasks An achievement task is a set G of “good” or “goal” states: G ⊂ E

The agent succeeds if it is guaranteed to bring about at least one of these states There is no preference ordering on the states

A maintenance goal is a set B of “bad” states: B ⊂ E

The agent succeeds in a particular environment if it manages to avoid all states in B That is, it never performs actions which result in any state in B occurring.

Can express these goals in terms of utility functions

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 32 / 35

Applications of Agents Agent characteristics Agent architecture Summary

Content

1

Applications of Agents

2

Agent characteristics

3

Agent architecture

4

Summary

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 33 / 35 Applications of Agents Agent characteristics Agent architecture Summary

Summary

Agent characteristics Is an agent something new? Intentional perspective on software systems Abstract agent architecture: agent + environments Kinds of agents: reactive, perceptive, stateful Utility functions: one way to say what, not how

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 34 / 35 Applications of Agents Agent characteristics Agent architecture Summary

Directed Reading

Wooldridge Chs.1-2. Russell and Norvig Ch.1. Russell, S. and Norvig, P . (2003). Artificial Intelligence: A Modern Approach. Prentice Hall. ISBN: 0-13-080302-2. Wooldridge, M. (2009). An introduction to multiagent systems (second edition). Wiley. ISBN: 978-0-470-51946-2.

De Vos/Padget (Bath/CS) CM30174/Intro October 5, 2010 35 / 35