A B Intelligent Agents Chapter 2 Percepts: location and contents, - PDF document

Vacuum-cleaner world A B Intelligent Agents Chapter 2 Percepts: location and contents, e.g., [ A, Dirty ] Actions: Left , Right , Suck , NoOp Chapter 2 1 Chapter 2 4 Outline A vacuum-cleaner agent ♦ Agents and environments Percept sequence Action [ A, Clean ] ♦ Rationality [ A, Dirty ] ♦ PEAS (Performance measure, Environment, Actuators, Sensors) [ B, Clean ] [ B, Dirty ] ♦ Environment types [ A, Clean ] , [ A, Clean ] [ A, Clean ] , [ A, Dirty ] ♦ Agent types . . . Chapter 2 2 Chapter 2 5 Agents and environments A vacuum-cleaner agent sensors Percept sequence Action percepts [ A, Clean ] Right ? [ A, Dirty ] Suck environment agent [ B, Clean ] Left actions [ B, Dirty ] Suck [ A, Clean ] , [ A, Clean ] Right actuators [ A, Clean ] , [ A, Dirty ] Suck . . . . . . Agents include humans, robots, softbots, thermostats, etc. The agent function maps from percept histories to actions: function Reflex-Vacuum-Agent ([ location , status ]) returns an action f : P ∗ → A if status = Dirty then return Suck else if location = A then return Right The agent program runs on the physical architecture to produce f else if location = B then return Left What is the right function? Can it be implemented in a small agent program? Chapter 2 3 Chapter 2 6

Rationality Rationality Fixed performance measure evaluates the environment sequence Fixed performance measure evaluates the environment sequence – one point per square cleaned up in time T ? – one point per square cleaned up in time T ? – one point per clean square per time step, minus one per move? – penalize for > k dirty squares? A rational agent chooses whichever action maximizes the expected value of the performance measure given the percept sequence to date Rational � = omniscient – percepts may not supply all relevant information Rational � = clairvoyant – action outcomes may not be as expected Hence, rational � = successful Rational ⇒ exploration, learning, autonomy Chapter 2 7 Chapter 2 10 Rationality PEAS Fixed performance measure evaluates the environment sequence To design a rational agent, we must specify the task environment – one point per square cleaned up in time T ? Consider, e.g., the task of designing an automated taxi: – one point per clean square per time step, minus one per move? Performance measure?? Environment?? Actuators?? Sensors?? Chapter 2 8 Chapter 2 11 Rationality PEAS Fixed performance measure evaluates the environment sequence To design a rational agent, we must specify the task environment – one point per square cleaned up in time T ? Consider, e.g., the task of designing an automated taxi: – one point per clean square per time step, minus one per move? – penalize for > k dirty squares? Performance measure?? safety, destination, profits, legality, comfort, . . . Environment?? streets in Lower Mainland, traffic, pedestrians, weather, . . . Actuators?? steering, accelerator, brake, horn, speaker/display, . . . Sensors?? video, accelerometers, gauges, engine sensors, keyboard, GPS, . . . Chapter 2 9 Chapter 2 12

Internet shopping agent Environment types Performance measure?? 8-Puzzle Backgammon Internet shopping Taxi Observable?? Yes Yes No No Environment?? Deterministic?? Actuators?? Sensors?? Chapter 2 13 Chapter 2 16 Internet shopping agent Environment types Performance measure?? price, quality, appropriateness, efficiency 8-Puzzle Backgammon Internet shopping Taxi Observable?? Yes Yes No No Environment?? current and future WWW sites, vendors, shippers Deterministic?? Yes No Partly No Actuators?? display to user, follow URL, fill in form Episodic?? Sensors?? HTML pages (text, graphics, scripts) Chapter 2 14 Chapter 2 17 Environment types Environment types 8-Puzzle Backgammon Internet shopping Taxi 8-Puzzle Backgammon Internet shopping Taxi Observable?? Observable?? Yes Yes No No Deterministic?? Yes No Partly No Episodic?? No No No No Static?? Chapter 2 15 Chapter 2 18

Environment types Agent types Four basic types in order of increasing generality: 8-Puzzle Backgammon Internet shopping Taxi – simple reflex agents Observable?? Yes Yes No No – reflex agents with state Deterministic?? Yes No Partly No – goal-based agents Episodic?? No No No No – utility-based agents Static?? Yes Yes Semi No Discrete?? All these can be turned into learning agents Chapter 2 19 Chapter 2 22 Environment types Simple reflex agents Agent 8-Puzzle Backgammon Internet shopping Taxi Sensors Observable?? Yes Yes No No Deterministic?? Yes No Partly No What the world Episodic?? No No No No is like now Environment Static?? Yes Yes Semi No Discrete?? Yes Yes Yes No Single-agent?? What action I Condition−action rules should do now Actuators Chapter 2 20 Chapter 2 23 Environment types Example 8-Puzzle Backgammon Internet shopping Taxi function Reflex-Vacuum-Agent ([ location , status ]) returns an action Observable?? Yes Yes No No if status = Dirty then return Suck Deterministic?? Yes No Partly No else if location = A then return Right Episodic?? No No No No else if location = B then return Left Static?? Yes Yes Semi No Discrete?? Yes Yes Yes No Single-agent?? Yes No Yes (except auctions) No The environment type largely determines the agent design The real world is (of course) partially observable, stochastic, sequential, dynamic, continuous, multi-agent Chapter 2 21 Chapter 2 24

Reflex agents with state Utility-based agents Sensors Sensors State State What the world What the world How the world evolves How the world evolves is like now is like now Environment Environment What it will be like What my actions do What my actions do if I do action A How happy I will be Utility in such a state What action I What action I Condition−action rules should do now should do now Agent Agent Actuators Actuators Chapter 2 25 Chapter 2 28 Example Learning agents Performance standard function Reflex-Vacuum-Agent ([ location , status ]) returns an action static : last A, last B , numbers, initially ∞ Sensors Critic if status = Dirty then . . . feedback Environment changes Learning Performance element element knowledge learning goals Problem generator Agent Actuators Chapter 2 26 Chapter 2 29 Goal-based agents Summary Agents interact with environments through actuators and sensors Sensors State The agent function describes what the agent does in all circumstances What the world How the world evolves The performance measure evaluates the environment sequence is like now Environment A perfectly rational agent maximizes expected performance What it will be like What my actions do if I do action A Agent programs implement (some) agent functions PEAS descriptions define task environments What action I Environments are categorized along several dimensions: Goals should do now observable? deterministic? episodic? static? discrete? single-agent? Agent Several basic agent architectures exist: Actuators reflex, reflex with state, goal-based, utility-based Chapter 2 27 Chapter 2 30