CS344M Autonomous Multiagent Systems Todd Hester Department or - PowerPoint PPT Presentation

CS344M Autonomous Multiagent Systems Todd Hester Department or Computer Science The University of Texas at Austin

Good Afternoon, Colleagues Todd Hester

Good Afternoon, Colleagues Are there any questions? Todd Hester

Logistics • Questions about the syllabus? Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist • Problems with the assignment? Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist • Problems with the assignment? • Piazza vs. mailing list Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist • Problems with the assignment? • Piazza vs. mailing list − CC Elad, Patrick, and me on everything Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist • Problems with the assignment? • Piazza vs. mailing list − CC Elad, Patrick, and me on everything • Last week’s slides are up Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist • Problems with the assignment? • Piazza vs. mailing list − CC Elad, Patrick, and me on everything • Last week’s slides are up • Next week’s readings are up: − Brooks’ reactive robots − A more deliberative architecture − RoboCup challenge paper Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist • Problems with the assignment? • Piazza vs. mailing list − CC Elad, Patrick, and me on everything • Last week’s slides are up • Next week’s readings are up: − Brooks’ reactive robots − A more deliberative architecture − RoboCup challenge paper • Overlap with Intro to AI Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist • Problems with the assignment? • Piazza vs. mailing list − CC Elad, Patrick, and me on everything • Last week’s slides are up • Next week’s readings are up: − Brooks’ reactive robots − A more deliberative architecture − RoboCup challenge paper • Overlap with Intro to AI • C/C++ issues Todd Hester

Logistics • Questions about the syllabus? • Class registration and waitlist • Problems with the assignment? • Piazza vs. mailing list − CC Elad, Patrick, and me on everything • Last week’s slides are up • Next week’s readings are up: − Brooks’ reactive robots − A more deliberative architecture − RoboCup challenge paper • Overlap with Intro to AI • C/C++ issues Todd Hester

Words without (accepted) definitions • Intelligence • Agent Todd Hester

Words without (accepted) definitions • Intelligence • Agent All proposed definitions include too much or leave gaps. Todd Hester

Words without (accepted) definitions • Intelligence • Agent All proposed definitions include too much or leave gaps. But there are examples. . . Todd Hester

Thermostats • Are they agents or not? • How does Wooldridge resolve this? Todd Hester

Intelligent (autonomous) Agents • Autonomous robot Todd Hester

Intelligent (autonomous) Agents • Autonomous robot • Information gathering agent − Find me the cheapest? Todd Hester

Intelligent (autonomous) Agents • Autonomous robot • Information gathering agent − Find me the cheapest? • E-commerce agents − Decides what to buy/sell and does it Todd Hester

Intelligent (autonomous) Agents • Autonomous robot • Information gathering agent − Find me the cheapest? • E-commerce agents − Decides what to buy/sell and does it • Air-traffic controller Todd Hester

Intelligent (autonomous) Agents • Autonomous robot • Information gathering agent − Find me the cheapest? • E-commerce agents − Decides what to buy/sell and does it • Air-traffic controller • Meeting scheduler Todd Hester

Intelligent (autonomous) Agents • Autonomous robot • Information gathering agent − Find me the cheapest? • E-commerce agents − Decides what to buy/sell and does it • Air-traffic controller • Meeting scheduler • Computer-game-playing agent Todd Hester

Not Intelligent Agents • Thermostat • Telephone • Answering machine • Pencil • Java object Todd Hester

Your Agent Examples Todd Hester

Your Agent Examples • Automotive: Stop light, Autonomous Car • Physical Control: Roomba, Automatic sliding door • Software: antivirus software, Google Now, Laptop battery management, Macbook light intensity controller, Parasolid • Game/entertainment: StarCraft SCV, Counterstrike • Service: Stock trading agent Todd Hester

An Example Todd Hester

An Example • You, as a class, act as a learning agent Todd Hester

An Example • You, as a class, act as a learning agent • Actions : Wave, Stand, Clap Todd Hester

An Example • You, as a class, act as a learning agent • Actions : Wave, Stand, Clap • Observations : colors, reward Todd Hester

An Example • You, as a class, act as a learning agent • Actions : Wave, Stand, Clap • Observations : colors, reward • Goal : Find an optimal policy Todd Hester

An Example • You, as a class, act as a learning agent • Actions : Wave, Stand, Clap • Observations : colors, reward • Goal : Find an optimal policy − Way of selecting actions that gets you the most reward Todd Hester

How did you do it? Todd Hester

How did you do it? • What is your policy? • What does the world look like? Todd Hester

Formalizing My Example Knowns: Todd Hester

Formalizing My Example Knowns: • O = { Blue , Red , Green , Black , . . . } • Rewards in IR • A = { Wave, Clap, Stand } o 0 , a 0 , r 0 , o 1 , a 1 , r 1 , o 2 , . . . Todd Hester

Formalizing My Example Knowns: • O = { Blue , Red , Green , Black , . . . } • Rewards in IR • A = { Wave, Clap, Stand } o 0 , a 0 , r 0 , o 1 , a 1 , r 1 , o 2 , . . . Unknowns: Todd Hester

Formalizing My Example Knowns: • O = { Blue , Red , Green , Black , . . . } • Rewards in IR • A = { Wave, Clap, Stand } o 0 , a 0 , r 0 , o 1 , a 1 , r 1 , o 2 , . . . Unknowns: • S = 4x3 grid • R : S × A �→ IR • P = S �→ O • T : S × A �→ S Todd Hester

Formalizing My Example Knowns: • O = { Blue , Red , Green , Black , . . . } • Rewards in IR • A = { Wave, Clap, Stand } o 0 , a 0 , r 0 , o 1 , a 1 , r 1 , o 2 , . . . Unknowns: • S = 4x3 grid • R : S × A �→ IR • P = S �→ O • T : S × A �→ S o i = P ( s i ) Todd Hester

Formalizing My Example Knowns: • O = { Blue , Red , Green , Black , . . . } • Rewards in IR • A = { Wave, Clap, Stand } o 0 , a 0 , r 0 , o 1 , a 1 , r 1 , o 2 , . . . Unknowns: • S = 4x3 grid • R : S × A �→ IR • P = S �→ O • T : S × A �→ S o i = P ( s i ) r i = R ( s i , a i ) Todd Hester

Formalizing My Example Knowns: • O = { Blue , Red , Green , Black , . . . } • Rewards in IR • A = { Wave, Clap, Stand } o 0 , a 0 , r 0 , o 1 , a 1 , r 1 , o 2 , . . . Unknowns: • S = 4x3 grid • R : S × A �→ IR • P = S �→ O • T : S × A �→ S o i = P ( s i ) r i = R ( s i , a i ) s i +1 = T ( s i , a i ) Todd Hester