CS344M Autonomous Multiagent Systems Todd Hester Department of - - PowerPoint PPT Presentation

▶
cs344m autonomous multiagent systems
SMART_READER_LITE
LIVE PREVIEW

CS344M Autonomous Multiagent Systems Todd Hester Department of - - PowerPoint PPT Presentation

Jan 19, 2024 163 likes •651 views

CS344M Autonomous Multiagent Systems Todd Hester Department of Computer Science The University of Texas at Austin Good Afternoon, Colleagues Are there any questions? Todd Hester Good Afternoon, Colleagues Are there any questions? Todd

slide-1
SLIDE 1

CS344M Autonomous Multiagent Systems

Todd Hester Department of Computer Science The University of Texas at Austin

slide-2
SLIDE 2

Good Afternoon, Colleagues

Are there any questions?

Todd Hester

slide-3
SLIDE 3

Good Afternoon, Colleagues

Are there any questions?

Todd Hester

slide-4
SLIDE 4

Logistics

  • Progress reports due in 2 weeks

Todd Hester

slide-5
SLIDE 5

Logistics

  • Progress reports due in 2 weeks
  • Readings for next week

Todd Hester

slide-6
SLIDE 6

Game Theory Premises

  • Simultaneous actions
  • No communication
  • Outcome depends on combination of actions
  • Utility (payoff) encapsulates everything about preferences
  • ver outcomes

Todd Hester

slide-7
SLIDE 7

Solution Concepts

  • Dominant strategy
  • Nash equilibrium
  • Pareto optimality
  • Maximum social welfare
  • Maximin strategy

Todd Hester

slide-8
SLIDE 8

Prisoner’s Dilemma

❈♦❧✉♠♥ ❈✭✶✮ ❉✭✷✮ ❈✭✶✮ ✸✱✸ ✵✱✺ ❘♦✇ ❉✭✷✮ ✺✱✵ ✶✱✶

Todd Hester

slide-9
SLIDE 9

Chicken

❈♦❧✉♠♥ ❈✭✶✮ ❉✭✷✮ ❈✭✶✮ ✸✱✸ ✶✱✺ ❘♦✇ ❉✭✷✮ ✺✱✶ ✵✱✵

Todd Hester

slide-10
SLIDE 10

Bach/Stravinsky

  • My wife and I agree to meet at a concert

Todd Hester

slide-11
SLIDE 11

Bach/Stravinsky

  • My wife and I agree to meet at a concert
  • Unfortunately, there are 2: Bach and Stravinsky

Todd Hester

slide-12
SLIDE 12

Bach/Stravinsky

  • My wife and I agree to meet at a concert
  • Unfortunately, there are 2: Bach and Stravinsky
  • No time to get in touch with each other

Todd Hester

slide-13
SLIDE 13

Bach/Stravinsky

  • My wife and I agree to meet at a concert
  • Unfortunately, there are 2: Bach and Stravinsky
  • No time to get in touch with each other
  • I prefer Stravinsky, she prefers Bach

Todd Hester

slide-14
SLIDE 14

Bach/Stravinsky

  • My wife and I agree to meet at a concert
  • Unfortunately, there are 2: Bach and Stravinsky
  • No time to get in touch with each other
  • I prefer Stravinsky, she prefers Bach
  • But most of all, we want to be together

Todd Hester

slide-15
SLIDE 15

Bach/Stravinsky

  • My wife and I agree to meet at a concert
  • Unfortunately, there are 2: Bach and Stravinsky
  • No time to get in touch with each other
  • I prefer Stravinsky, she prefers Bach
  • But most of all, we want to be together

– If not, so distraught we don’t care what we’re listening to

Todd Hester

slide-16
SLIDE 16

Bach/Stravinsky

  • My wife and I agree to meet at a concert
  • Unfortunately, there are 2: Bach and Stravinsky
  • No time to get in touch with each other
  • I prefer Stravinsky, she prefers Bach
  • But most of all, we want to be together

– If not, so distraught we don’t care what we’re listening to

  • Propose a payoff matrix

Todd Hester

slide-17
SLIDE 17

Bach/Stravinsky

❲✐❢❡ ❙ ❇ ❙ ✷✱✶ ✵✱✵ ▼❡ ❇ ✵✱✵ ✶✱✷

Todd Hester

slide-18
SLIDE 18

Nash Equilibrium

  • Does every game have a pure strategy Nash equilibrium?

Todd Hester

slide-19
SLIDE 19

Matching Pennies

  • We each put a penny down covered
  • If they match, I win, if they don’t, you win

Todd Hester

slide-20
SLIDE 20

Matching Pennies

  • We each put a penny down covered
  • If they match, I win, if they don’t, you win

P❧❛②❡r ✷ ❍ ❚ ❍ ✶✱✲✶ ✲✶✱✶ P❧❛②❡r ✶ ❚ ✲✶✱✶ ✶✱✲✶

Todd Hester

slide-21
SLIDE 21

Matching Pennies

  • We each put a penny down covered
  • If they match, I win, if they don’t, you win

P❧❛②❡r ✷ ❍ ❚ ❍ ✶✱✲✶ ✲✶✱✶ P❧❛②❡r ✶ ❚ ✲✶✱✶ ✶✱✲✶

Nash equilibrium?

Todd Hester

slide-22
SLIDE 22

Nash Equilibrium

  • Every game has at least one Nash equilibrium

Todd Hester

slide-23
SLIDE 23

Nash Equilibrium

  • Every game has at least one Nash equilibrium

– Nobel prize and academy award!

Todd Hester

slide-24
SLIDE 24

Nash Equilibrium

  • Every game has at least one Nash equilibrium

– Nobel prize and academy award!

  • Not known if complexity of finding one is NP-complete or

in P

Todd Hester

slide-25
SLIDE 25

Some theory

  • Prove that if each player plays a dominant strategy, the

result is a Nash equilibrium

Todd Hester

slide-26
SLIDE 26

Some theory

  • Prove that if each player plays a dominant strategy, the

result is a Nash equilibrium

  • Are all Nash equilibria the result of playing dominant

strategies?

Todd Hester

slide-27
SLIDE 27

Some theory

  • Prove that if each player plays a dominant strategy, the

result is a Nash equilibrium

  • Are all Nash equilibria the result of playing dominant

strategies?

  • Is the outcome of a Nash equilibrium necessarily Pareto
  • ptimal?

Todd Hester

slide-28
SLIDE 28

Some theory

  • Prove that if each player plays a dominant strategy, the

result is a Nash equilibrium

  • Are all Nash equilibria the result of playing dominant

strategies?

  • Is the outcome of a Nash equilibrium necessarily Pareto
  • ptimal?
  • Is a Pareto optimal outcome necessarily the result of Nash

equilibrium strategies?

Todd Hester

slide-29
SLIDE 29

Some theory

  • Prove that if each player plays a dominant strategy, the

result is a Nash equilibrium

  • Are all Nash equilibria the result of playing dominant

strategies?

  • Is the outcome of a Nash equilibrium necessarily Pareto
  • ptimal?
  • Is a Pareto optimal outcome necessarily the result of Nash

equilibrium strategies?

  • Is the maximum social welfare outcome necessarily Pareto
  • ptimal?

Todd Hester

slide-30
SLIDE 30

Some theory

  • Prove that if each player plays a dominant strategy, the

result is a Nash equilibrium

  • Are all Nash equilibria the result of playing dominant

strategies?

  • Is the outcome of a Nash equilibrium necessarily Pareto
  • ptimal?
  • Is a Pareto optimal outcome necessarily the result of Nash

equilibrium strategies?

  • Is the maximum social welfare outcome necessarily Pareto
  • ptimal?
  • If both players play maximin, is it necessarily a Nash

equilibrium?

Todd Hester

slide-31
SLIDE 31

Activity

P❧❛②❡r ✷ ❘♦❝❦ P❛♣❡r ❙❝✐ss♦rs ❘♦❝❦ ✵✱✵ ✲✶✱✶ ✶✱✲✶ P❧❛②❡r ✶ P❛♣❡r ✶✱✲✶ ✵✱✵ ✲✶✱✶ ❙❝✐ss♦rs ✲✶✱✶ ✶✱✲✶ ✵✱✵

Todd Hester

slide-32
SLIDE 32

Activity

P❧❛②❡r ✷ ❘♦❝❦ P❛♣❡r ❙❝✐ss♦rs ❘♦❝❦ ✵✱✵ ✲✶✱✶ ✶✱✲✶ P❧❛②❡r ✶ P❛♣❡r ✶✱✲✶ ✵✱✵ ✲✶✱✶ ❙❝✐ss♦rs ✲✶✱✶ ✶✱✲✶ ✵✱✵

Todd Hester

slide-33
SLIDE 33

Mixed strategy equilibrium

P❧❛②❡r ✷ ❆❝t✐♦♥ ✶ ❆❝t✐♦♥ ✷ ❆❝t✐♦♥ ✶ ✹✱✽ ✷✱✵ P❧❛②❡r ✶ ❆❝t✐♦♥ ✷ ✻✱✷ ✵✱✽

Todd Hester

slide-34
SLIDE 34

Mixed strategy equilibrium

P❧❛②❡r ✷ ❆❝t✐♦♥ ✶ ❆❝t✐♦♥ ✷ ❆❝t✐♦♥ ✶ ✹✱✽ ✷✱✵ P❧❛②❡r ✶ ❆❝t✐♦♥ ✷ ✻✱✷ ✵✱✽

  • What if player 2 picks action 1 3/4 of the time?

Todd Hester

slide-35
SLIDE 35

Mixed strategy equilibrium

P❧❛②❡r ✷ ❆❝t✐♦♥ ✶ ❆❝t✐♦♥ ✷ ❆❝t✐♦♥ ✶ ✹✱✽ ✷✱✵ P❧❛②❡r ✶ ❆❝t✐♦♥ ✷ ✻✱✷ ✵✱✽

  • What if player 2 picks action 1 3/4 of the time?
  • What if player 2 picks action 1 1/4 of the time?

Todd Hester

slide-36
SLIDE 36

Mixed strategy equilibrium

P❧❛②❡r ✷ ❆❝t✐♦♥ ✶ ❆❝t✐♦♥ ✷ ❆❝t✐♦♥ ✶ ✹✱✽ ✷✱✵ P❧❛②❡r ✶ ❆❝t✐♦♥ ✷ ✻✱✷ ✵✱✽

  • What if player 2 picks action 1 3/4 of the time?
  • What if player 2 picks action 1 1/4 of the time?
  • Player 1 must be indifferent between actions 1 and 2

Todd Hester

slide-37
SLIDE 37

Mixed strategy equilibrium

P❧❛②❡r ✷ ❆❝t✐♦♥ ✶ ❆❝t✐♦♥ ✷ ❆❝t✐♦♥ ✶ ✹✱✽ ✷✱✵ P❧❛②❡r ✶ ❆❝t✐♦♥ ✷ ✻✱✷ ✵✱✽

  • What if player 2 picks action 1 3/4 of the time?
  • What if player 2 picks action 1 1/4 of the time?
  • Player 1 must be indifferent between actions 1 and 2
  • Player 2 must be indifferent between actions 1 and 2

Todd Hester

slide-38
SLIDE 38

Mixed strategy equilibrium

P❧❛②❡r ✷ ❆❝t✐♦♥ ✶ ❆❝t✐♦♥ ✷ ❆❝t✐♦♥ ✶ ✹✱✽ ✷✱✵ P❧❛②❡r ✶ ❆❝t✐♦♥ ✷ ✻✱✷ ✵✱✽

  • What if player 2 picks action 1 3/4 of the time?
  • What if player 2 picks action 1 1/4 of the time?
  • Player 1 must be indifferent between actions 1 and 2
  • Player 2 must be indifferent between actions 1 and 2

Do actual numbers matter?

Todd Hester

slide-39
SLIDE 39

Rock/Paper/Scissors

  • Nash equilibrium?

Todd Hester

slide-40
SLIDE 40

Rock/Paper/Scissors

  • Nash equilibrium?
  • Why is anything else not an equilibrium?

Todd Hester

slide-41
SLIDE 41

Rock/Paper/Scissors

  • Nash equilibrium?
  • Why is anything else not an equilibrium?
  • Rock Paper Scissors tournament

Todd Hester

slide-42
SLIDE 42

Rock/Paper/Scissors

  • Nash equilibrium?
  • Why is anything else not an equilibrium?
  • Rock Paper Scissors tournament
  • Poker

Todd Hester

slide-43
SLIDE 43

Discussion

  • What is an example game within robot soccer?

Todd Hester

slide-44
SLIDE 44

Discussion

  • What is an example game within robot soccer?
  • ♦❛❧✐❡

❇❧♦❝❦ ❘✐❣❤t ▲❡❢t ▲❡❢t ✶✱✲✶ ✲✶✱✶ ❑✐❝❦❡r ❘✐❣❤t ✲✶✱✶ ✶✱✲✶

Todd Hester

slide-45
SLIDE 45

Discussion

  • What is an example game within robot soccer?
  • ♦❛❧✐❡

❇❧♦❝❦ ❘✐❣❤t ▲❡❢t ▲❡❢t ✶✱✲✶ ✲✶✱✶ ❑✐❝❦❡r ❘✐❣❤t ✲✶✱✶ ✶✱✲✶

  • Can we use game theory to devise better strategies?

Todd Hester

slide-46
SLIDE 46

Correlated Equilibria

Sometimes mixing isn’t enough: Bach/Stravinsky

❲✐❢❡ ❙ ❇ ❙ ✷✱✶ ✵✱✵ ▼❡ ❇ ✵✱✵ ✶✱✷

Todd Hester

slide-47
SLIDE 47

Correlated Equilibria

Sometimes mixing isn’t enough: Bach/Stravinsky

❲✐❢❡ ❙ ❇ ❙ ✷✱✶ ✵✱✵ ▼❡ ❇ ✵✱✵ ✶✱✷

Want only S,S or B,B - 50% each

Todd Hester