SLIDE 1
Good Afternoon, Colleagues
Are there any questions?
Patrick MacAlpine
CS344M Autonomous Multiagent Systems Patrick MacAlpine Department - - PowerPoint PPT Presentation
CS344M Autonomous Multiagent Systems Patrick MacAlpine Department - - PowerPoint PPT Presentation
CS344M Autonomous Multiagent Systems Patrick MacAlpine Department of Computer Science The University of Texas at Austin Good Afternoon, Colleagues Are there any questions? Patrick MacAlpine Good Afternoon, Colleagues Are there any
SLIDE 2
SLIDE 3
Good Afternoon, Colleagues
Are there any questions?
- How is SMDP different from MDP?
- Advantages of tile coding vs other approaches?
- What about SPAR (Strategic Position by Attraction and
Repulsion)?
Patrick MacAlpine
SLIDE 4
Logistics
- Progress reports due at beginning of class today
Patrick MacAlpine
SLIDE 5
Logistics
- Progress reports due at beginning of class today
- Progress report peer review due next Thursday – reports to
review will be sent out shortly
Patrick MacAlpine
SLIDE 6
Logistics
- Progress reports due at beginning of class today
- Progress report peer review due next Thursday – reports to
review will be sent out shortly
- Prize for winning class tournament
Patrick MacAlpine
SLIDE 7
Logistics
- Progress reports due at beginning of class today
- Progress report peer review due next Thursday – reports to
review will be sent out shortly
- Prize for winning class tournament
- 10+ students went to Undergraduate Writing Center :)
Patrick MacAlpine
SLIDE 8
Logistics
- Progress reports due at beginning of class today
- Progress report peer review due next Thursday – reports to
review will be sent out shortly
- Prize for winning class tournament
- 10+ students went to Undergraduate Writing Center :)
Patrick MacAlpine
SLIDE 9
Reinforcement Learning
Image from wikipedia Patrick MacAlpine
SLIDE 10
Reinforcement Learning
Image from wikipedia
Markov Decision Process (MDP)
Patrick MacAlpine
SLIDE 11
Reinforcement Learning
Image from wikipedia
Markov Decision Process (MDP) Important questions:
- What is your state space?
Patrick MacAlpine
SLIDE 12
Reinforcement Learning
Image from wikipedia
Markov Decision Process (MDP) Important questions:
- What is your state space?
- What is your action space?
Patrick MacAlpine
SLIDE 13
Reinforcement Learning
Image from wikipedia
Markov Decision Process (MDP) Important questions:
- What is your state space?
- What is your action space?
- What is your reward function?
Patrick MacAlpine
SLIDE 14
SARSA (st,at,rt,st+1,at+1)
Image from wikipedia Patrick MacAlpine
SLIDE 15
SARSA (st,at,rt,st+1,at+1)
Image from wikipedia
Learn Q table (value function) for state - action pairs Q(st, at) ← Q(st, at) + α[rt+1 + γQ(st+1, at+1) − Q(st, at)]
Patrick MacAlpine
SLIDE 16
Keepaway
- Keepaway videos
Patrick MacAlpine
SLIDE 17
Keepaway
- Keepaway videos
- Slides
Patrick MacAlpine
SLIDE 18
Keepaway Discussion
- Could you use learned policies for full soccer game?
Patrick MacAlpine
SLIDE 19
Keepaway Discussion
- Could you use learned policies for full soccer game?
- Could we apply competitve co-evolution?
Patrick MacAlpine
SLIDE 20
Keepaway Discussion
- Could you use learned policies for full soccer game?
- Could we apply competitve co-evolution?
- Other sub-tasks in soccer that might be learnable?
Patrick MacAlpine
SLIDE 21
Keepaway Discussion
- Could you use learned policies for full soccer game?
- Could we apply competitve co-evolution?
- Other sub-tasks in soccer that might be learnable?
Patrick MacAlpine
SLIDE 22
Half Field Offense
<Slides>
Patrick MacAlpine
SLIDE 23
Policy Search vs Value Function Based RL
Policy Search Value Function Based Learn Policy parameters Value function Good For Tuning parameter values Learning discrete actions Evaluation Fitness function Reward function Algorithms CMA-ES, genetic algorithms, etc. SARSA, Q-learning, etc.
Patrick MacAlpine