2 3 Markov Decision Process r k+1 s k+1 Environment Environment - - PowerPoint PPT Presentation

▶
2 3 markov decision process
SMART_READER_LITE
LIVE PREVIEW

2 3 Markov Decision Process r k+1 s k+1 Environment Environment - - PowerPoint PPT Presentation

Dec 06, 2023 307 likes •629 views

2 3 Markov Decision Process r k+1 s k+1 Environment Environment Action a k State s k Reward r k Agent 4 5 6 7 8 9 r k+1 s k+1 Environment Action a k Reward r k Critic Value Function State s k TD Error Policy Actor Agent 10 11 12

slide-1
SLIDE 1
slide-2
SLIDE 2

2

slide-3
SLIDE 3

3

slide-4
SLIDE 4

4

Environment

State sk Action ak

Agent

Reward rk

rk+1 sk+1 Markov Decision Process Environment

slide-5
SLIDE 5

5

slide-6
SLIDE 6

6

slide-7
SLIDE 7

7

slide-8
SLIDE 8

8

slide-9
SLIDE 9

9

slide-10
SLIDE 10

10

State sk Action ak

Actor

Reward rk rk+1 sk+1

Critic

Policy Value Function

TD Error

Environment Agent

slide-11
SLIDE 11

11

slide-12
SLIDE 12

12

slide-13
SLIDE 13

13

slide-14
SLIDE 14

14

slide-15
SLIDE 15

15

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008.

slide-16
SLIDE 16

16

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008.

slide-17
SLIDE 17

17

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008.

slide-18
SLIDE 18

18

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008.

Temporal- difference RL Game Theory Direct Policy Search

slide-19
SLIDE 19

19

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008. Task Type -> Agent Awareness Cooperative Competitive Mixed Independent Coordination-free Opponent- independent Agent-independent Tracking Coordination-based

  • Agent-tracking

Aware Indirect coordination Opponent-aware Agent-aware

slide-20
SLIDE 20

20

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008.

slide-21
SLIDE 21

21

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008. Q L2 S2 R2 L1 10

  • 5

S1

  • 5
  • 10
  • 5

R1

  • 10
  • 5

10

1 2

Obstacle L1 S1 R1 L2 S2 R2

slide-22
SLIDE 22

22

  • C. Guestrin, M. Lagoudakis, and R. Parr, “Coordinated reinforcement learning,” in Proc.

Int’l Conf. Machine Learning (ICML-02), Jul. 2002.

2 3

Q1 Q2 Q3 Q4

1 4

f4

slide-23
SLIDE 23

23

  • C. Guestrin, M. Lagoudakis, and R. Parr, “Coordinated reinforcement learning,” in Proc.

Int’l Conf. Machine Learning (ICML-02), Jul. 2002.

2 3

Q1 Q2 Q3 Q4

1 4

f4

slide-24
SLIDE 24

24

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008.

slide-25
SLIDE 25

25

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008. Q1 L2 R2 L1 1 R1

  • 10

10

1 2

L1 R1 L2 R2

Q2 L2 R2 L1

  • 1

R1 10

  • 10
slide-26
SLIDE 26

26

  • L. M. Littman, “Markov games as a framework for multi-agent reinforcement learning,”

in Proc. Int’l Conf. Machine Learning (ICML-94), Jul. 1994.

slide-27
SLIDE 27

27

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008.

slide-28
SLIDE 28

28

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008. Q1 L2 R2 L1 3 R1 2 Q2 L2 R2 L1 2 R1 3

1 2

L1 R1 L2 R2 Right Room Left Room

slide-29
SLIDE 29

29

  • L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent

reinforcement learning,” IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Reviews, vol. 38, no.2, Mar. 2008.

slide-30
SLIDE 30

30

slide-31
SLIDE 31

31