Action Robust Reinforcement Learning and Applications in Continuous - - PowerPoint PPT Presentation

▶

Oct 20, 2023 414 likes •507 views

Action Robust Reinforcement Learning and Applications in Continuous Control Chen Tessler *, Yonathan Efroni* and Shie Mannor *equal contribution Poster #272 Action Robust Reinforcement Learning and Applications in Continuous Control Robust MDPs

SLIDE 1

Action Robust Reinforcement Learning and Applications in Continuous Control

Chen Tessler*, Yonathan Efroni* and Shie Mannor

*equal contribution

Poster #272

SLIDE 2

Robust MDPs

Important model, yet not feasible in practical applications.

Action Robust Reinforcement Learning and Applications in Continuous Control

SLIDE 3

Action Robustness in Robotics

Abrupt disturbances Model uncertainty

Action Robust Reinforcement Learning and Applications in Continuous Control

SLIDE 4

Action Robust MDPs

AR-MDPs are a special case of RMDPs, which consider uncertainty in the performed action.

Action Robust Reinforcement Learning and Applications in Continuous Control

SLIDE 5

Algorithm

Theorem 1. This procedure converges to the Nash equilibrium.

Action Robust Reinforcement Learning and Applications in Continuous Control

Update adversary towards the 1-step greedy policy Find optimal actor policy Evaluate joint policy

SLIDE 6

Results

Action Robust Reinforcement Learning and Applications in Continuous Control

Ours(𝛽=1) Baseline

SLIDE 7

Conclusions

Robustness enables coping with uncertainty and transfer

to unseen domains

A gradient based approach for robust reinforcement

learning with convergence guarantees

Does not require explicit definition of the uncertainty set
Application to Deep RL

Action Robust Reinforcement Learning and Applications in Continuous Control

SLIDE 8

Come visit @ Poster #272

Action Robust Reinforcement Learning and Applications in Continuous Control