Distributed Planning for Large Teams Prasanna Velagapudi Thesis - PowerPoint PPT Presentation

Distributed Planning for Large Teams Prasanna Velagapudi Thesis Committee: Katia Sycara (co-chair) Paul Scerri (co-chair) J. Andrew Bagnell Edmund H. Durfee Distributed Planning for Large Teams

Outline • Motivation • Background • Approach – SI-Dec-POMDP – DIMS • Preliminary Work – DPP – D-TREMOR • Proposed Work • Conclusion Distributed Planning for Large Teams 2

Motivation • 100s to 1000s of robots, agents, people • Complex, collaborative tasks • Dynamic, uncertain environment • Offline planning Distributed Planning for Large Teams 3

Motivation • Scaling planning to large teams is hard – Need to plan (with uncertainty) for each agent in team – Agents must consider the actions of a growing number of teammates – Full, joint problem has NEXP complexity [Bernstein 2002] • Optimality is going to be infeasible • Find and exploit structure in the problem • Make good plans in reasonable amount of time Distributed Planning for Large Teams 4

Motivation • Exploit three characteristics of these domains 1. Explicit Interactions • Specific combinations of states and actions where effects depend on more than one agent 2. Sparsity of Interactions • Many potential interactions could occur between agents • Only a few will occur in any given solution 3. Distributed Computation • Each agent has access to local computation • A centralized algorithm has access to 1 unit of computation • A distributed algorithm has access to N units of computation Distributed Planning for Large Teams 5

Example: Interactions Rescue robot Debris Victim Cleaner robot Distributed Planning for Large Teams 6

Example: Sparsity Distributed Planning for Large Teams 7

Related Work Scalability Generality Distributed Planning for Large Teams 8

Related Work Structured Dec-(PO)MDP planners Scalability – JESP [Nair 2003] – TD-Dec-POMDP [Witwicki 2010] – EDI-CR [Mostafa 2009] – SPIDER [Marecki 2009] • Restrict generality Generality slightly to get scalability • High optimality Distributed Planning for Large Teams 9

Related Work Heuristic Dec-(PO)MDP planners Scalability – TREMOR [Varakantham 2009] – OC-Dec-MDP [Beynier 2005] • Sacrifice optimality for scalability • High generality Generality Distributed Planning for Large Teams 10

Related Work Structured multiagent path planners Scalability – DPC [Bhattacharya 2010] – Optimal Decoupling [Van den Berg 2009] • Sacrifice generality further to get scalability • High optimality Generality Distributed Planning for Large Teams 11

Related Work Heuristic multiagent path planners Scalability – Dynamic Networks [Clark 2003] – Prioritized Planning [Van den Berg 2005] • Sacrifice optimality to get scalability • Poor generality Generality Distributed Planning for Large Teams 12

Related Work Our approach: Scalability • Fix high scalability and generality • Explore what level of optimality is possible Generality Distributed Planning for Large Teams 13

Distributed, Iterative Planning • Reduce the full joint problem • Inspiration: into a set of smaller, – TREMOR independent sub-problems [Varankantham 2009] – JESP • Solve independent sub- problems with local algorithm [Nair 2003] • Modify sub-problems to push locally optimal solutions towards high-quality joint solution Distributed Planning for Large Teams 14

Thesis Statement Agents in a large team with known sparse interactions can find computationally efficient high-quality solutions to planning problems through an iterative process of estimating the actions of teammates, locally planning based on these by exchanging estimates, and refining their estimates coordination messages . Distributed Planning for Large Teams 15

Outline • Motivation • Background • Approach Problem Formulation – SI-Dec-POMDP – DIMS Proposed Algorithm • Preliminary Work – DPP – D-TREMOR • Proposed Work • Conclusion Distributed Planning for Large Teams 16

Problem Formulation Sparse-Interaction Dec-POMDP Dec-POMDP POMDP Distributed Planning for Large Teams 17

Review: POMDP : Set of States : Set of Actions : Set of Observations : Transition function : Reward function : Observation function Distributed Planning for Large Teams 18

Review: Dec-POMDP : Joint Transition : Joint Reward : Joint Observation Distributed Planning for Large Teams 19

Dec-POMDP  SI-Dec-POMDP Distributed Planning for Large Teams 20

Sparse Interaction Dec-POMDP : : : Distributed Planning for Large Teams 21

Proposed Approach: DIMS D istributed I terative M odel S haping Task Allocation • Reduce the full joint problem into a set of smaller, independent sub-problems (one for each agent) Local Planning • Solve independent sub-problems with existing state-of-the-art algorithms Interaction • Modify sub-problems to such that local Exchange optimum solution corresponds to high- quality joint solution Model Shaping Distributed Planning for Large Teams 22

Proposed Approach: DIMS D istributed I terative M odel S haping • Assign tasks to agents Task Allocation • Reduce search space considered by agent • Define local sub-problem for each robot Local Planning Interaction Exchange Model Shaping Distributed Planning for Large Teams 23

Proposed Approach: DIMS D istributed I terative M odel S haping • Assign tasks to agents Task Allocation • Reduce search space considered by agent • Define local sub-problem for each robot Full SI-Dec-POMDP Local Planning Interaction Exchange Model Shaping Local (Independent) POMDP Distributed Planning for Large Teams 24

Proposed Approach: DIMS D istributed I terative M odel S haping • Solve local sub-problems using off-the- Task Allocation shelf centralized solver • Result: Locally-optimal policy Local Planning Interaction Exchange Model Shaping Distributed Planning for Large Teams 25

Proposed Approach: DIMS D istributed I terative M odel S haping • Given local policy: estimate local Task Allocation probability and value of interactions • Communicate local probability and value of relevant interactions to team members • Sparsity  Relatively small # of messages Local Planning Interaction Exchange Model Shaping Distributed Planning for Large Teams 26

Proposed Approach: DIMS D istributed I terative M odel S haping • Modify local sub-problems to account for Task Allocation presence of interactions Local Planning Interaction Exchange Model Shaping Distributed Planning for Large Teams 27

Proposed Approach: DIMS D istributed I terative M odel S haping • Reallocate tasks or re-plan using modified Task Allocation local sub-problem Local Planning Interaction Exchange Model Shaping Distributed Planning for Large Teams 28

Proposed Approach: DIMS D istributed I terative M odel S haping Any decentralized allocation Task Allocation mechanism (e.g. auctions) Stock graph, MDP, POMDP Local Planning solver Interaction Lightweight local evaluation Exchange and low-bandwidth messaging Methods to alter local problem Model Shaping to incorporate non-local effects Distributed Planning for Large Teams 29

Outline • Motivation • Background • Approach – SI-Dec-POMDP – DIMS • Preliminary Work – DPP – D-TREMOR • Proposed Work • Conclusion Distributed Planning for Large Teams 30

Preliminary Results Distributed Team REshaping of Distributed Prioritized Planning MOdels for Rapid execution (DPP) (D-TREMOR) 18 16 14 12 10 8 6 4 2 5 10 15 P. Velagapudi, P. Varakantham, P. Velagapudi, K. Sycara, and P. Scerri, K. Sycara, and P. Scerri, “Decentralized prioritized planning in large “Distributed Model Shaping for Scaling to multirobot teams,” Decentralized POMDPs with hundreds of agents,” IROS 2010. AAMAS 2011 (in submission). Distributed Planning for Large Teams 31

Preliminary Results Distributed Team REshaping of Distributed Prioritized Planning MOdels for Rapid execution (DPP) (D-TREMOR) 18 16 14 12 10 8 6 4 2 5 10 15 • No uncertainty • Action/Observation uncertainty • Many potential interactions • Fewer potential interactions • Simple interactions • Complex interactions Distributed Planning for Large Teams 32

Multiagent Path Planning Start 18 16 14 12 10 8 6 4 2 Goal 5 10 15 Distributed Planning for Large Teams 33

Multiagent Path Planning  SI-Dec-POMDP 18 16 • Only one interaction: collision 14 12 10 • Many potential collisions 8 6 • Few collisions in any solution 4 2 5 10 15 Distributed Planning for Large Teams 34

DIMS: Distributed Prioritized Planning Task Allocation (Given) Local Planning A* Interaction Path messages Exchange Prioritized configuration-time Model Shaping obstacles Distributed Planning for Large Teams 35

Prioritized Planning [van den Berg, et al 2005] • Assign priorities to agents based on path length • Longer path length estimate  higher priority [van den Berg, et al 2005] Distributed Planning for Large Teams 36

Prioritized Planning [van den Berg, et al 2005] • Sequentially plan from highest to lowest priority – Takes n steps for n agents • Use previous agents as dynamic obstacles Distributed Planning for Large Teams 37