angelic hierarchical planning
play

Angelic Hierarchical Planning: Optimal and Online Algorithms - PowerPoint PPT Presentation

Angelic Hierarchical Planning: Optimal and Online Algorithms Bhaskara Marthi Stuart Russell Jason Wolfe MIT/Willow Garage UC Berkeley UC Berkeley bhaskara@csail.mit.edu russell@cs.berkeley.edu jawolfe@cs.berkeley.edu ICAPS 08 1


  1. Deterministic Planning Problems • Here, a planning problem = • State space S • Initial state s 0 , terminal set G • Primitive action set • Transition function: S × A → S • Cost function : S × A → R ∪ { ∞ } S s 0 -5 4 ∞ 8 G ∞ 3 Transitions & costs for action a 1 7

  2. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A T1 T2 T3 T4 8

  3. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 T1 T2 T3 T4 8

  4. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  5. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  6. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  7. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  8. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  9. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  10. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  11. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  12. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  13. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  14. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  15. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  16. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  17. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  18. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  19. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  20. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  21. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  22. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  23. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  24. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  25. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row C A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  26. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B C • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  27. Running Example: Warehouse World HLAs L D GetR U Turn D PutL 9

  28. Running Example: Warehouse World HLAs Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9

  29. Running Example: Warehouse World HLAs NavT( 2,3 ) NavT( 2,3 ) Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9

  30. Running Example: Warehouse World HLAs Move( C,A ) NavT( 2,3 ) NavT( 2,3 ) Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9

  31. Running Example: Warehouse World HLAs Act ... Move( C,A ) ... NavT( 2,3 ) NavT( 2,3 ) ... Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) ... L D GetR U Turn D PutL 9

  32. Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act [Act] 10

  33. Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] 10

  34. Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] ... [NavT( left of B ), GetR, NavT( left of target ), PutR] ... 10

  35. Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] ... [NavT( left of B ), GetR, NavT( left of target ), PutR] ... ... ... 10

  36. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) 11

  37. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans Act 11

  38. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act 11

  39. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act 11

  40. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act 11

  41. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11

  42. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11

  43. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11

  44. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Act Move( A,C ) Act 11

  45. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11

  46. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11

  47. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11

  48. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act ) C , B ( e v o M . . . Act Move( B,C ) Nav( x C -1,y C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11

  49. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) • Nodes have optimistic & pessimistic valuations 6/6 8/8 3/3 Act ) C , B ( e v o M 1/1 . . . 5/7 7/9 Act Move( B,C ) Nav( x C -1,y C ) 2/4 Move( C,A ) Act 4/8 9/ ∞ 0/0 Move( C,B ) Act Act 3/ ∞ 8/ ∞ Move( A,C ) Act 11

  50. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy NavT( 0,1 ) 12

  51. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • NavT( 0,1 ) 6 8 10 ... ... 7 5 12

  52. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • NavT( 0,1 ) 6 � � 6 � � � 5 5 � 12

  53. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • Exact description of h = valuation of h from each s • NavT( 0,1 ) � � � � � � 6 � 6 � 6 � � � � � ... � � 5 � 5 � 5 � 12

  54. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • Exact description of h = valuation of h from each s • But this description has no compact, efficient representation in general • NavT( 0,1 ) � � � � � � 6 � 6 � 6 � � � � � ... � � 5 � 5 � 5 � 12

  55. Optimistic and Pessimistic Valuations • Instead, use approximate valuations � � Exact 6 4 3 1 5 � 13

  56. Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set � � Exact 6 4 3 1 5 � 13

  57. Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set • Optimistic valuations never overestimate best achievable cost 1 Optimistic � � Exact 6 4 3 1 5 � 13

  58. Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set • Optimistic valuations never overestimate best achievable cost • Pessimistic valuations never underestimate best achievable cost 1 / 4 Optimistic � � Exact 6 4 Pessimistic 3 1 5 � 13

  59. Representing Descriptions: NCSTRIPS 14

  60. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA 14

  61. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA NavT( x t ,y t ) (Pre: At( x s ,y s )) 14

  62. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | 14

  63. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 14

  64. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 x ELSE: s t nil 14

  65. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state • An simple algorithm progresses a valuation (DNF + #) through an NCSTRIPS description to produce next valuation NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 x ELSE: s t nil 14

  66. Angelic Hierarchical A* (AHA*) • Construct an ALT with the single plan [Act] • Loop • Select a plan with minimal optimistic cost to G • If primitive, return it • Otherwise, refine one of its HLAs • Prune dominated refinements 15

  67. AHA*: Intuitive Picture G s 0 Act highest-level primitive 16

  68. AHA*: Intuitive Picture G s 0 Act highest-level primitive 17

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend