Pattern Database Heuristics for Fully Observable Nondeterministic - - PowerPoint PPT Presentation

▶
pattern database heuristics for fully observable
SMART_READER_LITE
LIVE PREVIEW

Pattern Database Heuristics for Fully Observable Nondeterministic - - PowerPoint PPT Presentation

Jan 24, 2023 1.13k likes •1.54k views

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments Pattern Database Heuristics for Fully Observable Nondeterministic Planning Robert Mattmller 1 , Manuela Ortlieb 1 , Malte Helmert 1 , and Pascal Bercher 2 1

slide-1
SLIDE 1

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Pattern Database Heuristics for Fully Observable Nondeterministic Planning

Robert Mattmüller1, Manuela Ortlieb1, Malte Helmert1, and Pascal Bercher2

1University of Freiburg 2Ulm University

May 14, 2010

ICAPS 2010 Toronto

PDB Heuristics for Nondeterministic Planning

slide-2
SLIDE 2

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Motivation

Successful techniques for classical planning:

◮ Heuristic search, ◮ Various heuristics: abstraction, delete-relaxation, . . .

Classical planning too restricted for many applications.

⇒ Extend applicability of techniques to more expressive models.

PDB Heuristics for Nondeterministic Planning

slide-3
SLIDE 3

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Problem

◮ Problem: nondeterministic planning ◮ Environment: fully observable, static, discrete ◮ Solutions: strong cyclic plans ◮ Solution Technique: progression search with PDB heuristic ◮ Example: blocksworld with slippery gripper

(blocks can fall down) B A C D init

  • B C D

A goal

PDB Heuristics for Nondeterministic Planning

slide-4
SLIDE 4

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example

B A C D

PDB Heuristics for Nondeterministic Planning

slide-5
SLIDE 5

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example

B A C D B C D A B C D A

(pick-up A B)

PDB Heuristics for Nondeterministic Planning

slide-6
SLIDE 6

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example

B A C D B C D A B C D A

(pick-up A B)

B C D A

(put-on A C)

PDB Heuristics for Nondeterministic Planning

slide-7
SLIDE 7

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example

B A C D B C D A B C D A

(pick-up A B)

B C D A

(put-on A C) (pick-up-from-table A)

PDB Heuristics for Nondeterministic Planning

slide-8
SLIDE 8

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Strong Cyclic Planning

Question: How to compute a strong cyclic plan? Answer: Possible approaches are . . .

◮ Symbolic regression search [Cimatti et al. 2003,

Kissmann and Edelkamp 2009],

◮ Advantage: good data structure (BDDs) ◮ Disadvantage: uninformed

◮ Iteratively apply classical planner [Kuter et al. 2008], or

◮ Advantage: informed ◮ Disadvantage: detour via classical planning

◮ Informed explicit-state progression search.

◮ Advantage: informed, no classical planner needed ◮ Disadvantage: explicit state representation

PDB Heuristics for Nondeterministic Planning

slide-9
SLIDE 9

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001] ◮ start with initial node ◮ while initial node unsolved:

◮ trace most promising partial solution ◮ expand unexpanded nongoal node(s) ◮ initialize heuristics for new nodes ◮ update heuristics of ancestors ◮ run solve-labeling procedure

◮ return solution graph

PDB Heuristics for Nondeterministic Planning

slide-10
SLIDE 10

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001] ◮ start with initial node ◮ while initial node unsolved:

◮ trace most promising partial solution ◮ expand unexpanded nongoal node(s) ◮ initialize heuristics for new nodes ◮ update heuristics of ancestors ◮ run solve-labeling procedure

◮ return solution graph

PDB Heuristics for Nondeterministic Planning

slide-11
SLIDE 11

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001] ◮ start with initial node ◮ while initial node unsolved:

◮ trace most promising partial solution ◮ expand unexpanded nongoal node(s) ◮ initialize heuristics for new nodes ◮ update heuristics of ancestors ◮ run solve-labeling procedure

◮ return solution graph

2.25 1.5 1 1 8 7

PDB Heuristics for Nondeterministic Planning

slide-12
SLIDE 12

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001] ◮ start with initial node ◮ while initial node unsolved:

◮ trace most promising partial solution ◮ expand unexpanded nongoal node(s) ◮ initialize heuristics for new nodes ◮ update heuristics of ancestors ◮ run solve-labeling procedure

◮ return solution graph

2.25 1.5 1 1 8 7

PDB Heuristics for Nondeterministic Planning

slide-13
SLIDE 13

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001] ◮ start with initial node ◮ while initial node unsolved:

◮ trace most promising partial solution ◮ expand unexpanded nongoal node(s) ◮ initialize heuristics for new nodes ◮ update heuristics of ancestors ◮ run solve-labeling procedure

◮ return solution graph

2.25 1.5 1 1 8 7

PDB Heuristics for Nondeterministic Planning

slide-14
SLIDE 14

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001] ◮ start with initial node ◮ while initial node unsolved:

◮ trace most promising partial solution ◮ expand unexpanded nongoal node(s) ◮ initialize heuristics for new nodes ◮ update heuristics of ancestors ◮ run solve-labeling procedure

◮ return solution graph

2.25 1.5 1 1 8 7 1 2 1

PDB Heuristics for Nondeterministic Planning

slide-15
SLIDE 15

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001] ◮ start with initial node ◮ while initial node unsolved:

◮ trace most promising partial solution ◮ expand unexpanded nongoal node(s) ◮ initialize heuristics for new nodes ◮ update heuristics of ancestors ◮ run solve-labeling procedure

◮ return solution graph

2.5 1.5 1.5 1 8 7 1 2 1

PDB Heuristics for Nondeterministic Planning

slide-16
SLIDE 16

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001] ◮ start with initial node ◮ while initial node unsolved:

◮ trace most promising partial solution ◮ expand unexpanded nongoal node(s) ◮ initialize heuristics for new nodes ◮ update heuristics of ancestors ◮ run solve-labeling procedure

◮ return solution graph

2.5 1.5 1.5 1 8 7 1 2 1

PDB Heuristics for Nondeterministic Planning

slide-17
SLIDE 17

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Computing Strong Cyclic Plans

Variant of LAO* search [Hansen and Zilberstein 2001]

Details (for cyclic graphs and solutions):

◮ Solve labeling?

◮ Nested fixpoint iteration.

◮ Updating heuristic estimates?

◮ Value iteration (use discounting to ensure termination).

◮ Initializing heuristic estimates?

◮ PDB heuristic. Following slides.

PDB Heuristics for Nondeterministic Planning

slide-18
SLIDE 18

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Pattern Database Heuristics

[Culberson and Schaeffer 1998, Edelkamp 2001]

Basic Idea:

◮ Create abstract problem by ignoring some state variables. ◮ Use abstract costs as heuristic in original problem. ◮ Precompute abstract costs and store them in PDB.

Additive Pattern Databases:

◮ Compute several abstractions. ◮ Use sum of abstract costs as heuristic.

PDB Heuristics for Nondeterministic Planning

slide-19
SLIDE 19

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

A B

PDB Heuristics for Nondeterministic Planning

slide-20
SLIDE 20

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

Abstraction to pattern

  • pos( A )
  • . No-ops ignored.

A A B A

(pick-up-from-table A) (put-on A B) (put-on-table A)

PDB Heuristics for Nondeterministic Planning

slide-21
SLIDE 21

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

Cost: expected number of steps to goal (equal outcome probabilities). A A B A

(pick-up-from-table A) (put-on A B) (put-on-table A)

h(n1) = 4 h(n⋆) = 0 h(n0) = 6

PDB Heuristics for Nondeterministic Planning

slide-22
SLIDE 22

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

Current explicit graph: A B A B A B

PDB Heuristics for Nondeterministic Planning

slide-23
SLIDE 23

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

Current explicit graph: A B A B A B A

h = 4

abstraction to

{ pos(A) }

PDB Heuristics for Nondeterministic Planning

slide-24
SLIDE 24

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

Current explicit graph: A B A B A B A

h = 4

abstraction to

{ pos(A) }

(pick-up A): f = 6

PDB Heuristics for Nondeterministic Planning

slide-25
SLIDE 25

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

Current explicit graph: A B A B A B A

h = 4

abstraction to

{ pos(A) }

(pick-up A): f = 6 A

h = 6

abstraction to

{ pos(A) }

PDB Heuristics for Nondeterministic Planning

slide-26
SLIDE 26

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

Current explicit graph: A B A B A B A

h = 4

abstraction to

{ pos(A) }

(pick-up A): f = 6 A

h = 6

abstraction to

{ pos(A) }

(pick-up B): f = 8

PDB Heuristics for Nondeterministic Planning

slide-27
SLIDE 27

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Example: Stack A on B

A B A B A B (pick-up A): f = 6 (pick-up B): f = 8

PDB Heuristics for Nondeterministic Planning

slide-28
SLIDE 28

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Pattern Selection

[Haslum et al. 2007]

Question: Which abstractions to use? Problem: In general no domain knowledge. Then how to compute suitable patterns?

◮ Local search in space of additive pattern collections. ◮ Quality criterion: minimize expected number of node expansions

  • f IDA* search with pattern collection.

PDB Heuristics for Nondeterministic Planning

slide-29
SLIDE 29

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Experiments

◮ Compared planners:

◮ LAO* + PDB heuristic. ◮ LAO* + determinization + delete-relaxation heuristic. ◮ LAO* without heuristic. ◮ Gamer [Kissmann and Edelkamp 2009], which uses BDD-based

symbolic reachability analysis and regression search.

◮ Tasks: IPC 2008 benchmarks (FOND track).

PDB Heuristics for Nondeterministic Planning

slide-30
SLIDE 30

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Experiments: Problems Solved (15 Minutes per Problem)

Heuristic Domain (probs) PDB DR None Gamer blocksworld (30) 10 10 10 10 faults (55) 55 54 33 34 first-responders (100) 23 24 19 19 forest (90) 6 6 3 6

  • verall (275)

94 94 65 69

PDB Heuristics for Nondeterministic Planning

slide-31
SLIDE 31

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Experiments: Coverage over Time

10 20 30 40 50 60 70 80 90 100 0.01 0.1 1 10 100 Coverage Time (sec) PDB DR 0/1 Gamer

PDB Heuristics for Nondeterministic Planning

slide-32
SLIDE 32

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Experiments: Runtimes and Guidance (Node Expansions)

PDBs Delete Relaxation None Gamer Problem

t n t n t n t

bw-1 25.10 43 0.20 50 0.16 296 220.73 bw-2 3.91 293 0.28 293 0.07 92 211.27 bw-3 4.23 931 0.48 931 0.85 2335 206.07 bw-4 4.90 8515 4.84 23154 6.39 24406 203.46 bw-5 4.88 4899 1.39 5968 0.92 3476 202.66 bw-6 4.39 2960 0.87 2960 0.72 2710 196.37 bw-7 5.43 10277 1.95 8549 1.28 5373 198.17 bw-8 5.90 14515 1.84 8718 3.22 15754 197.07 bw-9 4.42 34 0.35 626 0.29 534 203.51 bw-10 4.66 1988 0.95 3080 0.89 3904 205.38 faults-5-5 26.14 329 0.73 509 43.75 6138 168.35 faults-6-4 19.26 5987 1.75 7072 35.03 13157 88.39 faults-7-4 51.92 46964 3.00 15152 157.93 39895 25.34 faults-8-3 23.54 26311 4.76 26304 58.39 32351 107.69 faults-9-3 42.62 23836 9.03 49068 240.82 82410 285.15 faults-10-2 27.54 882 2.13 15012 48.02 20358 84.25 fr-1-6 2.83 9776 2.27 9776 119.56 7414 1.23 fr-2-4 1.74 1191 2.17 7780 8.13 6400 38.31 fr-4-3 2.61 8060 2.19 8060 24.66 20928 631.95 forest-2-5 21.31 6378 13.43 4138 229.22 6841 2.03

PDB Heuristics for Nondeterministic Planning

slide-33
SLIDE 33

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Discussion

◮ Delete relaxation + determinization surprisingly good. ◮ High preprocessing cost for PDB heuristics. ◮ But: PDBs provide good guidance, fast lookup. ◮ Generally: Informed progression search feasible approach

to strong cyclic planning.

PDB Heuristics for Nondeterministic Planning

slide-34
SLIDE 34

Introduction Strong Cyclic Planning Pattern Database Heuristics Experiments

Discussion

◮ Delete relaxation + determinization surprisingly good. ◮ High preprocessing cost for PDB heuristics. ◮ But: PDBs provide good guidance, fast lookup. ◮ Generally: Informed progression search feasible approach

to strong cyclic planning. Thank you!

PDB Heuristics for Nondeterministic Planning