Generalization of Cycle-Covering Heuristics Clemens B uchner - - PowerPoint PPT Presentation

Generalization of Cycle-Covering Heuristics

Master’s Thesis Presentation

Generalization of Cycle-Covering Heuristics

Clemens B¨ uchner

Department of Mathematics and Computer Science University of Basel

May 14, 2020

Generalization of Cycle-Covering Heuristics Introduction

DMI Spiegelgasse

Generalization of Cycle-Covering Heuristics Introduction

Outline

• 1. Background
• 2. Cycle-covering heuristic
• 3. Experimental results

Generalization of Cycle-Covering Heuristics Background

Background

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with

L

To

L

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state,

L

To

L

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0,

A B C

C

To

B

To

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0, ◮ the goal condition γ, and

A B C

C

To

B

To

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0, ◮ the goal condition γ, and ◮ a finite set of actions A to transition between states.

A B C

C

To

B

To

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0, ◮ the goal condition γ, and ◮ a finite set of actions A to transition between states.

A B C

C

To

B

To

move(A, B)

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0, ◮ the goal condition γ, and ◮ a finite set of actions A to transition between states.

A B C

C

To

B

To

move(A, B)

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0, ◮ the goal condition γ, and ◮ a finite set of actions A to transition between states.

A B C

C

To

B

To

move(B, C)

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0, ◮ the goal condition γ, and ◮ a finite set of actions A to transition between states.

A B C

C

To

B

To

move(B, C)

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0, ◮ the goal condition γ, and ◮ a finite set of actions A to transition between states.

A B C

C

To

B

To

move(C, A)

Generalization of Cycle-Covering Heuristics Background

Planning

Simplistic world model for specific problem purposes:

Definition (planning task)

A planning task is a 4-tuple T = V, s0, γ, A with ◮ a finite set of variables V to describe each world state, ◮ the initial state s0, ◮ the goal condition γ, and ◮ a finite set of actions A to transition between states.

A B C

C

To

B

To

move(A, B)

Generalization of Cycle-Covering Heuristics Background

State Space and Heuristic Search

A B C

C

To

B

To

A B C

C

To

B

To

A B C

C

To

B

To

A B C

C

To

B

To

A B C

C

To

B

To

A B C

C

To

B

To

A B C

C

To

B

To

... move(A, B) move(B, C) move(C, B) move(A, C) move(C, B) move(B, C)

Generalization of Cycle-Covering Heuristics Background

Landmarks

Properties that must hold along all plans: ◮ move to B to pick up yellow package ◮ move to C to deliver yellow package ◮ the blue package induces the same landmarks

A B C

C

To

B

To

Generalization of Cycle-Covering Heuristics Background

Landmarks

Properties that must hold along all plans:

Definition (disjunctive action landmark)

Let T = V, s0, G, A be a planning task and let s be a state of T . A disjunctive action landmark of s is a non-empty set of actions ℓ ⊆ A such that every s-plan contains an action a ∈ ℓ.

A B C

C

To

B

To

Generalization of Cycle-Covering Heuristics Background

Landmarks

Properties that must hold along all plans:

Definition (disjunctive action landmark)

Let T = V, s0, G, A be a planning task and let s be a state of T . A disjunctive action landmark of s is a non-empty set of actions ℓ ⊆ A such that every s-plan contains an action a ∈ ℓ. ◮ in example: {move(A, B), move(C, B)} and {move(A, C), move(B, C)}

A B C

C

To

B

To

Generalization of Cycle-Covering Heuristics Background

Landmarks

Properties that must hold along all plans:

Definition (disjunctive action landmark)

Let T = V, s0, G, A be a planning task and let s be a state of T . A disjunctive action landmark of s is a non-empty set of actions ℓ ⊆ A such that every s-plan contains an action a ∈ ℓ. ◮ in example: {move(A, B), move(C, B)} and {move(A, C), move(B, C)} ◮ landmark generation is not the topic of this thesis

A B C

C

To

B

To

Generalization of Cycle-Covering Heuristics Background

Landmark Heuristic hLM

◮ one action from each landmark must be part of every plan ◮ minimum hitting set approach

◮ cheapest set of actions that hits each landmark

◮ solve with linear programming min

• a∈A

Ya · cost(a) s.t. Ya ≥ 0 for all a ∈ A and

• a∈ℓ

Ya ≥ 1 for all ℓ ∈ L ◮ this corresponds to the operator-counting framework ◮ use objective value as heuristic estimate

Generalization of Cycle-Covering Heuristics Background

Landmark Orderings

Landmark orderings denote dependencies between landmarks. ◮ natural orderings must hold along all plans

◮ e.g., impossible to unload package before loaded

◮ reasonable orderings are rather “suggestions”

◮ e.g., move to the package’s origin before its destination

Generalization of Cycle-Covering Heuristics Background

Landmark Orderings

Landmark orderings denote dependencies between landmarks. ◮ natural orderings must hold along all plans

◮ e.g., impossible to unload package before loaded

◮ reasonable orderings are rather “suggestions”

◮ e.g., move to the package’s origin before its destination

Represent landmarks and orderings in landmark graphs: {move(A, B), move(C, B)} {move(A, C), move(B, C)} r r

Generalization of Cycle-Covering Heuristics Cycle-Covering

Cycle-Covering

Generalization of Cycle-Covering Heuristics Cycle-Covering

Valuable Information in Landmark Graphs

◮ cyclical dependencies between landmarks ◮ sub-goal must be achieved multiple times to resolve cycle ◮ one landmark per cycle necessary twice in every plan ◮ again minimum hitting set problem for cycles

A B C

C

To

B

To

Generalization of Cycle-Covering Heuristics Cycle-Covering

Cycle-Covering Heuristic hcycle

Extending the landmark heuristic with cycle constraints: min

• a∈A

Ya · cost(a) s.t. Ya ≥ 0 for all a ∈ A and

• a∈ℓ

Ya ≥ 1 for all ℓ ∈ L and

• ℓ∈c
• a∈ℓ

Ya ≥ |c| + 1 for all c ∈ C with C the set of cycles in the landmark graph

Generalization of Cycle-Covering Heuristics Cycle-Covering

Heuristics Applied to Running Example

{move(A, B), move(C, B)} {move(A, C), move(B, C)} r r min YA→B + YC→B + YA→C + YB→C s.t. YA→B + YC→B ≥ 1 YA→C + YB→C ≥ 1

Generalization of Cycle-Covering Heuristics Cycle-Covering

Heuristics Applied to Running Example

{move(A, B), move(C, B)} {move(A, C), move(B, C)} r r min YA→B + YC→B + YA→C + YB→C s.t. YA→B + YC→B ≥ 1 YA→C + YB→C ≥ 1 ◮ hLM(s0) = 2 (YA→B = 1, YB→C = 1)

Generalization of Cycle-Covering Heuristics Cycle-Covering

Heuristics Applied to Running Example

{move(A, B), move(C, B)} {move(A, C), move(B, C)} r r min YA→B + YC→B + YA→C + YB→C s.t. YA→B + YC→B ≥ 1 YA→C + YB→C ≥ 1 YA→B + YC→B + YA→C + YB→C ≥ 3 ◮ hLM(s0) = 2 (YA→B = 1, YB→C = 1)

Generalization of Cycle-Covering Heuristics Cycle-Covering

Heuristics Applied to Running Example

{move(A, B), move(C, B)} {move(A, C), move(B, C)} r r min YA→B + YC→B + YA→C + YB→C s.t. YA→B + YC→B ≥ 1 YA→C + YB→C ≥ 1 YA→B + YC→B + YA→C + YB→C ≥ 3 ◮ hLM(s0) = 2 (YA→B = 1, YB→C = 1) ◮ hcycle(s0) = 3 (YA→B = 1, YB→C = 1, YC→B = 1)

Generalization of Cycle-Covering Heuristics Cycle-Covering

Ordering-Aware Cycle-Covering Heuristic hord

◮ natural orderings are acyclic by definition ◮ candidates for resolving cycles must have an incoming reasonable ordering min Ya + Yb + Yc s.t. Ya ≥ 1 Yb ≥ 1 Yc ≥ 1 Ya + Yb ≥ 3 Yb + Yc ≥ 3 ◮ hcycle(s) = 4 (Ya = 1, Yb = 2, Yc = 1) {a} {b} {c} r r r n

Generalization of Cycle-Covering Heuristics Cycle-Covering

Ordering-Aware Cycle-Covering Heuristic hord

◮ natural orderings are acyclic by definition ◮ candidates for resolving cycles must have an incoming reasonable ordering min Ya + Yb + Yc s.t. Ya ≥ 1 Yb ≥ 1 Yc ≥ 1 Ya + Yb ≥ 3

✘✘ ✘ ❳❳ ❳

Yb + Yc ≥ ✁

❆

3 2 ◮ hcycle(s) = 4 (Ya = 1, Yb = 2, Yc = 1) ◮ hord(s) = 5 (Ya = 1, Yb = 2, Yc = 2) {a} {b} {c} r r r n

Generalization of Cycle-Covering Heuristics Cycle-Covering

Ordering-Aware Cycle-Covering Heuristic hord

◮ natural orderings are acyclic by definition ◮ candidates for resolving cycles must have an incoming reasonable ordering Ordering-aware cycle-covering heuristic: min

• a∈A

Ya · cost(a) s.t. Ya ≥ 0 for all a ∈ A and

• a∈ℓ

Ya ≥ 1 for all ℓ ∈ L and

• ℓ∈cr
• a∈ℓ

Ya ≥ |cr| + 1 for all c ∈ C with cr ⊆ c the set of landmarks with incoming reasonable orderings {a} {b} {c} r r r n

Generalization of Cycle-Covering Heuristics Experimental Evaluation

Experimental Evaluation

Generalization of Cycle-Covering Heuristics Experimental Evaluation

Cycles in Landmark Graphs

100 101 102 103 104 105 50 100 150 200 250 amount of cycles tasks caldera blocks spider sokoban agricola flashfill pathways airport scanalyzer thoughtful nomystery parcprinter logistics depot miconic zenotravel elevators

Generalization of Cycle-Covering Heuristics Experimental Evaluation

Initial h-Value – hLM vs. hcycle

25 50 75 100 125 25 50 75 100 125 hLM hcycle agricola blocks depot elevators logistics miconic nomystery zenotravel

Generalization of Cycle-Covering Heuristics Experimental Evaluation

Initial h-Value – hLM vs. hord

25 50 75 100 125 25 50 75 100 125 hLM hord agricola blocks depot elevators logistics miconic nomystery zenotravel caldera flashfill spider

Generalization of Cycle-Covering Heuristics Experimental Evaluation

Aiming for Optimality

25 50 75 100 25 50 75 100 h∗ hord 25 50 75 100 25 50 75 100 h+ higher lower equal

Generalization of Cycle-Covering Heuristics Experimental Evaluation

Aiming for Optimality

25 50 75 100 25 50 75 100 h∗ hord 25 50 75 100 25 50 75 100 h+ higher lower equal

Generalization of Cycle-Covering Heuristics Experimental Evaluation

Planning with the Cycle-Covering Heuristic

Coverage barely affected ◮ tasks with many cycles prone to exceed memory limit ◮ increased complexity of optimization problems Overall results are inconclusive and vary depending on several factors: ◮ Which landmark generator is used?

◮ various options in Fast Downward

◮ How to update landmarks in encountered states?

◮ recomputing vs. tracking based on previous state

Generalization of Cycle-Covering Heuristics Experimental Evaluation

Planning with the Cycle-Covering Heuristic

100 101 102 103 100 101 102 103 uns. uns. hLM hord Search Time 100 102 104 100 102 104 uns. uns. hLM Expanded States

Generalization of Cycle-Covering Heuristics Summary

Summary

◮ cyclical dependencies between landmarks contain valuable information ◮ cycle-covering heuristic dominates minimum hitting set landmark heuristic for the same landmark graph ◮ considering ordering types improves cycle-covering heuristic ◮ increased heuristic accuracy in practice ◮ does not (yet) pay off in coverage

Generalization of Cycle-Covering Heuristics Appendix

Appendix

Generalization of Cycle-Covering Heuristics Appendix

Initial h-Values – LP vs. IP

◮ IP solutions identical to LP-relaxation ◮ possible explanation: totally unimodular matrices

◮ all squared sub-matrices have det ∈ {−1, 0, 1} ◮ LP solutions are integral

◮ but not generally the case

◮ Counterexample with LP = IP

100 101 102 100 101 102 LP IP

Generalization of Cycle-Covering Heuristics Appendix

Comparison to LM-Cut

100 101 102 103 100 101 102 103 hord hLM-cut Initial h-Value 10−1 102 105 10−1 102 105 uns. uns. hord Expanded States 10−2 100 102 10−2 100 102 uns. uns. hord Search Time

Generalization of Cycle-Covering Heuristics Appendix

Decomposing Cycle-Covering from Landmark Hitting Set

{a} {b} {c} {a, c} ◮ Minimum landmark hitting set: 3, minimum cycle hitting set: 1 ⇒ h = 4

Generalization of Cycle-Covering Heuristics Appendix

Decomposing Cycle-Covering from Landmark Hitting Set

{a} {b} {c} {a, c} ◮ Minimum landmark hitting set: 3, minimum cycle hitting set: 1 ⇒ h = 4 ◮ Optimal plan: a, b, c ⇒ h∗ = 3

Generalization of Cycle-Covering Heuristics Appendix

Decomposing Cycle-Covering from Landmark Hitting Set

{a} {b} {c} {a, c} ◮ Minimum landmark hitting set: 3, minimum cycle hitting set: 1 ⇒ h = 4 ◮ Optimal plan: a, b, c ⇒ h∗ = 3

Generalization of Cycle-Covering Heuristics Appendix

Decomposing Cycle-Covering from Landmark Hitting Set

{a} {b} {c} {a, c} ◮ Minimum landmark hitting set: 3, minimum cycle hitting set: 1 ⇒ h = 4 ◮ Optimal plan: a, b, c ⇒ h∗ = 3

Generalization of Cycle-Covering Heuristics Appendix

Recomputing vs. Tracking Landmarks

Update landmarks in every state: ◮ always recompute

+ potentially find new landmarks − might be time-consuming

◮ or track along paths

+ compute landmarks only once in the beginning − applying actions can only decrease heuristic estimates ± heuristic is path-dependent

Generalization of Cycle-Covering Heuristics Appendix

Recomputing vs. Tracking Landmarks

100 102 104 106 100 102 104 106 uns. uns. track recomp Expanded States 100 101 102 103 100 101 102 103 uns. uns. track Search Time

Generalization of Cycle-Covering Heuristics Appendix

Coverage Results

◮ 870 planning tasks from domains with cyclical initial states LMRHW LMhm recomp track recomp track hLM 342 308 222 298 hcycle 336 305 228 305 hord 340 306 231 308

Generalization of Cycle-Covering Heuristics Appendix

Coverage Results

◮ 870 planning tasks from domains with cyclical initial states LMRHW LMhm recomp track recomp track hLM 342 308 222 298 hcycle 336 305 228 305 hord 340 306 231 308 ◮ success of recomputing vs. tracking depends on landmark generator

Generalization of Cycle-Covering Heuristics Appendix

Coverage Results

◮ 870 planning tasks from domains with cyclical initial states LMRHW LMhm recomp track recomp track hLM 342 308 222 298 hcycle 336 305 228 305 hord 340 306 231 308 ◮ success of recomputing vs. tracking depends on landmark generator ◮ considering cycles is not always beneficial

◮ memory is a limitation ◮ optimization problems have increased complexity