Online Saturated Cost Partitioning for Classical Planning Jendrik - - PowerPoint PPT Presentation

Online Saturated Cost Partitioning for Classical Planning

Jendrik Seipp October 21, 2020

University of Basel 1/17

Setting

• optimal classical planning
• A∗ search + admissible heuristic
• multiple abstraction heuristics
• cost partitioning

2/17

Setting

• optimal classical planning
• A∗ search + admissible heuristic
• multiple abstraction heuristics
• saturated cost partitioning

2/17

Coverage over time

100 101 102 103 200 400 600 800 1,000 1,200

time in seconds solved tasks

• ffline-1000s

3/17

Background

s1,s2 s3 s4,s5 4 4 1 1 s1 s2,s3,s4 s5 4 4 1 1

h1 s2 5 h2 s2 4 maximize over estimates:

• h s2

5

• only selects best heuristic
• does not combine heuristics

4/17

Background

s1,s2 s3 s4,s5 4 4 1 1 s1 s2,s3,s4 s5 4 4 1 1

h1(s2) = 5 h2(s2) = 4 maximize over estimates:

• h s2

5

• only selects best heuristic
• does not combine heuristics

4/17

Background

s1,s2 s3 s4,s5 4 4 1 1 s1 s2,s3,s4 s5 4 4 1 1

h1(s2) = 5 h2(s2) = 4 maximize over estimates:

• h(s2) = 5
• only selects best heuristic
• does not combine heuristics

4/17

Background

s1,s2 s3 s4,s5 4 4 1 1 s1 s2,s3,s4 s5 4 4 1 1

h1(s2) = 5 h2(s2) = 4 maximize over estimates:

• h(s2) = 5
• only selects best heuristic
• does not combine heuristics

4/17

Background

Cost partitioning

• split action costs among heuristics
• sum of costs ≤ original cost

s1,s2 s3 s4,s5 4 4 1 1 s1 s2,s3,s4 s5 4 4 1 1

h s2 3 3 6

5/17

Background

Cost partitioning

• split action costs among heuristics
• sum of costs ≤ original cost

s1,s2 s3 s4,s5 2 1 1 s1 s2,s3,s4 s5 1 3 1

h s2 3 3 6

5/17

Background

Cost partitioning

• split action costs among heuristics
• sum of costs ≤ original cost

s1,s2 s3 s4,s5 2 1 1 s1 s2,s3,s4 s5 1 3 1

h(s2) = 3 + 3 = 6

5/17

Background

Saturated cost partitioning

• order heuristics, then for each heuristic h:
• use minimum costs preserving all estimates of h
• use remaining costs for subsequent heuristics

s1,s2 s3 s4,s5 4 4 1 1 s1 s2,s3,s4 s5 4 4 1 1

hSCP s2 5 3 8

6/17

Background

Saturated cost partitioning

• order heuristics, then for each heuristic h:
• use minimum costs preserving all estimates of h
• use remaining costs for subsequent heuristics

s1,s2 s3 s4,s5 4 4 1 1 s1 s2,s3,s4 s5

hSCP s2 5 3 8

6/17

Background

Saturated cost partitioning

• order heuristics, then for each heuristic h:
• use minimum costs preserving all estimates of h
• use remaining costs for subsequent heuristics

s1,s2 s3 s4,s5 4 1 1 s1 s2,s3,s4 s5

hSCP s2 5 3 8

6/17

Background

Saturated cost partitioning

• order heuristics, then for each heuristic h:
• use minimum costs preserving all estimates of h
• use remaining costs for subsequent heuristics

s1,s2 s3 s4,s5 4 1 1 s1 s2,s3,s4 s5 3 1

hSCP s2 5 3 8

6/17

Background

Saturated cost partitioning

• order heuristics, then for each heuristic h:
• use minimum costs preserving all estimates of h
• use remaining costs for subsequent heuristics

s1,s2 s3 s4,s5 4 1 1 s1 s2,s3,s4 s5 3

hSCP s2 5 3 8

6/17

Background

Saturated cost partitioning

• order heuristics, then for each heuristic h:
• use minimum costs preserving all estimates of h
• use remaining costs for subsequent heuristics

s1,s2 s3 s4,s5 4 1 1 s1 s2,s3,s4 s5 3

hSCP(s2) = 5 + 3 = 8

6/17

Background

Order matters:

• hSCP

⟨h1,h2⟩(s2) = 8

• hSCP

⟨h2,h1⟩(s2) = 7

use multiple orders and maximize over estimates: hSCP

h1 h2 s2

hSCP

h2 h1 s2 7/17

Background

Order matters:

• hSCP

⟨h1,h2⟩(s2) = 8

• hSCP

⟨h2,h1⟩(s2) = 7

→ use multiple orders and maximize over estimates: max(hSCP

⟨h1,h2⟩(s2), hSCP ⟨h2,h1⟩(s2)) 7/17

Background

Offline diversification

• sample 1000 states
• start with empty set of orders
• until time limit is reached:
• compute order for new sample
• store order if a sample profits from it

8/17

Coverage over time

100 101 102 103 200 400 600 800 1,000 1,200

time in seconds solved tasks

• ffline-1000s

9/17

Coverage over time

100 101 102 103 200 400 600 800 1,000 1,200

time in seconds solved tasks

• ffline-1000s
• nline-nodiv

9/17

Coverage over time

100 101 102 103 200 400 600 800 1,000 1,200

time in seconds solved tasks

• ffline-1000s
• nline-nodiv
• nline-1000s

9/17

Online diversification

ComputeHeuristic(s)

• if Select(s) and not time limit reached
• compute order for s
• store order if s profits from it
• return maximum over all stored orders for s

10/17

Offline vs. online diversification

Offline

• compute orders for samples for T seconds
• store order if one of 1000 samples profits from it

Online

• compute orders for subset of evaluated states for at most T seconds
• store order if single evaluated state profits from it

11/17

Selection strategies

Select

• Interval
• Novelty (Lipovetzky and Geffner 2012)
• Bellman (Eifler and Fickert 2018):

h(s) ≥ mins

a

− →t∈T(h(t) + cost(a)) Bellman Novelty 1 Novelty 2 Interval 1–100K Coverage 1145 1153 1157 1153–1159

12/17

Selection strategies

Select

• Interval
• Novelty (Lipovetzky and Geffner 2012)
• Bellman (Eifler and Fickert 2018):

h(s) ≥ mins

a

− →t∈T(h(t) + cost(a)) Bellman Novelty 1 Novelty 2 Interval 1–100K Coverage 1145 1153 1157 1153–1159

12/17

Coverage

100 101 102 103 1,040 1,060 1,080 1,100 1,120 1,140

diversification time in seconds solved tasks

• ffline
• nline

13/17

Time score

100 101 102 103 200 400 600 800 1,000

diversification time in seconds time score

• ffline
• nline

14/17

Stored orders

100 101 102 103 100 101 102 103 failed failed

• ffline
• nline

15/17

Before the A∗ search can start

Choose which abstractions to build

• e.g., patterns for PDBs

future work Build abstractions

• e.g., Cartesian abstractions and symbolic PDBs
• nline refinement (e.g., Eifler and Fickert 2018, Franco and Torralba 2019)

Compute orders and cost partitionings

• e.g., saturated cost partitioning

Bellman, novelty, interval

16/17

Before the A∗ search can start

Choose which abstractions to build

• e.g., patterns for PDBs

future work Build abstractions

• e.g., Cartesian abstractions and symbolic PDBs

→ online refinement (e.g., Eifler and Fickert 2018, Franco and Torralba 2019) Compute orders and cost partitionings

• e.g., saturated cost partitioning

Bellman, novelty, interval

16/17

Before the A∗ search can start

Choose which abstractions to build

• e.g., patterns for PDBs

future work Build abstractions

• e.g., Cartesian abstractions and symbolic PDBs

→ online refinement (e.g., Eifler and Fickert 2018, Franco and Torralba 2019) Compute orders and cost partitionings

• e.g., saturated cost partitioning

→ Bellman, novelty, interval

16/17

Before the A∗ search can start

Choose which abstractions to build

• e.g., patterns for PDBs

future work Build abstractions

• e.g., Cartesian abstractions and symbolic PDBs

→ online refinement (e.g., Eifler and Fickert 2018, Franco and Torralba 2019) Compute orders and cost partitionings

• e.g., saturated cost partitioning

→ Bellman, novelty, interval

16/17

Before the A∗ search can start

Choose which abstractions to build

• e.g., patterns for PDBs

→ future work Build abstractions

• e.g., Cartesian abstractions and symbolic PDBs

→ online refinement (e.g., Eifler and Fickert 2018, Franco and Torralba 2019) Compute orders and cost partitionings

• e.g., saturated cost partitioning

→ Bellman, novelty, interval

16/17

Summary

Offline diversification Online computation Online diversification long precomputation no precomputation no precomputation samples states states fast evaluations slow evaluations fast evaluations high coverage low coverage high coverage

17/17

Offline vs. online diversification

1s 10s 100s 1000s 1200s 1500s Coverage

• ffline

1056 1145 1159 1156 1148 1128

• nline

1102 1135 1153 1159 1154 1146 Time Score

• ffline

791.2 690.7 420.3 86.8 59.2 25.9

• nline

920.6 929.7 919.1 908.7 906.3 906.6

Offline vs. online diversification

1s 10s 100s 1000s 1200s 1500s Coverage

• ffline

1056 1145 1159 1156 1148 1128

• nline

1102 1135 1153 1159 1154 1146 Time Score

• ffline

791.2 690.7 420.3 86.8 59.2 25.9

• nline

920.6 929.7 919.1 908.7 906.3 906.6