Iterative Budgeted Exponential Search Malte Helmert 1 , Tor Lattimore - - PowerPoint PPT Presentation

▶
iterative budgeted exponential search
SMART_READER_LITE
LIVE PREVIEW

Iterative Budgeted Exponential Search Malte Helmert 1 , Tor Lattimore - - PowerPoint PPT Presentation

Mar 05, 2023 461 likes •1.18k views

Iterative Budgeted Exponential Search Malte Helmert 1 , Tor Lattimore 2 , Levi H.S. Lelis 3 , Laurent Orseau 2 , Nathan Sturtevant 4 1 University of Basel, Switzerland 2 DeepMind, UK 3 Federal de Vi cosa, Brazil 4 University of Alberta, Canada

slide-1
SLIDE 1

Iterative Budgeted Exponential Search

Malte Helmert1, Tor Lattimore2, Levi H.S. Lelis3, Laurent Orseau2, Nathan Sturtevant4

1University of Basel, Switzerland 2DeepMind, UK 3Federal de Vi¸

cosa, Brazil

4University of Alberta, Canada

IJCAI 2019 - Macau, China

Talk jointly presented by Laurent Orseau and Nathan Sturtevant

slide-2
SLIDE 2

5 4 3 2 1 1 2 3 4 5

Start Goal h = 11

slide-3
SLIDE 3

Shortest Path Algorithm History

1960 1970 1980 1990 2000 2010

Dijkstra’s Algorithm A* A* worst case† IDA* IBEX IDA* worst case

slide-4
SLIDE 4

Iterative Budgeted Exponential Search

The N2 problem

  • Algorithms perform

re-expansions of N states

  • IDA*:

expansions of states

  • A*:

expansions of N states

  • B & B’:

(Martelli 1977; Mero, 1984)

  • Weighted A*:

(Chen et al, 2019)

N2 N2 N = bd 2N N2 N2

4

slide-5
SLIDE 5

Iterative Budgeted Exponential Search

IDA* Refresher

  • IDA* does iterative deepening search on f-costs
  • f(n) = g(n) + h(n)
  • Next iteration f-cost:
  • Smallest unexplored from previous iteration
  • If f-cost layers grow exponentially
  • IDA* is asymptotically optimal in node expansions

5

slide-6
SLIDE 6

IDA* - Unit Costs

slide-7
SLIDE 7

2 States 16 States 79 States x8 x5 f-cost 11 f-cost 13 f-cost 15

slide-8
SLIDE 8

Iterative Budgeted Exponential Search

IDA* Worst Case

  • f-cost layers grow exponentially
  • What if f-cost layers grew linearly?
  • Happens with non-unit edge costs:
  • STP: Cost of moving tile t:

1 + b + b2 + b3 + … + bd ≈ bd 1 + 2 + 3 + 4 + … + bd ≈ (bd)2 t + 2 t + 1

8

slide-9
SLIDE 9
slide-10
SLIDE 10

13.83 13.87 13.7 13.62 13.5 11.25 13.45 f-cost 11

slide-11
SLIDE 11

Iterative Budgeted Exponential Search

Previous Work

  • Iterative Deepening
  • IDA* (Korf, 1985)
  • IDA*CR (Sarkar et al, 1990)
  • IDA*IM (Burns & Ruml, 2013)
  • EDA* (Sharon et al, 2014)

11

slide-12
SLIDE 12
slide-13
SLIDE 13

Iterative Budgeted Exponential Search

A* (B/B’)

  • A* with a consistent heuristic is optimal*
  • With an inconsistent heuristic A* might do 2N

expansions of N states

  • B and B’ immediately propagate shorter paths
  • Will do at most N2 expansions of N states

13

slide-14
SLIDE 14

15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

N states N states

slide-15
SLIDE 15
slide-16
SLIDE 16

Iterative Budgeted Exponential Search

Previous Work

  • Inconsistent Heuristics
  • B/B’ (Martelli 1977; Mero, 1984)
  • Delay (Sturtevant et al, 2008)
  • BPMX (Felner et al, 2011)

16

slide-17
SLIDE 17
slide-18
SLIDE 18

Iterative Budgeted Exponential Search

Demos

  • All demos shown here runnable online:
  • https://www.movingai.com/SAS/BTS/
  • Also running on my iPad; can give detailed demo

18

slide-19
SLIDE 19

IDA* — Good Case

slide-20
SLIDE 20

IDA* — Good Case

slide-21
SLIDE 21

IDA* — Good Case

slide-22
SLIDE 22

IDA* — Good Case

slide-23
SLIDE 23

IDA* — Good Case

slide-24
SLIDE 24

IDA* — Good Case

slide-25
SLIDE 25

IDA* — Bad Case

slide-26
SLIDE 26

IDA* — Bad Case

slide-27
SLIDE 27

IDA* — Bad Case

slide-28
SLIDE 28

IDA* — Bad Case

slide-29
SLIDE 29

IDA* — Bad Case

slide-30
SLIDE 30

IDA* — Bad Case

slide-31
SLIDE 31

IDA* — Bad Case

slide-32
SLIDE 32

IDA* — Bad Case

slide-33
SLIDE 33

IDA* — Bad Case

slide-34
SLIDE 34

EDA*

slide-35
SLIDE 35

EDA*

slide-36
SLIDE 36

EDA*

slide-37
SLIDE 37

EDA*

slide-38
SLIDE 38

EDA*

slide-39
SLIDE 39

EDA*

slide-40
SLIDE 40

EDA*

slide-41
SLIDE 41

IBEX (main idea)

For budget = 2, 4, 8, 16, ...: cost_bound = Oracle(budget) search within cost_bound and within budget Oracle(budget) returns largest cost for which budget is sufficient.

slide-42
SLIDE 42

IBEX (main idea): With an oracle

budget f -cost expansions 2

slide-43
SLIDE 43

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2

slide-44
SLIDE 44

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2 2

slide-45
SLIDE 45

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2 2 4

slide-46
SLIDE 46

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2 2 4 13.5

slide-47
SLIDE 47

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2 2 4 13.5 4

slide-48
SLIDE 48

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2 2 4 13.5 4 8 13.9 8

slide-49
SLIDE 49

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2 2 4 13.5 4 8 13.9 8 16 14.4 16

slide-50
SLIDE 50

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2 2 4 13.5 4 8 13.9 8 16 14.4 16 32 16.8 30

slide-51
SLIDE 51

IBEX (main idea): With an oracle

budget f -cost expansions 2 11.2 2 4 13.5 4 8 13.9 8 16 14.4 16 32 16.8 30 64 17.7 62

slide-52
SLIDE 52

IBEX (main idea)

For budget = 2, 4, 8, 16, ...: cost_bound = Oracle(budget) search within cost_bound and within budget

slide-53
SLIDE 53

IBEX (main idea)

For budget = 2, 4, 8, 16, ...: cost_bound = Oracle(budget) search within cost_bound and within budget Oracle(budget) returns largest cost for which budget is sufficient. Oracle(budget) = (budgeted) exponential search

slide-54
SLIDE 54

IBEX (main idea)

slide-55
SLIDE 55

IBEX (main idea)

slide-56
SLIDE 56

IBEX (main idea)

slide-57
SLIDE 57

IBEX (main idea)

slide-58
SLIDE 58

IBEX (main idea)

slide-59
SLIDE 59

IBEX (main idea)

slide-60
SLIDE 60

IBEX (main idea)

slide-61
SLIDE 61

IBEX (main idea)

slide-62
SLIDE 62

IBEX (main idea)

slide-63
SLIDE 63

IBEX (main idea)

N ↓ For budget = 2, 4, 8, 16, 32, 64 : ≤ 4N With exponential search: Refine lower and upper bound on cost Proves no solution when lower = upper    O(log C ∗)

slide-64
SLIDE 64

Application: Tree Search

IBEX + Depth-First Search = Budgeted Tree Search (BTS) Alg. Worst case IDA*(1985) Θ(N2) EDA*(2014) unbounded BTS O(N log C ∗) C ∗: cost of the optimal solution N: number of nodes of cost at most C ∗

All algorithms return an optimal solution (under usual assumptions).

slide-65
SLIDE 65

Application: Graph Search

IBEX + Uniform-Cost Search = Budgeted Graph Search (BGS) Alg. Admissible Heuristic Consistent Inconsistent A*(1968) O(N) Ω(2N) B(1977) O(N) Θ(N2) BGS O(N)† O(N log C ∗) C ∗: cost of the optimal solution N: number of nodes of cost at most C ∗

All algorithms return an optimal solution (under usual assumptions).

slide-66
SLIDE 66

Enhancements: Tree search

Recover IDA* when exponential unit-cost domains

1) Infinite iteration at next f -cost 2) If expansions grow exponentially, go to 1) else exponential search

slide-67
SLIDE 67

Enhancements: Tree search

Recover IDA* when exponential unit-cost domains

1) Infinite iteration at next f -cost 2) If expansions grow exponentially, go to 1) else exponential search

Budget window [2, 8] to be more aggressive

slide-68
SLIDE 68

Experiments

Algorithm 15-Puzzle (unit) 15-Puzzle (real) Coconut Solved Exp.×106 Solved Exp.×106 Solved Exp.×104 BTS 100 242.5 100 673.1 100 84.7 EDA* 99 5 586.0 100 2 882.3 3 ≥ 554 249.0 IDA* CR 100 868.4 100 700.6 3 ≥ 516 937.7 IDA* 100 242.5 57 62 044.3 100 5 484.2 N 100 100.8 100 258.1 100 2.7

BTS (Budgeted Tree Search) = IBEX + Depth-First Search

slide-69
SLIDE 69

Experiments

Algorithm 15-Puzzle (unit) 15-Puzzle (real) Coconut Solved Exp.×106 Solved Exp.×106 Solved Exp.×104 BTS 100 242.5 100 673.1 100 84.7 EDA* 99 5 586.0 100 2 882.3 3 ≥ 554 249.0 IDA* CR 100 868.4 100 700.6 3 ≥ 516 937.7 IDA* 100 242.5 57 62 044.3 100 5 484.2 N 100 100.8 100 258.1 100 2.7

BTS (Budgeted Tree Search) = IBEX + Depth-First Search

slide-70
SLIDE 70

Experiments

Algorithm 15-Puzzle (unit) 15-Puzzle (real) Coconut Solved Exp.×106 Solved Exp.×106 Solved Exp.×104 BTS 100 242.5 100 673.1 100 84.7 EDA* 99 5 586.0 100 2 882.3 3 ≥ 554 249.0 IDA* CR 100 868.4 100 700.6 3 ≥ 516 937.7 IDA* 100 242.5 57 62 044.3 100 5 484.2 N 100 100.8 100 258.1 100 2.7

BTS (Budgeted Tree Search) = IBEX + Depth-First Search

slide-71
SLIDE 71

Conclusion

IBEX: Budgeting → O(N log C) Tree search: drop-in replacement for IDA* DovIBEX: For applications where oracle returns only if a solution is found

Interleaves budget iterations

Do you have a problem where the current worst case is O(N2), due to being cautious? Maybe (Dov)IBEX can help!

Come and see our poster!