A Guide to Budgeted Tree Search Nathan R. Sturtevant University of - - PowerPoint PPT Presentation

▶
a guide to budgeted tree search
SMART_READER_LITE
LIVE PREVIEW

A Guide to Budgeted Tree Search Nathan R. Sturtevant University of - - PowerPoint PPT Presentation

May 14, 2023 48 likes •1.37k views

A Guide to Budgeted Tree Search Nathan R. Sturtevant University of Alberta Amii Fellow, CIFAR Chair Malte Helmert Universitt Basel Talk Overview Budgeted Tree Search (BTS) is a new algorithm with better worst-case guarantees than IDA*

slide-1
SLIDE 1

A Guide to Budgeted Tree Search

Nathan R. Sturtevant University of Alberta Amii Fellow, CIFAR Chair Malte Helmert Universität Basel

slide-2
SLIDE 2

A Guide to Budgeted Tree Search

Talk Overview

  • Budgeted Tree Search (BTS) is a new algorithm with

better worst-case guarantees than IDA*

2

slide-3
SLIDE 3

A Guide to Budgeted Tree Search

Talk Overview

  • Budgeted Tree Search (BTS) is a new algorithm with

better worst-case guarantees than IDA*

  • Companion work to original paper on IBEX

(Helmert, Lattimore, Lelis, Orseau, Sturtevant, IJCAI 2019)

2

slide-4
SLIDE 4

A Guide to Budgeted Tree Search

Talk Overview

  • Budgeted Tree Search (BTS) is a new algorithm with

better worst-case guarantees than IDA*

  • Companion work to original paper on IBEX

(Helmert, Lattimore, Lelis, Orseau, Sturtevant, IJCAI 2019)

  • Why do we need BTS?

2

slide-5
SLIDE 5

A Guide to Budgeted Tree Search

Talk Overview

  • Budgeted Tree Search (BTS) is a new algorithm with

better worst-case guarantees than IDA*

  • Companion work to original paper on IBEX

(Helmert, Lattimore, Lelis, Orseau, Sturtevant, IJCAI 2019)

  • Why do we need BTS?
  • How does BTS work?

2

slide-6
SLIDE 6

5 4 3 2 1

slide-7
SLIDE 7

5 4 3 2 1

Start

slide-8
SLIDE 8

5 4 3 2 1 1 2 3 4 5

Start Goal

slide-9
SLIDE 9

5 4 3 2 1 1 2 3 4 5

Start Goal h = 11

slide-10
SLIDE 10

5 4 3 2 1 1 2 3 4 5

Start Goal h = 11

slide-11
SLIDE 11

A Guide to Budgeted Tree Search

Search Problem

  • Given:

4

slide-12
SLIDE 12

A Guide to Budgeted Tree Search

Search Problem

  • Given:
  • Start state

4

slide-13
SLIDE 13

A Guide to Budgeted Tree Search

Search Problem

  • Given:
  • Start state
  • Goal state

4

slide-14
SLIDE 14

A Guide to Budgeted Tree Search

Search Problem

  • Given:
  • Start state
  • Goal state
  • Successor function

4

slide-15
SLIDE 15

A Guide to Budgeted Tree Search

Search Problem

  • Given:
  • Start state
  • Goal state
  • Successor function
  • Cost function

4

slide-16
SLIDE 16

A Guide to Budgeted Tree Search

Search Problem

  • Given:
  • Start state
  • Goal state
  • Successor function
  • Cost function

4

← Defines implicit graph

slide-17
SLIDE 17

A Guide to Budgeted Tree Search

Search Problem

  • Given:
  • Start state
  • Goal state
  • Successor function
  • Cost function
  • Heuristic function

4

← Defines implicit graph

slide-18
SLIDE 18

A Guide to Budgeted Tree Search

Search Problem

  • Given:
  • Start state
  • Goal state
  • Successor function
  • Cost function
  • Heuristic function
  • Find:

4

← Defines implicit graph

slide-19
SLIDE 19

A Guide to Budgeted Tree Search

Search Problem

  • Given:
  • Start state
  • Goal state
  • Successor function
  • Cost function
  • Heuristic function
  • Find:
  • Optimal path between start/goal

4

← Defines implicit graph

slide-20
SLIDE 20

Why do we need BTS? If the nodes in each iteration do not grow exponentially

slide-21
SLIDE 21

A Guide to Budgeted Tree Search

IDA* Refresher

  • IDA* does iterative deepening search on f-costs
  • f(n) = g(n) + h(n)

6

slide-22
SLIDE 22

A Guide to Budgeted Tree Search

IDA* Refresher

  • IDA* does iterative deepening search on f-costs
  • f(n) = g(n) + h(n)
  • Next iteration f-cost:
  • Smallest unexplored from previous iteration

6

slide-23
SLIDE 23

IDA* - Unit Costs

slide-24
SLIDE 24

IDA* - Unit Costs

slide-25
SLIDE 25

2 States f-cost 11

slide-26
SLIDE 26

2 States 16 States x8 f-cost 11 f-cost 13

slide-27
SLIDE 27

2 States 16 States 79 States x8 x5 f-cost 11 f-cost 13 f-cost 15

slide-28
SLIDE 28

A Guide to Budgeted Tree Search

IDA* Worst Case

9

slide-29
SLIDE 29

A Guide to Budgeted Tree Search

IDA* Worst Case

  • f-cost layers grow exponentially

9

slide-30
SLIDE 30

A Guide to Budgeted Tree Search

IDA* Worst Case

  • f-cost layers grow exponentially
  • 1 + b + b2 + b3 + … + bd ≈ bd

9

slide-31
SLIDE 31

A Guide to Budgeted Tree Search

IDA* Worst Case

  • f-cost layers grow exponentially
  • 1 + b + b2 + b3 + … + bd ≈ bd

9

slide-32
SLIDE 32

A Guide to Budgeted Tree Search

IDA* Worst Case

  • f-cost layers grow exponentially
  • 1 + b + b2 + b3 + … + bd ≈ bd
  • What if f-cost layers grew linearly?

9

slide-33
SLIDE 33

A Guide to Budgeted Tree Search

IDA* Worst Case

  • f-cost layers grow exponentially
  • 1 + b + b2 + b3 + … + bd ≈ bd
  • What if f-cost layers grew linearly?
  • 1 + 2 + 3 + 4 + … + bd ≈ (bd)2

9

slide-34
SLIDE 34

A Guide to Budgeted Tree Search

IDA* Worst Case

  • f-cost layers grow exponentially
  • 1 + b + b2 + b3 + … + bd ≈ bd
  • What if f-cost layers grew linearly?
  • 1 + 2 + 3 + 4 + … + bd ≈ (bd)2

9

slide-35
SLIDE 35

A Guide to Budgeted Tree Search

IDA* Worst Case

  • f-cost layers grow exponentially
  • 1 + b + b2 + b3 + … + bd ≈ bd
  • What if f-cost layers grew linearly?
  • 1 + 2 + 3 + 4 + … + bd ≈ (bd)2
  • Happens with non-unit edge costs:
  • STP: Cost of moving tile t: t + 2

t + 1

9

slide-36
SLIDE 36

A Guide to Budgeted Tree Search

IDA* Worst Case

  • f-cost layers grow exponentially
  • 1 + b + b2 + b3 + … + bd ≈ bd
  • What if f-cost layers grew linearly?
  • 1 + 2 + 3 + 4 + … + bd ≈ (bd)2
  • Happens with non-unit edge costs:
  • STP: Cost of moving tile t: t + 2

t + 1

9

Tile Cost 1 3 7 9

1 + 2 1 + 1 = 1.5 9 + 2 9 + 1 = 1.1 3 + 2 3 + 1 = 1.25 7 + 2 7 + 1 = 1.125

slide-37
SLIDE 37
slide-38
SLIDE 38
slide-39
SLIDE 39

f-cost 11

slide-40
SLIDE 40

11.25 f-cost 11

slide-41
SLIDE 41

11.25 13.45 f-cost 11

slide-42
SLIDE 42

13.5 11.25 13.45 f-cost 11

slide-43
SLIDE 43

13.62 13.5 11.25 13.45 f-cost 11

slide-44
SLIDE 44

13.7 13.62 13.5 11.25 13.45 f-cost 11

slide-45
SLIDE 45

13.83 13.7 13.62 13.5 11.25 13.45 f-cost 11

slide-46
SLIDE 46

13.83 13.87 13.7 13.62 13.5 11.25 13.45 f-cost 11

slide-47
SLIDE 47

Why do we need BTS? If the nodes in each iteration do not grow exponentially

slide-48
SLIDE 48

A Guide to Budgeted Tree Search

Getting the next bound

13

slide-49
SLIDE 49

A Guide to Budgeted Tree Search

Getting the next bound

  • Can be conservative:
  • IDA* (Korf, 1985)

13

slide-50
SLIDE 50

A Guide to Budgeted Tree Search

Getting the next bound

  • Can be conservative:
  • IDA* (Korf, 1985)
  • Can try to build a predictor based on past:
  • IDA*CR (Sarkar et al, 1990)
  • IDA*IM (Burns & Ruml, 2013)

13

slide-51
SLIDE 51

A Guide to Budgeted Tree Search

Getting the next bound

  • Can be conservative:
  • IDA* (Korf, 1985)
  • Can try to build a predictor based on past:
  • IDA*CR (Sarkar et al, 1990)
  • IDA*IM (Burns & Ruml, 2013)
  • Can model the state space growth:
  • EDA* (Sharon et al, 2014)

13

slide-52
SLIDE 52

A Guide to Budgeted Tree Search

Getting the next bound

  • Can be conservative:
  • IDA* (Korf, 1985)
  • Can try to build a predictor based on past:
  • IDA*CR (Sarkar et al, 1990)
  • IDA*IM (Burns & Ruml, 2013)
  • Can model the state space growth:
  • EDA* (Sharon et al, 2014)
  • Want to guarantee exponential growth in expansions

13

slide-53
SLIDE 53

1 node

f = 11

slide-54
SLIDE 54

1 node

f = 11

[2, 8]

slide-55
SLIDE 55

2 nodes

f = 11.25

slide-56
SLIDE 56

2 nodes

f = 11.25

[4, 16]

slide-57
SLIDE 57

11 nodes

f = 13.97

slide-58
SLIDE 58

11 nodes

f = 13.97

[22, 88]

slide-59
SLIDE 59

47 nodes

f = 17.17

slide-60
SLIDE 60

47 nodes

f = 17.17

[94, 376]

slide-61
SLIDE 61

99 nodes

f = 18.32

slide-62
SLIDE 62

99 nodes

f = 18.32

[198, 495]

slide-63
SLIDE 63

117 nodes

f = 19.35

slide-64
SLIDE 64

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

slide-65
SLIDE 65

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

slide-66
SLIDE 66

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

slide-67
SLIDE 67

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

<

slide-68
SLIDE 68

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

<<

slide-69
SLIDE 69

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

<< <

slide-70
SLIDE 70

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

<< < <

slide-71
SLIDE 71

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

<< < < <

slide-72
SLIDE 72

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

<< < < < <

slide-73
SLIDE 73

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

<< < < < < >

slide-74
SLIDE 74

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

…

<< < < < < >

slide-75
SLIDE 75

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

… Binary Search

<< < < < < >

slide-76
SLIDE 76

A Guide to Budgeted Tree Search

Exponential Search

  • Bentley and Yao, 1976
  • Algorithm for searching sorted/unbounded array

20

… Binary Search

  • Running time: log(i)

<< < < < < >

slide-77
SLIDE 77

A Guide to Budgeted Tree Search

Nodes and f-costs

  • Exponential Search:
  • Find value in unbounded sorted array
  • Tree Search:
  • Find (node expansions) in (f-costs)
  • Nodes expansions non-decreasing with f-cost

21

slide-78
SLIDE 78

How does BTS work? If the nodes in each iteration do not grow exponentially

slide-79
SLIDE 79

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

slide-80
SLIDE 80

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

slide-81
SLIDE 81

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

slide-82
SLIDE 82

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

ni = 100

slide-83
SLIDE 83

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

ni = 100 200

slide-84
SLIDE 84

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

ni = 100 200 800

slide-85
SLIDE 85

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

ni = 100 < 200 200 800

slide-86
SLIDE 86

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

ni = 100 < 200 200 800 ≥ 800

slide-87
SLIDE 87

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

ni = 100 < 200 200 800 ≥ 800 Budget: Stop when exceeded

slide-88
SLIDE 88

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

23

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

ni = 100 < 200 200 800 ≥ 800 Budget: Stop when exceeded

slide-89
SLIDE 89

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

slide-90
SLIDE 90

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

slide-91
SLIDE 91

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

slide-92
SLIDE 92

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

slide-93
SLIDE 93

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

slide-94
SLIDE 94

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 <

slide-95
SLIDE 95

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 <

slide-96
SLIDE 96

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 <

slide-97
SLIDE 97

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 < 18 <

slide-98
SLIDE 98

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 < 18 < 26 <

slide-99
SLIDE 99

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 < 18 < 26 < 42 >

slide-100
SLIDE 100

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 < 18 < 26 < 42 > 34 >

slide-101
SLIDE 101

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 < 18 < 26 < 42 > 34 > 30 =

slide-102
SLIDE 102

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 < 18 < 26 < 42 > 34 > 30 =

nilog(fi)

slide-103
SLIDE 103

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 < 18 < 26 < 42 > 34 > 30 = i

∑ nilog(fi)

slide-104
SLIDE 104

A Guide to Budgeted Tree Search

Budgeted search

  • Like exponential search on f-costs

24

f = 10

f = ∞

f = 27.2 (2×) f = 30.7 (8×)

11 < 12 < 14 < 18 < 26 < 42 > 34 > 30 = i

∑ nilog(fi) < (

i

∑ ni)log(C*) ≈ N log(C*)

slide-105
SLIDE 105

A Guide to Budgeted Tree Search

BTS Phases

Find next f-cost bound:

25

slide-106
SLIDE 106

A Guide to Budgeted Tree Search

BTS Phases

Find next f-cost bound:

  • 1. Search with conservative f, ∞ budget (IDA*)

25

slide-107
SLIDE 107

A Guide to Budgeted Tree Search

BTS Phases

Find next f-cost bound:

  • 1. Search with conservative f, ∞ budget (IDA*)
  • 2. Grow f exponentially, constant budget

25

slide-108
SLIDE 108

A Guide to Budgeted Tree Search

BTS Phases

Find next f-cost bound:

  • 1. Search with conservative f, ∞ budget (IDA*)
  • 2. Grow f exponentially, constant budget
  • 3. Do binary search on f, constant budget

25

slide-109
SLIDE 109

A Guide to Budgeted Tree Search

BTS Phases

Find next f-cost bound:

  • 1. Search with conservative f, ∞ budget (IDA*)
  • 2. Grow f exponentially, constant budget
  • 3. Do binary search on f, constant budget

25

slide-110
SLIDE 110

f-limit: (13.50+14.45)/2=13.97 nodes: [4,16] expand: 11

Previous Iteration

slide-111
SLIDE 111

f-limit: (13.50+14.45)/2=13.97 nodes: [4,16] expand: 11

Previous Iteration

slide-112
SLIDE 112

f-limit: (13.50+14.45)/2=13.97 nodes: [4,16] expand: 11

Previous Iteration

slide-113
SLIDE 113

IDA*

slide-114
SLIDE 114

IDA*

slide-115
SLIDE 115

IDA*

slide-116
SLIDE 116

IDA*

slide-117
SLIDE 117

IDA*

slide-118
SLIDE 118

EXP

slide-119
SLIDE 119

EXP

slide-120
SLIDE 120

EXP

slide-121
SLIDE 121

EXP

slide-122
SLIDE 122

EXP

slide-123
SLIDE 123

EXP

slide-124
SLIDE 124

EXP

slide-125
SLIDE 125

BIN

slide-126
SLIDE 126

BIN

slide-127
SLIDE 127

BIN

slide-128
SLIDE 128

With budget { c b a

How does BTS work? IDA* Exponential Search Binary Search

slide-129
SLIDE 129

IDA* BTS*

slide-130
SLIDE 130

IDA* BTS*

slide-131
SLIDE 131

A Guide to Budgeted Tree Search

Conclusions

  • BTS reduces worst case of IDA*
  • Same performance as IDA* if tree grows

exponentially

  • IBEX solves similar problems in different contexts

33

slide-132
SLIDE 132

A Guide to Budgeted Tree Search

Conclusions

  • BTS reduces worst case of IDA*
  • Same performance as IDA* if tree grows

exponentially

  • IBEX solves similar problems in different contexts
  • Demos & videos online:
  • https://www.movingai.com/SAS/
  • https://www.movingai.com/SAS/BTS/

33