Informed Search A* Algorithm CE417: Introduction to Artificial - - PowerPoint PPT Presentation

CE417: Introduction to Artificial Intelligence Sharif University of Technology Spring 2019 “Artificial Intelligence: A Modern Approach”, Chapter 3 Most slides have been adopted from Klein and Abdeel, CS188, UC Berkeley.

Informed Search A* Algorithm

Soleymani

Outline

} Heuristics } Greedy (best-first) search } A* search } Finding heuristics

2

Uninformed Search

3

Uniform Cost Search

} Strategy: expand lowest path cost } The good: UCS is complete and optimal! } The bad:

}

Explores options in every “direction”

}

No information about goal location

Start Goal … c £ 3 c £ 2 c £ 1 4

UCS Example

5

Informed Search

6

Search Heuristics

§ A heuristic is:

§ A function that estimates how close a state is to a goal § Designed for a particular search problem § Examples: Manhattan distance, Euclidean distance for pathing

10 5 11.2 7

Heuristic Function

} Incorporating problem-specific knowledge in search

} Information more than problem definition } In order to come to an optimal solution as rapidly as possible

} ℎ 𝑜 : estimated cost of cheapest path from 𝑜 to a goal

} Depends only on 𝑜 (not path from root to 𝑜) } If 𝑜 is a goal state then ℎ(𝑜)=0 } ℎ(𝑜) ≥ 0

} Examples of heuristic functions include using a rule-of-thumb,

an educated guess, or an intuitive judgment

8

Example: Heuristic Function

h(x)

9

Greedy Search

10

Greedy search

} Priority queue based on ℎ(𝑜)

} e.g., ℎ'() 𝑜 = straight-line distance from n to Bucharest

} Greedy search expands the node that appears to be

closest to goal

Greedy

11

Example: Heuristic Function

h(x)

12

Romania with step costs in km

13

Greedy best-first search example

14

Greedy best-first search example

15

Greedy best-first search example

16

Greedy best-first search example

17

Greedy Search

} Expand the node that seems closest… } What can go wrong?

18

Greedy Search

} Strategy: expand a node that you think is closest to a

goal state

}

Heuristic: estimate of distance to nearest goal for each state

} A common case:

}

Best-first takes you straight to the (wrong) goal

} Worst-case: like a badly-guided DFS

… b … b 19

Properties of greedy best-first search

} Complete? No

} Similar to DFS, only graph search version is complete in finite spaces } Infinite loops, e.g., (Iasi to Fagaras) Iasi à Neamt à Iasi à Neamt

} Time

} 𝑃(𝑐𝑛), but a good heuristic can give dramatic improvement

} Space

} 𝑃(𝑐𝑛): keeps all nodes in memory

} Optimal? No

20

Greedy Search

21

A* Search

22

A* search

} Idea: minimizing the total estimated solution cost } Evaluation function for priority 𝑔 𝑜 = 𝑕 𝑜 + ℎ(𝑜)

} 𝑕 𝑜 = cost so far to reach 𝑜 } ℎ 𝑜 = estimated cost of the cheapest path from 𝑜 to goal } So, 𝑔 𝑜 = estimated total cost of path through 𝑜 to goal

24

start n … goal … Actual cost 𝑕 𝑜 Estimated cost ℎ 𝑜 𝑔 𝑜 = 𝑕 𝑜 + ℎ(𝑜)

Combining UCS and Greedy

}

Uniform-cost orders by path cost, or backward cost g(n)

}

Greedy orders by goal proximity, or forward cost h(n)

}

A* Search orders by the sum: f(n) = g(n) + h(n)

S a d b G h=5 h=6 h=2 1 8 1 1 2 h=6 h=0 c h=7 3 e h=1 1 Example: Teg Grenager S a b c e d d G G g = 0 h=6 g = 1 h=5 g = 2 h=6 g = 3 h=7 g = 4 h=2 g = 6 h=0 g = 9 h=1 g = 10 h=2 g = 12 h=0

25

When should A* terminate?

} Should we stop when we enqueue a goal? } No: only stop when we dequeue a goal

S B A G 2 3 2 2

h = 1 h = 2 h = 0 h = 3 26

Is A* Optimal?

• What went wrong?
• Actual bad goal cost < estimated good goal cost
• We need estimates to be less than actual costs!

A G S 1 3

h = 6 h = 0

5

h = 7

27

Admissible Heuristics

28

Idea: Admissibility

Inadmissible (pessimistic) heuristics break

• ptimality by trapping good plans on the frontier

Admissible (optimistic) heuristics slow down bad plans but never outweigh true costs

29

Admissible Heuristics

} A heuristic h is admissible (optimistic) if:

where is the true cost to a nearest goal

} Examples: } Coming up with admissible heuristics is most of what’s

involved in using A* in practice.

15

30

Optimality of A* Tree Search

31

Optimality of A* Tree Search

Assume:

}

A is an optimal goal node

}

B is a suboptimal goal node

}

h is admissible Claim:

}

A will exit the frontier before B

… 32

Optimality of A* Tree Search: Blocking

Proof:

}

Imagine B is on the frontier

}

Some ancestor n of A is on the frontier, too (maybe A!)

}

Claim: n will be expanded before B

1.

f(n) is less or equal to f(A)

Definition of f-cost Admissibility of h

…

h = 0 at a goal 𝑔 𝑜 ≤ 𝑕 𝑜 + ℎ∗(𝑜)

𝑕 𝐵 = 𝑕 𝑜 + ℎ∗(𝑜)

33

Optimality of A* Tree Search: Blocking

Proof:

}

Imagine B is on the frontier

}

Some ancestor n of A is on the frontier, too (maybe A!)

}

Claim: n will be expanded before B

1.

f(n) is less or equal to f(A)

2.

f(A) is less than f(B)

B is suboptimal h = 0 at a goal

… 34

Optimality of A* Tree Search: Blocking

Proof:

}

Imagine B is on the frontier

}

Some ancestor n of A is on the frontier, too (maybe A!)

}

Claim: n will be expanded before B

1.

f(n) is less or equal to f(A)

2.

f(A) is less than f(B)

3.

n expands before B

}

All ancestors of A expand before B

}

A expands before B

}

A* search is optimal

… 35

A* search

36

} Combines

advantages

• f

uniform-cost and greedy searches

} A* can be complete and optimal when ℎ(𝑜) has some

properties

A* search: example

37

A* search: example

38

A* search: example

39

A* search: example

40

A* search: example

41

A* search: example

42

Graph Search

43

} Failure to detect repeated states can cause exponentially more work.

Search Tree State Graph

Tree Search: Extra Work!

44

Recall: Graph Search

}

Idea: never expand a state twice

}

How to implement:

}

Tree search + set of expanded states (“closed set”)

}

Expand the search tree node-by-node, but…

}

Before expanding a node, check to make sure its state has never been expanded before

}

If not new, skip it, if new add to closed set

}

Important: store the closed set as a set, not a list

}

Can graph search wreck completeness? Why/why not?

}

How about optimality?

46

Optimality of A* Graph Search

47

A* Graph Search Gone Wrong?

S A B C G

1 1 1 2 3 h=2 h=1 h=4 h=1 h=0

S (0+2) A (1+4) B (1+1) C (2+1) G (5+0) C (3+1) G (6+0)

State space graph Search tree

48

Conditions for optimality of A*

} Admissibility: ℎ(𝑜) be a lower bound on the cost to reach goal

} Condition for optimality of TREE-SEARCH version of A*

} Consistency (monotonicity): ℎ 𝑜 ≤ 𝑑 𝑜, 𝑏, 𝑜8 + ℎ 𝑜8

} Condition for optimality of GRAPH-SEARCH version of A*

49

Consistent heuristics

50

} Triangle inequality

𝑜 𝑜′ 𝐻 ℎ(𝑜′) ℎ(𝑜) 𝑑(𝑜, 𝑏, 𝑜′) 𝑑 𝑜, 𝑏, 𝑜8 : cost of generating 𝑜′ by applying action to 𝑜 for every node 𝑜 and every successor 𝑜8 generated by any action 𝑏 ℎ 𝑜 ≤ 𝑑 𝑜, 𝑏, 𝑜8 + ℎ 𝑜8

Consistency of Heuristics

} Main idea: estimated heuristic costs ≤ actual costs

}

Admissibility: heuristic cost ≤ actual cost to goal h(A) ≤ actual cost from A to G

}

Consistency: heuristic “arc” cost ≤ actual cost for each arc h(A) – h(C) ≤ cost(A to C)

} Consequences of consistency:

}

The f value along a path never decreases h(A) ≤ cost(A to C) + h(C)

}

A* graph search is optimal

3

A C G

h=4 h=1 1 h=2

51

Admissible but not consistent: Example

52

} 𝑔 (for admissible heuristic) may decrease along a path } Is there any way to make ℎ consistent?

𝑜 𝑜′ 𝑑(𝑜, 𝑏, 𝑜’) = 1 ℎ(𝑜) = 9 ℎ(𝑜’) = 6 ⟹ ℎ 𝑜 ≰ ℎ 𝑜’ + 𝑑(𝑜, 𝑏, 𝑜’) 1 𝑕 𝑜 = 5 ℎ 𝑜 = 9 𝑔(𝑜) = 14 𝑕 𝑜′ = 6 ℎ 𝑜8 = 6 𝑔(𝑜′) = 12 G 10 10

ℎ D 𝑜′ = max (ℎ 𝑜8 , ℎ D 𝑜 − 𝑑(𝑜, 𝑏, 𝑜′))

Consistency implies admissibility

53

} Consistency ⇒ Admissblity

} All consistent heuristic functions are admissible } Nonetheless, most admissible heuristics are also consistent

ℎ 𝑜J ≤ 𝑑 𝑜J, 𝑏J, 𝑜K + ℎ(𝑜K) ≤ 𝑑 𝑜J, 𝑏J, 𝑜K + 𝑑 𝑜K, 𝑏K, 𝑜L + ℎ(𝑜L) … ≤ ∑ 𝑑 𝑜N, 𝑏N, 𝑜NOJ

P NQJ

+ ℎ(G) 𝑜J 𝑜K 𝑜L 𝑜P 𝐻 … 𝑑(𝑜J, 𝑏J, 𝑜K) 𝑑(𝑜P, 𝑏P, 𝐻) 𝑑(𝑜K, 𝑏K, 𝑜L) ⇒ ℎ 𝑜J ≤ cost of (every) path from 𝑜J to goal ≤ cost of optimal path from 𝑜J to goal

Optimality

} Tree search:

}

A* is optimal if heuristic is admissible

}

UCS is a special case (h = 0) } Graph search:

}

A* optimal if heuristic is consistent

}

UCS optimal (h = 0 is consistent) } Consistency implies admissibility } In general, most natural admissible heuristics tend to be

consistent, especially if from relaxed problems

54

Optimality of A* Graph Search

} Sketch: consider what A* does with a consistent heuristic:

}

Fact 1: In graph search, A* expands nodes in increasing total f value (f-contours)

}

Fact 2: For every state s, nodes that reach s optimally are expanded before nodes that reach s suboptimally

}

Result: A* graph search is optimal

… f £ 3 f £ 2 f £ 1 55

Optimality of A* (consistent heuristics)

Theorem: If ℎ(𝑜) is consistent, A* using GRAPH-SEARCH is

• ptimal

Lemma1: if ℎ(𝑜) is consistent then 𝑔(𝑜) values are non- decreasing along any path

Proof: Let 𝑜′ be a successor of 𝑜

I.

𝑔(𝑜′) = 𝑕(𝑜′) + ℎ(𝑜′)

II.

𝑕(𝑜′) = 𝑕(𝑜) + 𝑑(𝑜, 𝑏, 𝑜′)

III.

𝐽, 𝐽𝐽 ⇒ 𝑔 𝑜8 = 𝑕 𝑜 + 𝑑 𝑜, 𝑏, 𝑜8 + ℎ 𝑜8

IV.

ℎ 𝑜 is consistent ⇒ ℎ 𝑜 ≤ 𝑑 𝑜, 𝑏, 𝑜8 + ℎ 𝑜8

V.

𝐽𝐽𝐽, 𝐽𝑊 ⇒ 𝑔(𝑜′) ≥ 𝑕(𝑜) + ℎ(𝑜) = 𝑔(𝑜)

56

Optimality of A* (consistent heuristics)

Lemma 2: If A* selects a node 𝑜 for expansion, the optimal path to that node has been found.

Proof by contradiction: Another frontier node 𝑜8 must exist on the optimal path from initial node to 𝑜 (using graph separation property). Moreover, based

• n Lemma 1, 𝑔 𝑜8 ≤ 𝑔 𝑜 and thus 𝑜8 would have been selected first.

Lemma 1 & 2⇒ The sequence of nodes expanded by A* (using GRAPH- SEARCH) is in non-decreasing order of 𝑔(𝑜) Since ℎ = 0 for goal nodes, the first selected goal node for expansion is an

• ptimal solution (𝑔 is the true cost for goal nodes)

57

Admissible vs. consistent (tree vs. graph search)

58

} Consistent heuristic: When selecting a node for expansion, the

path with the lowest cost to that node has been found

} When an admissible heuristic is not consistent, a node will

need repeated expansion, every time a new best (so-far) cost is achieved for it.

Contours in the state space

} A* (using GRAPH-SEARCH) expands nodes in order of

increasing 𝑔 value

} Gradually adds "f-contours" of nodes

}

Contour 𝑗 has all nodes with 𝑔 = 𝑔

N where𝑔 N < 𝑔 N+1

A* expands all nodes with f(n) < C* A* expands some nodes with f(n) = C* (nodes on the goal contour) A* expands no nodes with f(n) > C* ⟹ pruning

59

Properties of A*

… b … b

Uniform-Cost A*

60

A* Example

61

Example

62

UCS Greedy A*

UCS vs A* Contours

} Uniform-cost (A*

using ℎ(𝑜)=0) expands equally in all “directions”

} A* expands mainly toward the goal, but

does hedge its bets to ensure optimality

} More accurate heuristics stretched toward the goal

(more narrowly focused around the optimal path)

Start Goal Start Goal

States are points in 2-D Euclidean space. g(n)=distance from start h(n)=estimate of distance from goal

63

Comparison

Greedy Uniform Cost A*

64

Properties of A*

} Complete?

}

Yes if nodes with 𝑔 ≤ 𝑔 𝐻 = 𝐷∗ are finite

} Step cost≥ 𝜁 > 0 and 𝑐 is finite

} Time?

}

Exponential

} But, with a smaller branching factor

¨ 𝑐d∗ed or when equal step costs 𝑐f×h∗ih

h∗

} Polynomial when |ℎ(𝑦) − ℎ∗(𝑦)| = 𝑃(𝑚𝑝𝑕 ℎ∗(𝑦))

}

However,A* is optimally efficient for any given consistent heuristic

} No optimal algorithm of this type is guaranteed to expand fewer nodes than A* (except

to node with 𝑔 = 𝐷∗)

} Space?

}

Keeps all leaf and/or explored nodes in memory

} Optimal?

}

Yes (expanding node in non-decreasing order of 𝑔)

65

Robot navigation example

66

} Initial state? Red cell } States? Cells on rectangular grid (except to obstacle) } Actions? Move to one of 8 neighbors (if it is not obstacle) } Goal test? Green cell } Path cost? Action cost is the Euclidean length of movement

A* vs. UCS: Robot navigation example

67

} Heuristic: Euclidean distance to goal } Expanded nodes: filled circles in red & green

} Color indicating 𝑕 value (red: lower, green: higher)

} Frontier: empty nodes with blue boundary } Nodes falling inside the obstacle are discarded

Adopted from: http://en.wikipedia.org/wiki/Talk%3AA*_search_algorithm

Robot navigation: Admissible heuristic

68

} Is Manhattan 𝑒o 𝑦, 𝑧 = 𝑦J − 𝑧J + 𝑦K − 𝑧K

distance an admissible heuristic for previous example?

A*: inadmissible heuristic

69

ℎ = ℎ_𝑇𝑀𝐸 ℎ = 5 ∗ ℎ_𝑇𝑀𝐸

Adopted from: http://en.wikipedia.org/wiki/Talk%3AA*_search_algorithm

A*: Summary

70

A*: Summary

} A* uses both backward costs and (estimates of) forward

costs

} A* is optimal with admissible / consistent heuristics } Heuristic design is key: often use relaxed problems

71

Creating Heuristics

72

Creating Admissible Heuristics

} Most of the work in solving hard search problems optimally is in

coming up with admissible heuristics

} Often, admissible heuristics are solutions to relaxed problems, where

new actions are available

} Inadmissible heuristics are often useful too

15 366

73

8 Puzzle: Heuristic h1

} Heuristic: Number of tiles misplaced } Why is it admissible? } h1(start) = 8

Average nodes expanded when the optimal path has… …4 steps …8 steps …12 steps UCS 112 6,300 3.6 x 106 A*(h1) 13 39 227

Start State Goal State

Statistics from Andrew Moore 74

Admissible heuristics: 8-puzzle

} ℎ1(𝑜) = number of misplaced tiles } ℎ2(𝑜) = sum of Manhattan distance of tiles from their target position

}

i.e., no. of squares from desired location of each tile

} ℎ1(𝑇) = } ℎ2(𝑇) =

8 3+1+2+2+2+3+3+2 = 18

75

8 Puzzle: Heuristic h2

}

What if we had an easier 8-puzzle where any tile could slide any direction at any time, ignoring other tiles?

}

Total Manhattan distance

}

Why is it admissible?

}

h2(start) = 3 + 1 + 2 + … = 18 Average nodes expanded when the optimal path has… …4 steps …8 steps …12 steps A*(h1) 13 39 227 A*(h2) 12 25 73

Start State Goal State

76

8 Puzzle: heuristic

} How about using the actual cost as a heuristic?

}

Would it be admissible?

}

Would we save on nodes expanded?

}

What’s wrong with it?

} With A*: a trade-off between quality of estimate and work per node

}

As heuristics get closer to the true cost, you will expand fewer nodes but usually do more work per node to compute the heuristic itself

77

Effect of heuristic on accuracy

} 𝑂: number of generated nodes by A* } 𝑒: solution depth } Effective branching factor 𝑐∗ : branching factor of a

uniform tree of depth 𝑒 containing 𝑂 + 1 nodes. 𝑂 + 1 = 1 + 𝑐∗ + (𝑐∗)K+ ⋯ + (𝑐∗)f

} Well-defined heuristic: 𝑐∗ is close to one

78

Comparison on 8-puzzle

𝑒 IDS A*(𝒊𝟐) A*(𝒊𝟑) 6 680 20 18 12 3644035 227 73 24

• 39135

1641 𝑒 IDS A*(𝒊𝟐) A*(𝒊𝟑) 6 2.87 1.34 1.30 12 2.78 1.42 1.24 24

• 1.48

1.26 Search Cost (𝑂) Effective branching factor (𝑐∗)

79

Heuristic quality

If ∀𝑜, ℎ2(𝑜) ≥ ℎ1(𝑜) (both admissible) then ℎ2 dominates ℎ1 and it is better for search

} Surely expanded nodes: 𝑔 𝑜 < 𝐷∗ ⇒ ℎ 𝑜 < 𝐷∗ − 𝑕 𝑜

} If ℎ2(𝑜) ≥ ℎ1(𝑜) then every node expanded for ℎ2 will also be surely

expanded with ℎ1 (ℎ1 may also causes some more node expansion)

80

More accurate heuristic

81

} Max of admissible heuristics is admissible (while it is a

more accurate estimate)

} How about using the actual cost as a heuristic?

} ℎ(𝑜) = ℎ∗(𝑜) for all 𝑜

} Will go straight to the goal ?!

} Trade of between accuracy and computation time

ℎ 𝑜 = max (ℎJ 𝑜 , ℎK 𝑜 )

Trivial Heuristics, Dominance

} Dominance: ha ≥ hc if } Heuristics form a semi-lattice:

}

Max of admissible heuristics is admissible

} Trivial heuristics

}

Bottom of lattice is the zero heuristic (what does this give us?)

}

T

• p of lattice is the exact heuristic

82

Generating heuristics

} Relaxed problems

} Inventing admissible heuristics automatically

} Sub-problems (pattern databases) } Learning heuristics from experience

83

Relaxed problem

84

} Relaxed problem: Problem with fewer restrictions on the

actions

} Optimal solution to the relaxed problem may be computed

easily (without search)

} The cost of an optimal solution to a relaxed problem is an

admissible heuristic for the original problem

} The optimal solution is the shortest path in the super-graph of the state-

space.

Relaxed problem: 8-puzzle

} 8-Puzzle: move a tile from square A to B if A is adjacent (left,

right, above, below) to B and B is blank

} Relaxed problems

1) can move from A to B if A is adjacent to B (ignore whether or not position is

blank)

2) can move from A to B if B is blank (ignore adjacency) 3) can move from A to B (ignore both conditions)

} Admissible heuristics for original problem (ℎ1(𝑜) and ℎ2(𝑜))

are optimal path costs for relaxed problems

} First case: a tile can move to any adjacent square ⇒ ℎ2(𝑜) } Third case: a tile can move anywhere ⇒ ℎ1(𝑜)

85

Sub-problem heuristic

} The cost to solve a sub-problem

} Store exact solution costs for every possible sub-problem

} Admissible?

} The cost of the optimal solution to this problem is a lower

bound on the cost of the complete problem

86

Pattern databases heuristics

} Storing the exact solution cost for every possible sub-

problem instance

} Combination (taking maximum) of heuristics resulted by

different sub-problems

} 15-Puzzle: 103 times reduction in no. of generated nodes vs. ℎ2

87

Disjoint pattern databases

} Adding these pattern-database heuristics yields an admissible

heuristic?!

} Dividing up the problem such that each move affects only one

sub-problem (disjoint sub-problems) and then adding heuristics

} 15-puzzle: 104 times reduction in no. of generated nodes vs. ℎ2 }

24-Puzzle: 106 times reduction in no. of generated nodes vs. ℎ2

} Can Rubik’s cube be divided up to disjoint sub-problems?

88

Learning heuristics from experience

} Machine Learning Techniques

} Learn ℎ(𝑜)

from samples of optimally solved problems (predicting solution cost for other states)

} Features of state (instead of raw state description)

} 8-puzzle

} number of misplaced tiles } number of adjacent pairs of tiles that are not adjacent in the goal state

} Linear Combination of features

89