Informed Search A* Algorithm CE417: Introduction to Artificial - - PowerPoint PPT Presentation

CE417: Introduction to Artificial Intelligence Sharif University of Technology Spring 2018 “Artificial Intelligence: A Modern Approach”, Chapter 3 Most slides have been adopted from Klein and Abdeel, CS188, UC Berkeley.

Informed Search A* Algorithm

Soleymani

Outline

 Heuristics  Greedy (best-first) search  A* search  Finding heuristics

2

Uninformed Search

Uniform Cost Search

 Strategy: expand lowest path cost  The good: UCS is complete and optimal!  The bad:

 Explores options in every “direction”  No information about goal location Start Goal … c  3 c  2 c  1

UCS Example

5

Informed Search

Search Heuristics

• A heuristic is:
• A function that estimates how close a state is to a goal
• Designed for a particular search problem
• Examples: Manhattan distance, Euclidean distance for pathing

10 5 11.2

Heuristic Function

 Incorporating problem-specific knowledge in search

 Information more than problem definition  In order to come to an optimal solution as rapidly as possible

 ℎ 𝑜 : estimated cost of cheapest path from 𝑜 to a goal

 Depends only on 𝑜 (not path from root to 𝑜)  If 𝑜 is a goal state then ℎ(𝑜)=0  ℎ(𝑜) ≥ 0

 Examples of heuristic functions include using a rule-of-thumb,

an educated guess, or an intuitive judgment

8

Example: Heuristic Function

h(x)

Greedy Search

Greedy search

 Priority queue based on ℎ(𝑜)

 e.g., ℎ𝑇𝑀𝐸 𝑜 = straight-line distance from n to Bucharest

 Greedy search expands the node that appears to be

closest to goal

Greedy

11

Example: Heuristic Function

h(x)

Romania with step costs in km

13

Greedy best-first search example

14

Greedy best-first search example

15

Greedy best-first search example

16

Greedy best-first search example

17

Greedy Search

 Expand the node that seems closest…  What can go wrong?

Greedy Search

 Strategy: expand a node that you think is closest to a

goal state

 Heuristic: estimate of distance to nearest goal for each state

 A common case:

 Best-first takes you straight to the (wrong) goal

 Worst-case: like a badly-guided DFS

… b … b

Properties of greedy best-first search

 Complete? No

 Similar to DFS, only graph search version is complete in finite spaces  Infinite loops, e.g., (Iasi to Fagaras) Iasi  Neamt  Iasi  Neamt

 Time

 𝑃(𝑐𝑛), but a good heuristic can give dramatic improvement

 Space

 𝑃(𝑐𝑛): keeps all nodes in memory

 Optimal? No

20

Greedy Search

21

A* Search

A* search

 Idea: minimizing the total estimated solution cost  Evaluation function for priority 𝑔 𝑜 = 𝑕 𝑜 + ℎ(𝑜)

 𝑕 𝑜 = cost so far to reach 𝑜  ℎ 𝑜 = estimated cost of the cheapest path from 𝑜 to goal  So, 𝑔 𝑜 = estimated total cost of path through 𝑜 to goal

24

start n … goal … Actual cost 𝑕 𝑜 Estimated cost ℎ 𝑜 𝑔 𝑜 = 𝑕 𝑜 + ℎ(𝑜)

Combining UCS and Greedy



Uniform-cost orders by path cost, or backward cost g(n)



Greedy orders by goal proximity, or forward cost h(n)



A* Search orders by the sum: f(n) = g(n) + h(n)

S a d b G h=5 h=6 h=2 1 8 1 1 2 h=6 h=0 c h=7 3 e h=1 1 Example: Teg Grenager S a b c e d d G G g = 0 h=6 g = 1 h=5 g = 2 h=6 g = 3 h=7 g = 4 h=2 g = 6 h=0 g = 9 h=1 g = 10 h=2 g = 12 h=0

When should A* terminate?

 Should we stop when we enqueue a goal?  No: only stop when we dequeue a goal

S B A G 2 3 2 2

h = 1 h = 2 h = 0 h = 3

Is A* Optimal?

 What went wrong?  Actual bad goal cost < estimated good goal cost  We need estimates to be less than actual costs!

A G S 1 3

h = 6 h = 0

5

h = 7

Admissible Heuristics

Idea: Admissibility

Inadmissible (pessimistic) heuristics break

• ptimality by trapping good plans on the frontier

Admissible (optimistic) heuristics slow down bad plans but never outweigh true costs

Admissible Heuristics

 A heuristic h is admissible (optimistic) if:

where is the true cost to a nearest goal

 Examples:  Coming up with admissible heuristics is most of what’s

involved in using A* in practice.

15

Optimality of A* Tree Search

Optimality of A* Tree Search

Assume:

 A is an optimal goal node  B is a suboptimal goal node  h is admissible

Claim:

 A will exit the frontier before B …

Optimality of A* Tree Search: Blocking

Proof:

 Imagine B is on the frontier  Some ancestor n of A is on the

frontier, too (maybe A!)

 Claim: n will be expanded before B 1.

f(n) is less or equal to f(A)

Definition of f-cost Admissibility of h

…

h = 0 at a goal

𝑔 𝑜 ≤ 𝑕 𝑜 + ℎ∗(𝑜)

𝑕 𝐵 = 𝑕 𝑜 + ℎ∗(𝑜)

Optimality of A* Tree Search: Blocking

Proof:

 Imagine B is on the frontier  Some ancestor n of A is on the

frontier, too (maybe A!)

 Claim: n will be expanded before B 1.

f(n) is less or equal to f(A)

2.

f(A) is less than f(B)

B is suboptimal h = 0 at a goal

…

Optimality of A* Tree Search: Blocking

Proof:



Imagine B is on the frontier



Some ancestor n of A is on the frontier, too (maybe A!)



Claim: n will be expanded before B

1.

f(n) is less or equal to f(A)

2.

f(A) is less than f(B)

3.

n expands before B



All ancestors of A expand before B



A expands before B



A* search is optimal

…

A* search

36

 Combines

advantages

• f

uniform-cost and greedy searches

 A* can be complete and optimal when ℎ(𝑜) has some

properties

A* search: example

37

A* search: example

38

A* search: example

39

A* search: example

40

A* search: example

41

A* search: example

42

Properties of A*

… b … b

Uniform-Cost A*

A* Example

44

Example

45

UCS Greedy A*

UCS vs A* Contours

 Uniform-cost (A*

using ℎ(𝑜)=0) expands equally in all “directions”

 A* expands mainly toward the goal, but

does hedge its bets to ensure optimality

 More accurate heuristics stretched toward the goal

(more narrowly focused around the optimal path)

Start Goal Start Goal

States are points in 2-D Euclidean space. g(n)=distance from start h(n)=estimate of distance from goal

Comparison

Greedy Uniform Cost A*

Graph Search

 Failure to detect repeated states can cause exponentially more work.

Search Tree State Graph

Tree Search: Extra Work!

Graph Search

 In BFS, for example, we shouldn’t bother expanding the circled nodes (why?)

S

a b d p a c e p h f r q q c

G

a q e p h f r q q c

G

a

Recall: Graph Search

 Idea: never expand a state twice  How to implement:



Tree search + set of expanded states (“closed set”)



Expand the search tree node-by-node, but…



Before expanding a node, check to make sure its state has never been expanded before



If not new, skip it, if new add to closed set

 Important: store the closed set as a set, not a list  Can graph search wreck completeness? Why/why not?  How about optimality?

Optimality of A* Graph Search

A* Graph Search Gone Wrong?

S A B C G

1 1 1 2 3 h=2 h=1 h=4 h=1 h=0

S (0+2) A (1+4) B (1+1) C (2+1) G (5+0) C (3+1) G (6+0)

State space graph Search tree

Conditions for optimality of A*

 Admissibility: ℎ(𝑜) be a lower bound on the cost to reach goal

 Condition for optimality of TREE-SEARCH version of A*

 Consistency (monotonicity): ℎ 𝑜 ≤ 𝑑 𝑜, 𝑏, 𝑜′ + ℎ 𝑜′

 Condition for optimality of GRAPH-SEARCH version of A*

54

Consistent heuristics

55

 Triangle inequality

𝑜 𝑜′ 𝐻 ℎ(𝑜′) ℎ(𝑜) 𝑑(𝑜, 𝑏, 𝑜′) 𝑑 𝑜, 𝑏, 𝑜′ : cost of generating 𝑜′ by applying action to 𝑜 for every node 𝑜 and every successor 𝑜′ generated by any action 𝑏 ℎ 𝑜 ≤ 𝑑 𝑜, 𝑏, 𝑜′ + ℎ 𝑜′

Consistency of Heuristics

 Main idea: estimated heuristic costs ≤ actual costs



Admissibility: heuristic cost ≤ actual cost to goal h(A) ≤ actual cost from A to G



Consistency: heuristic “arc” cost ≤ actual cost for each arc h(A) – h(C) ≤ cost(A to C)

 Consequences of consistency:



The f value along a path never decreases h(A) ≤ cost(A to C) + h(C)



A* graph search is optimal

3

A C G

h=4 h=1 1 h=2

Optimality

 Tree search:

 A* is optimal if heuristic is admissible  UCS is a special case (h = 0)

 Graph search:

 A* optimal if heuristic is consistent  UCS optimal (h = 0 is consistent)

 Consistency implies admissibility  In general, most natural admissible heuristics tend to be

consistent, especially if from relaxed problems

Consistency implies admissibility

58

 Consistency ⇒ Admissblity

 All consistent heuristic functions are admissible  Nonetheless, most admissible heuristics are also consistent

ℎ 𝑜1 ≤ 𝑑 𝑜1, 𝑏1, 𝑜2 + ℎ(𝑜2) ≤ 𝑑 𝑜1, 𝑏1, 𝑜2 + 𝑑 𝑜2, 𝑏2, 𝑜3 + ℎ(𝑜3) … ≤ 𝑗=1

𝑙

𝑑 𝑜𝑗, 𝑏𝑗, 𝑜𝑗+1 + ℎ(G) 𝑜1 𝑜2 𝑜3 𝑜𝑙 𝐻 … 𝑑(𝑜1, 𝑏1, 𝑜2) 𝑑(𝑜𝑙, 𝑏𝑙, 𝐻) 𝑑(𝑜2, 𝑏2, 𝑜3) ⇒ ℎ 𝑜1 ≤ cost of (every) path from 𝑜1 to goal ≤ cost of optimal path from 𝑜1 to goal

Admissible but not consistent: Example

59

 𝑔 (for admissible heuristic) may decrease along a path  Is there any way to make ℎ consistent?

𝑜 𝑜′ 𝑑(𝑜, 𝑏, 𝑜’) = 1 ℎ(𝑜) = 9 ℎ(𝑜’) = 6 ⟹ ℎ 𝑜 ≰ ℎ 𝑜’ + 𝑑(𝑜, 𝑏, 𝑜’) 1 𝑕 𝑜 = 5 ℎ 𝑜 = 9 𝑔(𝑜) = 14 𝑕 𝑜′ = 6 ℎ 𝑜′ = 6 𝑔(𝑜′) = 12 G 10 10

ℎ 𝑜′ = max(ℎ 𝑜′ , ℎ 𝑜 − 𝑑(𝑜, 𝑏, 𝑜′))

Optimality of A* Graph Search

 Sketch: consider what A* does with a consistent heuristic:

 Fact 1: In graph search, A* expands nodes in increasing total f value (f-contours)  Fact 2: For every state s, nodes that reach s optimally are expanded before nodes

that reach s suboptimally

 Result: A* graph search is optimal … f  3 f  2 f  1

Optimality of A* (consistent heuristics)

Theorem: If ℎ(𝑜) is consistent, A* using GRAPH-SEARCH is

• ptimal

Lemma1: if ℎ(𝑜) is consistent then 𝑔(𝑜) values are non- decreasing along any path

Proof: Let 𝑜′ be a successor of 𝑜

I.

𝑔(𝑜′) = 𝑕(𝑜′) + ℎ(𝑜′)

II.

𝑕(𝑜′) = 𝑕(𝑜) + 𝑑(𝑜, 𝑏, 𝑜′)

III.

𝐽, 𝐽𝐽 ⇒ 𝑔 𝑜′ = 𝑕 𝑜 + 𝑑 𝑜, 𝑏, 𝑜′ + ℎ 𝑜′

IV.

ℎ 𝑜 is consistent ⇒ ℎ 𝑜 ≤ 𝑑 𝑜, 𝑏, 𝑜′ + ℎ 𝑜′

V.

𝐽𝐽𝐽, 𝐽𝑊 ⇒ 𝑔(𝑜′) ≥ 𝑕(𝑜) + ℎ(𝑜) = 𝑔(𝑜)

61

Optimality of A* (consistent heuristics)

Lemma 2: If A* selects a node 𝑜 for expansion, the optimal path to that node has been found.

Proof by contradiction: Another frontier node 𝑜′ must exist on the optimal path from initial node to 𝑜 (using graph separation property). Moreover, based

• n Lemma 1, 𝑔 𝑜′ ≤ 𝑔 𝑜 and thus 𝑜′ would have been selected first.

Lemma 1 & 2⇒ The sequence of nodes expanded by A* (using GRAPH- SEARCH) is in non-decreasing order of 𝑔(𝑜) Since ℎ = 0 for goal nodes, the first selected goal node for expansion is an

• ptimal solution (𝑔 is the true cost for goal nodes)

62

Admissible vs. consistent (tree vs. graph search)

63

 Consistent heuristic: When selecting a node for expansion, the

path with the lowest cost to that node has been found

 When an admissible heuristic is not consistent, a node will

need repeated expansion, every time a new best (so-far) cost is achieved for it.

Contours in the state space

 A* (using GRAPH-SEARCH) expands nodes in order of

increasing 𝑔 value

 Gradually adds "f-contours" of nodes

 Contour 𝑗 has all nodes with 𝑔 = 𝑔 𝑗 where𝑔 𝑗 < 𝑔 𝑗+1

A* expands all nodes with f(n) < C* A* expands some nodes with f(n) = C* (nodes on the goal contour) A* expands no nodes with f(n) > C* ⟹ pruning

64

Properties of A*

 Complete?

 Yes if nodes with 𝑔 ≤ 𝑔 𝐻 = 𝐷∗ are finite

 Step cost≥ 𝜁 > 0 and 𝑐 is finite

 Time?

 Exponential

 But, with a smaller branching factor

 𝑐ℎ∗−ℎ or when equal step costs 𝑐𝑒×ℎ∗−ℎ

ℎ∗

 Polynomial when |ℎ(𝑦) − ℎ∗(𝑦)| = 𝑃(𝑚𝑝𝑕 ℎ∗(𝑦))

 However,A* is optimally efficient for any given consistent heuristic

 No optimal algorithm of this type is guaranteed to expand fewer nodes than A* (except

to node with 𝑔 = 𝐷∗)

 Space?

 Keeps all leaf and/or explored nodes in memory

 Optimal?

 Yes (expanding node in non-decreasing order of 𝑔) 65

Robot navigation example

66

 Initial state? Red cell  States? Cells on rectangular grid (except to obstacle)  Actions? Move to one of 8 neighbors (if it is not obstacle)  Goal test? Green cell  Path cost? Action cost is the Euclidean length of movement

A* vs. UCS: Robot navigation example

67

 Heuristic: Euclidean distance to goal  Expanded nodes: filled circles in red & green

 Color indicating 𝑕 value (red: lower, green: higher)

 Frontier: empty nodes with blue boundary  Nodes falling inside the obstacle are discarded

Adopted from: http://en.wikipedia.org/wiki/Talk%3AA*_search_algorithm

Robot navigation: Admissible heuristic

68

 Is Manhattan 𝑒𝑁 𝑦, 𝑧 = 𝑦1 − 𝑧1 + 𝑦2 − 𝑧2

distance an admissible heuristic for previous example?

A*: inadmissible heuristic

69

ℎ = ℎ_𝑇𝑀𝐸 ℎ = 5 ∗ ℎ_𝑇𝑀𝐸

Adopted from: http://en.wikipedia.org/wiki/Talk%3AA*_search_algorithm

A*: Summary

A*: Summary

 A* uses both backward costs and (estimates of) forward

costs

 A* is optimal with admissible / consistent heuristics  Heuristic design is key: often use relaxed problems

Creating Heuristics

Creating Admissible Heuristics

 Most of the work in solving hard search problems optimally is in

coming up with admissible heuristics

 Often, admissible heuristics are solutions to relaxed problems, where

new actions are available

 Inadmissible heuristics are often useful too

15 366

Example: 8 Puzzle



What are the states?



How many states?



What are the actions?



How many successors from the start state?



What should the costs be?

Start State Goal State Actions

8 Puzzle: Heuristic h1

 Heuristic: Number of tiles misplaced  Why is it admissible?  h1(start) = 8

Average nodes expanded when the optimal path has… …4 steps …8 steps …12 steps UCS 112 6,300 3.6 x 106 A*(h1) 13 39 227

Start State Goal State

Statistics from Andrew Moore

Admissible heuristics: 8-puzzle

 ℎ1(𝑜) = number of misplaced tiles  ℎ2(𝑜) = sum of Manhattan distance of tiles from their target position



i.e., no. of squares from desired location of each tile

 ℎ1(𝑇) =  ℎ2(𝑇) =

8 3+1+2+2+2+3+3+2 = 18

76

8 Puzzle: Heuristic h2

 What if we had an easier 8-puzzle where

any tile could slide any direction at any time, ignoring other tiles?

 Total Manhattan distance  Why is it admissible?  h2(start) = 3 + 1 + 2 + … = 18

Average nodes expanded when the optimal path has… …4 steps …8 steps …12 steps A*(h1) 13 39 227 A*(h2) 12 25 73

Start State Goal State

8 Puzzle: heuristic

 How about using the actual cost as a heuristic?

 Would it be admissible?  Would we save on nodes expanded?  What’s wrong with it?

 With A*: a trade-off between quality of estimate and work per node

 As heuristics get closer to the true cost, you will expand fewer nodes but usually

do more work per node to compute the heuristic itself

Effect of heuristic on accuracy

 𝑂: number of generated nodes by A*  𝑒: solution depth  Effective branching factor 𝑐∗ : branching factor of a

uniform tree of depth 𝑒 containing 𝑂 + 1 nodes. 𝑂 + 1 = 1 + 𝑐∗ + (𝑐∗)2+ ⋯ + (𝑐∗)𝑒

 Well-defined heuristic: 𝑐∗ is close to one

79

Comparison on 8-puzzle

𝑒 IDS A*(𝒊𝟐) A*(𝒊𝟑) 6 680 20 18 12 3644035 227 73 24

• 39135

1641 𝑒 IDS A*(𝒊𝟐) A*(𝒊𝟑) 6 2.87 1.34 1.30 12 2.78 1.42 1.24 24

• 1.48

1.26 Search Cost (𝑂) Effective branching factor (𝑐∗)

80

Heuristic quality

If ∀𝑜, ℎ2(𝑜) ≥ ℎ1(𝑜) (both admissible) then ℎ2 dominates ℎ1 and it is better for search

 Surely expanded nodes: 𝑔 𝑜 < 𝐷∗ ⇒ ℎ 𝑜 < 𝐷∗ − 𝑕 𝑜

 If ℎ2(𝑜) ≥ ℎ1(𝑜) then every node expanded for ℎ2 will also be surely

expanded with ℎ1 (ℎ1 may also causes some more node expansion)

81

More accurate heuristic

82

 Max of admissible heuristics is admissible (while it is a

more accurate estimate)

 How about using the actual cost as a heuristic?

 ℎ(𝑜) = ℎ∗(𝑜) for all 𝑜

 Will go straight to the goal ?!

 Trade of between accuracy and computation time

ℎ 𝑜 = max(ℎ1 𝑜 , ℎ2 𝑜 )

Trivial Heuristics, Dominance

 Dominance: ha ≥ hc if  Heuristics form a semi-lattice:



Max of admissible heuristics is admissible

 Trivial heuristics



Bottom of lattice is the zero heuristic (what does this give us?)



T

• p of lattice is the exact heuristic

Generating heuristics

 Relaxed problems

 Inventing admissible heuristics automatically

 Sub-problems (pattern databases)  Learning heuristics from experience

84

Relaxed problem

85

 Relaxed problem: Problem with fewer restrictions on the

actions

 Optimal solution to the relaxed problem may be computed

easily (without search)

 The cost of an optimal solution to a relaxed problem is an

admissible heuristic for the original problem

 The optimal solution is the shortest path in the super-graph of the state-

space.

Relaxed problem: 8-puzzle

 8-Puzzle: move a tile from square A to B if A is adjacent (left,

right, above, below) to B and B is blank

 Relaxed problems

1) can move from A to B if A is adjacent to B (ignore whether or not position is

blank)

2) can move from A to B if B is blank (ignore adjacency) 3) can move from A to B (ignore both conditions)

 Admissible heuristics for original problem (ℎ1(𝑜) and ℎ2(𝑜))

are optimal path costs for relaxed problems

 First case: a tile can move to any adjacent square ⇒ ℎ2(𝑜)  Third case: a tile can move anywhere ⇒ ℎ1(𝑜)

86

Sub-problem heuristic

 The cost to solve a sub-problem

 Store exact solution costs for every possible sub-problem

 Admissible?

 The cost of the optimal solution to this problem is a lower

bound on the cost of the complete problem

87

Pattern databases heuristics

 Storing the exact solution cost for every possible sub-

problem instance

 Combination (taking maximum) of heuristics resulted by

different sub-problems

 15-Puzzle: 103 times reduction in no. of generated nodes vs. ℎ2

88

Disjoint pattern databases

 Adding these pattern-database heuristics yields an admissible

heuristic?!

 Dividing up the problem such that each move affects only one

sub-problem (disjoint sub-problems) and then adding heuristics

 15-puzzle: 104 times reduction in no. of generated nodes vs. ℎ2 

24-Puzzle: 106 times reduction in no. of generated nodes vs. ℎ2

 Can Rubik’s cube be divided up to disjoint sub-problems?

89

Learning heuristics from experience

 Machine Learning Techniques

 Learn ℎ(𝑜)

from samples of optimally solved problems (predicting solution cost for other states)

 Features of state (instead of raw state description)

 8-puzzle

 number of misplaced tiles  number of adjacent pairs of tiles that are not adjacent in the goal state

 Linear Combination of features

90