Informed Search Algorithms and Beyond R&N 3.53.6, 4.1 Jacques - - PowerPoint PPT Presentation

T H E U N I V E R S I T Y O F E D I N B U R G H

Informed Search Algorithms and Beyond

R&N 3.5–3.6, 4.1 Jacques Fleuriot

University of Edinburgh, School of Informatics jdf@ed.ac.uk

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 1/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Overview

Review: General search Best-first search Greedy search A∗ search Hill-climbing Simulated annealing

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 2/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Review: General search

function GENERIC-SEARCH(problem) returns a solution, or failure frontier ← a queue initially containing one path, for the problem’s initial state reached ← a table of {state: the best path that reached state}; initially empty solution ← failure while frontier is not empty and solution can possibly be improved do parent ← some node that we choose to remove from frontier for child in successors(parent) do s ← child.state if s is not in reached or child is a cheaper path than reached[s] then reached[s] ← child add child to frontier if child is a goal and is cheaper than solution then solution = child return solution

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 3/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Best-first search

Instance of general search search strategy Idea: use an evaluation function f (n) for each node n – estimate of “desirability” ⇒ Expand most desirable unexpanded node, usually the node with the lowest evaluation Implementation: Order the nodes in frontier in decreasing order of desirability Special cases: Greedy best-first search A∗ search

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 4/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Romania with step costs in km

Bucharest Giurgiu Urziceni Hirsova Eforie Neamt Oradea Zerind Arad Timisoara Lugoj Mehadia Dobreta Craiova Sibiu Fagaras Pitesti Rimnicu Vilcea Vaslui Iasi Straight−line distance to Bucharest 160 242 161 77 151 241 366 193 178 253 329 80 199 244 380 226 234 374 98

Giurgiu Urziceni Hirsova Eforie Neamt Oradea Zerind Arad Timisoara Lugoj Mehadia Dobreta Craiova Sibiu Fagaras Pitesti Vaslui Iasi Rimnicu Vilcea Bucharest

71 75 118 111 70 75 120 151 140 99 80 97 101 211 138 146 85 90 98 142 92 87 86

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 5/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Greedy best-first search

Evaluation function h(n) (heuristic) = estimated cost of cheapest path from from state at node n to goal state Example: hSLD(n) = straight-line distance from n to Bucharest Greedy search expands the node that appears to be closest to goal

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 6/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Greedy search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 7/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Greedy search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 7/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Greedy search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 7/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Greedy search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 7/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of greedy search

Complete? Time? Space? Optimal?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 8/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of greedy search

Complete? No – can get stuck in loops, e.g. Iasi → Neamt → Iasi → Neamt → Complete in finite space with repeated-state checking Time? Space? Optimal?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 8/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of greedy search

Complete? No – can get stuck in loops, e.g. Iasi → Neamt → Iasi → Neamt → Complete in finite space with repeated-state checking Time? O(bm), but a good heuristic can give dramatic improvement Space? Optimal?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 8/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of greedy search

Complete? No – can get stuck in loops, e.g. Iasi → Neamt → Iasi → Neamt → Complete in finite space with repeated-state checking Time? O(bm), but a good heuristic can give dramatic improvement Space? O(bm)—keeps all nodes in memory Optimal?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 8/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of greedy search

Complete? No – can get stuck in loops, e.g. Iasi → Neamt → Iasi → Neamt → Complete in finite space with repeated-state checking Time? O(bm), but a good heuristic can give dramatic improvement Space? O(bm)—keeps all nodes in memory Optimal? No

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 8/26

T H E U N I V E R S I T Y O F E D I N B U R G H

A∗ search

Idea: avoid expanding paths that are already expensive Evaluation function f (n) = g(n) + h(n) g(n) = cost so far to reach n h(n) = estimated cost to goal from n f (n) = estimated total cost of path through n to goal A∗ is both complete and optimal if h(n) satisfies certain conditions

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 9/26

T H E U N I V E R S I T Y O F E D I N B U R G H

A∗ search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 10/26

T H E U N I V E R S I T Y O F E D I N B U R G H

A∗ search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 10/26

T H E U N I V E R S I T Y O F E D I N B U R G H

A∗ search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 10/26

T H E U N I V E R S I T Y O F E D I N B U R G H

A∗ search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 10/26

T H E U N I V E R S I T Y O F E D I N B U R G H

A∗ search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 10/26

T H E U N I V E R S I T Y O F E D I N B U R G H

A∗ search example

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 10/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Admissible heuristics

A heuristic h(n) is admissible if for every node n, h(n) ≤ h∗(n), where h∗(n) is the true cost to reach the goal state from n. A∗ search uses an admissible heuristic An admissible heuristic never overestimates the cost to reach the goal, i.e., it is optimistic. Thus for A∗, f (n) = g(n) + h(n) never overestimates the true cost of a solution. Example: hSLD(n) never overestimates the actual road distance. Theorem: A∗ search is optimal

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 11/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Optimality of A∗ (standard proof)

Suppose some suboptimal goal G2 has been generated and is in the

• frontier. Let n be an unexpanded node in the frontier such that n

is on a shortest path to an optimal goal G.

G n G2 Start

f (G2) = g(G2) since h(G2) = 0 > g(G) since G2 is suboptimal f (G) = g(G) since h(G) = 0 f (G2) > f (G) from above

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 12/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Optimality of A∗ (standard proof)

Suppose some suboptimal goal G2 has been generated and is in the

• frontier. Let n be an unexpanded node in the frontier such that n

is on a shortest path to an optimal goal G.

G n G2 Start

f (G2) > f (G) from above h(n) ≤ h∗(n) since h is admissible g(n) + h(n) ≤ g(n) + h∗(n) f (n) ≤ f (G)

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 12/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Optimality of A∗ (standard proof)

Suppose some suboptimal goal G2 has been generated and is in the

• frontier. Let n be an unexpanded node in the frontier such that n

is on a shortest path to an optimal goal G.

G n G2 Start

Hence f (G2) > f (n), and A∗ will never select G2 for expansion

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 12/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Consistent heuristics

A heuristic h is consistent if for every node n, every successor n′ of n generated by any action a, h(n) ≤ c(n, a, n′) + h(n′) where c(n, a, n′) is the cost from n to n′ under a. If h is consistent, we have f (n′) = g(n′) + h(n′) = g(n) + c(n, a, n′) + h(n′) ≥ g(n) + h(n) ≥ f (n) So, f (n) is non-decreasing along any path. Theorem: If h(n) is consistent, A* is optimal.

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 13/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Optimality of A∗

Lemma: A∗ expands nodes in order of increasing f value Gradually adds “f -contours” of nodes (cf. breadth-first adds layers) Contour i has all nodes with f = fi, where fi < fi+1

O Z A T L M D C R F P G B U H E V I N

380 400 420

S

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 14/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of A∗

Complete? Time? Space? Optimal?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 15/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of A∗

Complete? Yes, unless there are infinitely many n nodes with f (n) ≤ f (G) Time? Space? Optimal?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 15/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of A∗

Complete? Yes, unless there are infinitely many n nodes with f (n) ≤ f (G) Time? Exponential Space? Optimal?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 15/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of A∗

Complete? Yes, unless there are infinitely many n nodes with f (n) ≤ f (G) Time? Exponential Space? Keeps all nodes in memory Optimal?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 15/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Properties of A∗

Complete? Yes, unless there are infinitely many n nodes with f (n) ≤ f (G) Time? Exponential Space? Keeps all nodes in memory Optimal? Yes—cannot expand fi+1 until fi is finished

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 15/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Admissible heuristics

E.g., for the 8-puzzle: h1(n) = number of misplaced tiles h2(n) = total Manhattan distance (i.e., no. of squares from desired location of each tile)

Start State Goal State

2 4 5 6 7 8 1 2 3 4 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 5

h1(S) = ? h2(S) = ?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 16/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Admissible heuristics

E.g., for the 8-puzzle: h1(n) = number of misplaced tiles h2(n) = total Manhattan distance (i.e., no. of squares from desired location of each tile)

Start State Goal State

2 4 5 6 7 8 1 2 3 4 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 5

h1(S) = 7 h2(S) = ?

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 16/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Admissible heuristics

E.g., for the 8-puzzle: h1(n) = number of misplaced tiles h2(n) = total Manhattan distance (i.e., no. of squares from desired location of each tile)

Start State Goal State

2 4 5 6 7 8 1 2 3 4 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 5

h1(S) = 7 h2(S) = 2+3+3+2+4+2+0+2 = 18

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 16/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Dominance

If h2(n) ≥ h1(n) for all n (both admissible) then h2 dominates h1 and is better for search Typical search costs: d = 14 IDS = 3,473,941 nodes A∗(h1) = 539 nodes A∗(h2) = 113 nodes d = 24 IDS = too many nodes A∗(h1) = 39,135 nodes A∗(h2) = 1,641 nodes

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 17/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Relaxed problems

A problem with fewer restrictions on the actions is called a relaxed problem The cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem If the rules of the 8-puzzle are relaxed so that a tile can move anywhere, then h1(n) gives the shortest solution If the rules are relaxed so that a tile can move to any adjacent square, then h2(n) gives the shortest solution Can use relaxation to automatically generate admissible heuristics

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 18/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Iterative improvement algorithms

In many optimization problems, path is irrelevant; the goal state itself is the solution Then state space = set of “complete” configurations; find optimal configuration, e.g., TSP

• r, find configuration satisfying constraints, e.g., timetable

In such cases, can use iterative improvement algorithms; keep a single “current” state, try to improve it Constant space, suitable for online as well as offline search

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 19/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Example: Travelling Salesperson Problem

Find the shortest tour that visits each city exactly once Start with any complete tour, perform pairwise exchanges Variants of this approach get within 1% of optimal very quickly with thousands of cities

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 20/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Example: n-queens

Put n queens on an n × n board with no two queens on the same row, column, or diagonal Move a queen to reduce number of conflicts Almost always solves n-queens problems almost instantaneously for very large n

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 21/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Hill-climbing (or gradient ascent/descent) I

“Like climbing Everest in thick fog with amnesia”

function HILL-CLIMBING(problem) returns a state that is a local maximum current ← problem.INITIAL-STATE loop do neighbor ← a highest-valued successor of current if VALUE(neighbour) ≤ VALUE(current) then return current current ← neighbor

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 22/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Hill-climbing II

Problem: depending on initial state, can get stuck on local maxima

current state

• bjective function

state space global maximum local maximum “flat” local maximum shoulder

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 23/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Simulated annealing I

Idea: escape local maxima by allowing some “bad” moves but gradually decrease their size and frequency Combines hill-climbing with a random walk, where some downhill moves are allowed

function SIMULATED-ANNEALING(problem,schedule) returns a solution state current ← problem.INITIAL-STATE for t = 1 to ∞ do T ← schedule(t) if T = 0 then return current next ← a randomly selected successor of current ∆E ← VALUE(next) - VALUE(current) if ∆E > 0 then current ← next else current ← next only with probability e∆E/T

The schedule determines the value of the temperature T as a function of time.

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 24/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Simulated annealing II

If a random move improves the situation, it is always accepted,

• therwise move accepted with some probability less than 1.

Probability decreases exponentially with the “badness” of the move i.e. the amount ∆E by which the evaluation is worsened. Probability also decreases as the “temperature” T goes down: bad moves are more likely to be allowed at the start when T is high, and they become more unlikely as T decreases. T decreased slowly enough by schedule = ⇒ always reach best state. Widely used in VLSI layout, factory and airline scheduling, etc.

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 25/26

T H E U N I V E R S I T Y O F E D I N B U R G H

Summary

Informed search methods may have access to a heuristic function h(n) that estimates the cost of a solution from n.

Best-first search algorithm selects a node for expansion according to an evaluation function. Greedy best-first search expands nodes with minimal h(n). It is not optimal but is often efficient. A∗ search expands nodes with minimal f (n) = g(n) + h(n). It is complete and optimal, provided that h(n) is admissible or consistent.

The performance of heuristic search algorithms depends on the quality of the heuristic function. Local search methods such as hill climbing operate on complete-state formulations. Many stochastic algorithms have been developed, including simulated annealing, which returns optimal solutions when given an appropriate cooling schedule (See R&N 4.2-4.5, for

• ther approaches).

Jacques Fleuriot Informed Search Algorithms and Beyond, R&N 3.5–3.6, 4.1 26/26