Local Search for a Globally Optimal Solution Russell and Norvig - - PowerPoint PPT Presentation

Local Search for a Globally Optimal Solution

Russell and Norvig Chapter 4

Limitations of hill climbing

n Can we find a globally optimal solution? n Can’t guarantee that. A good alternative: an

approach that will give that “with high probability”.

n Need a more thorough exploration of the

state space

Limitations of hill climbing

n An algorithm that never makes downhill moves

will get stuck in local minima

Simulated Annealing

n Physical systems are good at finding

minimum energy configurations: a physical system slowly cooled down to absolute zero will settle into its minimum energy configuration.

n This is called annealing n Mimic this using a probabilistic process

Simulated Annealing

n Simulated annealing [Kirkpatrick, Gelatt, Vecchi, 1983].

q T large ⇒ probability of accepting an uphill move is large. q T small ⇒ uphill moves are almost never accepted. q Idea: turn knob to control T. q Cooling schedule: T = T(i) at iteration i.

n Physical analogy.

q When we take a molten solid and freeze it very abruptly, we do not

expect to get a perfect crystal.

q Annealing: cool material gradually from high temperature, allowing

it to reach equilibrium at succession of intermediate lower temperatures.

q Algorithm will find optimal solution with high probability if using a

sufficiently slow cooling schedule.

Example

Image from http://en.wikipedia.org/wiki/Simulated_annealing

Simulated annealing for TSP

Simulated Annealing

function SIMULATED-ANNEALING( problem, schedule) return a solution state input: problem, a problem schedule, a mapping from time to temperature current ← MAKE-NODE(INITIAL-STATE[problem]) for t ← 1 to ∞ do T ← schedule[t] if T = 0 then return current. next ← a randomly selected successor of current ∆E ← current.VALUE - next.VALUE if ∆E > 0 then current ← next else current ← next with probability e∆E /T

Temperature controls the probability of non-increasing steps.

Properties of Simulated Annealing

n As the number of moves at a given temperature goes to

infinity, the probability of a state becomes proportional to exp(-E/T) (Boltzman distribution)

n If temperature is lowered slowly enough - global optimum

will be found with high probability. A lot of research into what makes a good cooling schedule.

n Widely used in a variety of applications (VLSI layout,

airline scheduling, etc.)

Beam Search

n Variant of hill climbing:

q Initially: k random states q Next: determine all successors of k states q If any successor is optimal → done q Else select k best from successors and repeat.

n Major difference with random-restart search:

q Information is shared among k search threads.

n Can suffer from lack of diversity (somewhat addressed

by stochastic beam search which chooses k-best randomly according to their value).

Genetic algorithms

n Keep a population of solutions that undergo

recombination and mutation

Genetic algorithms

The state is the genetic material that makes an individual

chromosome Note: cross-over should happen with a certain probability: if an individual is already highly fit, it’s not a good idea to change it too much.

(a) Initial Population (b) Fitness Function (c) Selection (d) Crossover (e) Mutation

24 23 20 11 29% 31% 26% 14%

32752411 24748552 32752411 24415124 32748552 24752411 32752124 24415411 32252124 24752411 32748152 24415417 24748552 32752411 24415124 32543213

+ =

Crossover (aka recombination)

One-point crossover Two-point crossover

Genetic algorithms

function GENETIC_ALGORITHM(population, FITNESS-FN) return an individual input: population, a set of individuals FITNESS-FN, a function quantifying the quality of an individual repeat new_population ← empty set for i =1 to SIZE(population) do x ← RANDOM_SELECTION(population, FITNESS_FN) y ← RANDOM_SELECTION(population, FITNESS_FN) child ← REPRODUCE(x,y) MUTATE(child) add child to new_population population ← new_population until some individual is fit enough or enough time has elapsed return the best individual

Solving TSP

n Represent a tour as a permutation (i1,…,in) of {1,2,

…,n}

n Fitness of a solution: negative of the cost of the tour n Initialization: a random set of permutations n Need to define crossover and mutation operations.

Crossover

n Order crossover: choose a subsequence of a tour

from one parent and preserve the relative order of the cities from the other.

Example: p1 = (1 2 3 | 5 4 6 7 | 8 9) p2 = (4 5 2 | 1 8 7 6 | 9 3) c1 = (x x x | 5 4 6 7 | x x) c2 = (x x x | 1 8 7 6 | x x)

The tour in p2, starting from its second cut point, is 9 → 3 → 4 → 5 → 2 → 1 → 8 → 7 → 6. Remove the cities already in c1, obtaining the partial tour 9 → 3 → 2 → 1 → 8. Insert this partial tour after the second cut point of c1, resulting in c1 = (2 1 8 | 5 4 6 7 | 9 3 ).

Crossover (2)

n

Partially Mapped (PMX) crossover: choose a subsequence of a tour from one parent and preserve the order and position

• f as many cities as possible from the other

parent.

PMX crossover

p1 = (1 2 3 | 4 5 6 7 | 8 9) p2 = (4 5 2 | 1 8 7 6 | 9 3) c1 = (x x x | 4 5 6 7 | x x) c2 = (x x x | 1 8 7 6 | x x) Swap defines a mapping: 1 ↔ 4, 8 ↔ 5, 7 ↔ 6, 6 ↔ 7. The easy ones: c1 = (x 2 3 | 1 8 7 6 | x 9) c2 = (x x 2 | 4 5 6 7 | 9 3) For the rest, use the mapping: c1 = (4 2 3 | 1 8 7 6 | 5 9) c2 = (1 8 2 | 4 5 6 7 | 9 3).

Mutation

Can use a 2-opt operation: Select two points along the permutation, cut it at these points and re-insert the reversed string. Example:

(1 2 | 3 4 5 6 | 7 8 9) → (1 2 | 6 5 4 3 | 7 8 9)

The knapsack problem

Local search: summary

n Why I like local search algorithms

u Easy to implement u Widely applicable u Provide good results

Online search

n So far we have assumed deterministic actions and fully-

known environments

q Permits off-line search

n Consider a new problem:

q A robot is placed in the middle of a maze q The task is to find the exit q Actions are deterministic but the environment is unknown

Difference from offline agents: An online agent can only expand the node it is physically in. What search strategies are applicable?

Online agents

q Difference from offline agents: An online agent can only

expand the node it is physically in.

q Therefore agent needs to work locally: Online DFS, IDS. q Possible only when actions are reversible. q Heuristic search: LRTA*