Heuristic Optimization Thomas St utzle IRIDIA, CoDE Universit e - - PDF document

Heuristic Optimization

Thomas St¨ utzle IRIDIA, CoDE Universit´ e Libre de Bruxelles stuetzle@ulb.ac.be iridia.ulb.ac.be/∼stuetzle iridia.ulb.ac.be/∼stuetzle/Teaching/HO

Example problems

◮ imagine a very good friend from Germany visits you and he

wants to visit all 239 breweries in Belgium during his one week stay

Is this feasible? If yes, which route to take? The shortest certainly helps

◮ at brewery No. 49 your friend offers to pay all beers you take

• n the trip if you solve the following riddle

‘Last week my friends Anne, Carl, Eva, Gustaf and I went out for dinner every night, Monday through Friday. I missed the meal on Friday because I was visiting my sister and her family. But otherwise, every one of us had selected a restaurant for a particular night and served as a host for that

• dinner. Overall, the following restaurants were selected: a French bistro,

a sushi bar, a pizzeria, a Greek restaurant, and the Brauhaus. Eva took us out on Wednesday. The Friday dinner was at the Brauhaus. Carl, who doesn’t eat sushi, was the first host. Gustaf had selected the bistro for the night before one of the friends took everyone to the pizzeria. Tell me, who selected which restaurant for which night?

Heuristic Optimization, 2012 2

How to solve it?

◮ many possible approaches ◮ systematic enumeration is probably not realistic ◮ some people may eliminate certain assignments or partial

tours through careful reasoning

◮ other intuitive approach: start with some good guess and then

try to improve it iteratively The latter is an example of a heuristic approach to optimization

Heuristic Optimization, 2012 3

Optimization

Optimization refers to choosing the best element from some set of available alternatives.

Optimization problems . . .

◮ arise in a wide variety of applications ◮ arise in many different forms, e.g., continuous, combinatorial,

multi-objective, stochastic, etc.

◮ here we focus mainly on combinatorial problems

◮ range from quite easy to hard ones

◮ here we focus on the hard ones! Heuristic Optimization, 2012 4

.. an easy one

find the best (most valuable) element from the set of alternatives

Heuristic Optimization, 2012 5

.. a more difficult (but still “easy”) one

find best (shortest) route from A to B in an edge-weighted graph

Heuristic Optimization, 2012 6

.. a harder one

find best (shortest) round trip through some cities, aka Traveling Salesman Problem (TSP)

Heuristic Optimization, 2012 7

find best (shortest) round trip through some cities, aka Traveling Salesman Problem (TSP)

(see also http://www.tsp.gatech.edu//maps/)

Heuristic Optimization, 2012 8

Practical applications of the TSP

Heuristic Optimization, 2012 9

.. and a large instance

Heuristic Optimization, 2012 10

A more real-life like problem

TSP arises as sub-problem, e.g., in vehicle routing problems (VRPs)

Depot Customers

Heuristic Optimization, 2012 11

Depot Customers vehicle routes

Heuristic Optimization, 2012 12

◮ realistic problems can involve many complicating details ◮ examples in VRP case are

◮ time windows, access restrictions, priorities, split delivery, . . . ◮ capacity restrictions, different costs of vehicles, . . . ◮ working time restrictions, breaks, . . . ◮ stochastic travel times or demands, incoming new requests, . . .

◮ in lecture: focus on simplified models of (real-life) problems

◮ useful for illustrating algorithmic principles ◮ they are “hard” and capture essence of more

complex problems

◮ are treated in research to yield more general insights Heuristic Optimization, 2012 13

Solving (combinatorial) optimization problems

◮ systematic enumeration ◮ problem specific, dedicated algorithms ◮ generic methods for exact optimization ◮ heuristic methods

Heuristic Optimization, 2012 14

Heuristic methods

Heuristic methods intend to compute efficiently, good solutions to a problem with no guarantee of optimality

◮ range from rather simple to quite sophisticated approaches ◮ inspiration often from

◮ human problem solving ◮ rules of thumb, common sense rules ◮ design of techniques based on problem-solving experience ◮ natural processes ◮ evolution, swarm behaviors, annealing, . . .

◮ usually used when there is no other method to solve the

problem under given time or space constraints

◮ often simpler to implement / develop than other methods

Heuristic Optimization, 2012 15

Heuristic methods are the method of choice for solving many computationally hard problems. Recent Progress & Successes:

◮ Ability of solving hard combinatorial problems

has increased significantly

◮ Solution of large travelling salesman problems ◮ Solution of large graph coloring problems

◮ Excellent results in many application areas

Heuristic Optimization, 2012 16

Goals of this course

Provide answers to these questions:

◮ How can heuristic methods be used to solve computationally

hard problems?

◮ Which heuristic methods are available and what are their

features?

◮ How should heuristic methods be studied and analysed

empirically?

◮ How can heuristic algorithms be designed, developed, and

implemented?

Heuristic Optimization, 2012 17

Contents

Basics:

◮ introduction, SLS framework ◮ iterative improvement algorithms ◮ simple SLS methods ◮ hybrid and population-based SLS methods ◮ empirical analysis of SLS algorithms ◮ search space analysis

Additional topics:

◮ tuning, algorithm portfolios ◮ complex problem features ◮ SLS methods for continuous optimization

Heuristic Optimization, 2012 18

Heuristic Optimization field

SLS Appli- cations Computer Science Operations Research Statis- tics HO

Heuristic Optimization, 2012 19

Organizational matters

◮ webpages

iridia.ulb.ac.be/∼stuetzle/Teaching/HO http://www.sls-book.net/

◮ lectures and exercises

◮ Monday, 14:00 to 16:00, P.Forum.H (except Mar 12,

P.Forum.E)

◮ Wednesday, 10:00 to 12:00, P.OF.2070 (Feb 15, 22),

S.C5.130–IRIDIA (Feb 29, Mar 7), P.Forum.B (Mar 14), P.OF.2072 (Rest)

◮ lecture dates (preliminary schedule; check for updates)

◮ February 13, 15, 20, 22 ◮ March 5, 12, 26, 28 ◮ April 16, 23, 25, 30 Heuristic Optimization, 2012 20

◮ exercises and implementation tasks

◮ five exercise sessions ◮ exercise dates (preliminary schedule; check for updates) ◮ Feb 29, Mar 7, 14, April 18, May 2 ◮ two implementation exercises (second builds on first one) ◮ First: Feb 27 with short introductory lecture ◮ Second: March 28 Heuristic Optimization, 2012 21

◮ evaluation

◮ precondition for passing course: successful completion of the

implementation tasks (≥ 10 for both)

◮ oral exam at the end of semester (counts 75%) ◮ implementation exercises (counts 25%) ◮ final mark: weighted average of implementation exercises and

• ral exam

◮ course material, literature

◮ slides ◮ H. H. Hoos and T. St¨

• utzle. Stochastic Local Search:

Foundations and Applications. Morgan Kaufmann Publishers, 2005.

◮ additional literature will be given during the course Heuristic Optimization, 2012 22