Hyper-heuristics: Raising the Level of Generality of Search Methodologies Graham Kendall The University of Nottingham gxk@cs.nott.ac.uk http://www.cs.nott.ac.uk/~gxk
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Contents • What is a hyper-heuristic? • What motivates hyper-heuristic research? • Hyper-heuristic Methods • Observations and Future Potential • Questions/Discussions
Hyper-heuristics: Raising the Level of Generality of Search Methodologies What is a hyper-heuristic? • Simple Idea: Heuristics to choose heuristics • Operates on a search space of heuristics rather than directly on a search space of solutions • Meta-heuristics have been used with some success as hyper-heuristics, as have other approaches such as case based reasoning
Hyper-heuristics: Raising the Level of Generality of Search Methodologies What is the difference between a hyper- heuristic and a meta-heuristic? • All the term hyper-heuristic says is: “ Operate on a search space of heuristics ”. • We have a high level search method (the hyper- heuristic) - which may (or may not) be a meta- heuristic
Hyper-heuristics: Raising the Level of Generality of Search Methodologies What is the difference between a hyper-heuristic and a meta- heuristic? • A hyper-heuristic searches through a space of heuristics (which, potentially, could be meta-heuristics) • Most meta-heuristic implementations operate directly on a search space of potential solutions
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Different Search Spaces Hyper-heuristic Operates upon Heuristics Operates upon Potential Solutions
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Contents • What is a hyper-heuristic? • What motivates hyper-heuristic research? • Hyper-heuristic Methods • Observations and Future Potential • Questions/Discussions
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Motivations behind hyper-heuristic research • What is our strategic research vision? • What game to play?
Hyper-heuristics: Raising the Level of Generality of Search Methodologies The “Best Quality Solution” Game • We have a problem (e.g. exam timetabling) • We have a set of benchmark problems • We develop new methodologies (ever more sophisticated)
Hyper-heuristics: Raising the Level of Generality of Search Methodologies The “Best Quality Solution” Game • Apply methodologies to benchmarks • Compare with other “players” • The goal is to “get better quality solutions” than the other players • Good Game
Hyper-heuristics: Raising the Level of Generality of Search Methodologies The “Best Quality Solution” Game Benchmark Instances e.g. Exam Timetabling
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Consequences of “Best Quality Solution” Game • Handcrafted bespoke decision support methodologies • Made to Measure – NOT off the peg • Rolls-Royce systems • We compare Rolls-Royces with Bentleys with Mercedes with Ferraris…….. And it works
Hyper-heuristics: Raising the Level of Generality of Search Methodologies A New Game? • What about Ford Model Ts’? • Do we have the technology to mass produce decision support systems? • Develop decision support systems that are off the peg? • Can we develop the ability to automatically work well on different problems?
Hyper-heuristics: Raising the Level of Generality of Search Methodologies A new game to play: Many Walls? • Raising the level of generality • Still want to get as high up the wall as possible ……… BUT……… • We want to be able to operate on as many different walls as possible • The goal is to increase the number of walls you can operate on – while still getting acceptably high up each wall
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Raising the Level of Generality One Method
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Evaluating Different Methods • It IS a new game • Cannot sensibly compare a Model T to a Rolls Royce – different functions for different clientele • Raising the level of generality is the goal
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Evaluating Different Methods • Still want solution quality to be as high as possible though • Good Enough – Soon Enough – Cheap Enough
Hyper-heuristics: Raising the Level of Generality of Search Methodologies A Grand Challenge • Automating the heuristic design process • Deeper understanding of how high we can raise the level of generality: What are the limits?
Hyper-heuristics: Raising the Level of Generality of Search Methodologies A Grand Challenge The General Solver Doesn’t exist…. Significant scope for future research More General These situations exist Problem Specific Solvers
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Hyper-heuristics • Motivated by raising the level of generality • Role to play in BOTH games
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Contents • What is a hyper-heuristic? • What motivates hyper-heuristic research? • Hyper-heuristic Methods • Observations and Future Potential • Questions/Discussions
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Intellectual Roots • Hyper-heuristic research has many of its roots in the mid-nineties • But can be traced back to the 60’s
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Intellectual Roots • Fisher H. and Thompson G.L. Probabilistic Learning Combinations of Local Job-shop Scheduling Rules. In Factory Scheduling Conference, Carnegie Institute of Technology, May 10-12 , 1961 • Fisher H. and Thompson G.L. Probabilistic Learning Combinations of Local Job-shop Scheduling Rules. Ch 15,:225- 251, Prentice Hall, New Jersey , 1963 • Crowston W.B., Glover F., Thompson G.L. and Trawick J.D. Probabilistic and Parameter Learning Combinations of Local Job Shop Scheduling Rules. ONR Research Memorandum, GSIA, Carnegie Mellon University, Pittsburgh , (117), 1963 The learning mechanism used probabilistic weightings of low level heuristics which represented scheduling rules
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Intellectual Roots • H-L Fang, P.M.Ross and D.Corne. A Promising Hybrid GA/Heuristic Approach for Open-Shop Scheduling Problems'', in Proceedings of ECAI 94: 11 th European Conference on Artificial Intelligence , A. Cohn (ed), pp 590- 594, John Wiley and Sons Ltd, 1994 The chromosome was a set of heuristics that was chosen to schedule a job on a machine. As opposed to the “normal” method of having the chromosome as representing a set of jobs to be scheduled in a given order.
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Intellectual Roots • Hart E, Ross P. and Nelson J.A.D. Solving a Real World Problem using an Evolving Heuristically Driven Schedule Builder. Evolutionary Computing 6(1):61-80 , 1998 • Hart E, Ross P. and Nelson J.A.D. Scheduling Chicken Catching: An Investigation into the Success of a Genetic Algorithm on a Real World Scheduling Problem. Annals of Operations Research 92:363-380 , 1999 Problem is decomposed into two sub-problems, each being solved with a GA. The chromosomes represent heuristics.
Hyper-heuristics: Raising the Level of Generality of Search Methodologies A Hyper-heuristic Framework Hyper-heuristic Data flow Domain Barrier Data flow Set of low level heuristics H 1 H 2 H n …… Evaluation Function
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Choice Function Hyper-heuristic f1 + f2 + f3 Data flow How well has each heuristic done + Domain Barrier How well have pairs of heuristics done + Data flow Time since last called Set of low level Applied to sales summit scheduling, heuristics CS&IT presentations, nurse rostering …… H n H 2 H 1 Evaluation Function
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Choice function Hyper-heuristic Sales Summit Scheduling Low level heuristics are based on three types of neighbourhood moves : • Add / Remove one delegate to / from the current solution • Add / remove a meeting to / from the current solution (random, 1 st improving, best improving, etc) • Remove excess of meetings from an overloaded delegate
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Choice function Hyper-heuristic CS&IT Presentation Scheduling • Produced solutions that were better than manually produced solutions • Solutions used in CS&IT Nurse Rostering • Competitive with other results in the literature • Same algorithm worked across a range of problems
Hyper-heuristics: Raising the Level of Generality of Search Methodologies Tabu Search Hyper-heuristic • Simple moving strategies as low level heuristics • Low level heuristics compete with each other • Recent heuristics in tabu • Rank low level heuristics based on their estimated performance potential • Easily applicable on both nurse rostering and course timetabling: Competitive results on both problems with the same settings
Recommend
More recommend
