Functional Abstractions for Simulated Annealing Richard Senington - - PowerPoint PPT Presentation

▶
functional abstractions for simulated annealing
SMART_READER_LITE
LIVE PREVIEW

Functional Abstractions for Simulated Annealing Richard Senington - - PowerPoint PPT Presentation

Jan 03, 2024 457 likes •872 views

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Functional Abstractions for Simulated Annealing Richard Senington University Of Leeds School Of Computing Faculty of Engineering sc06r2s@leeds.ac.uk

slide-1
SLIDE 1

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion

Functional Abstractions for Simulated Annealing

Richard Senington

University Of Leeds

School Of Computing Faculty of Engineering sc06r2s@leeds.ac.uk

5th May 2011

For York, May 2011 Richard Senington Functional Abstractions for Simulated Annealing

slide-2
SLIDE 2

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Introduction

The Wider Project Investigating Local Search Meta-heuristics Finding combinators for describing them and combinators for combining/hybridising them Description, abstraction and implementation in Haskell This Mini-Project Simulated Annealing (SA) is a family of Meta-heuristics The variations on SA, provide illustration of wider issues Description, abstraction and implementation in Haskell

Richard Senington Functional Abstractions for Simulated Annealing

slide-3
SLIDE 3

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Local Search

Sub-Field of Operational Research Optimisation of Combinatorial Problems NP-Hard, cannot solve for large instances (Meta-)heuristics are often used for these large instances Exploring space to find good/better solutions Making decisions based upon limited/local information Heuristic methods to make decisions

Richard Senington Functional Abstractions for Simulated Annealing

slide-4
SLIDE 4

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Meta-Heuristics

Patterns, or Templates for Heuristics Work on most or many problems E.g. hill climbing, restart, genetic algorithms, simulated annealing Each type forms a family of variations on the theme Families can be hybridised

Richard Senington Functional Abstractions for Simulated Annealing

slide-5
SLIDE 5

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Simulated Annealing

1 Initialise Seed Solution, Temperature & Random Numbers Richard Senington Functional Abstractions for Simulated Annealing

slide-6
SLIDE 6

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Simulated Annealing

1 Initialise Seed Solution, Temperature & Random Numbers 3 Permute Current Solution Richard Senington Functional Abstractions for Simulated Annealing

slide-7
SLIDE 7

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Simulated Annealing

1 Initialise Seed Solution, Temperature & Random Numbers 3 Permute Current Solution 4 Choose between old and new

depends on temperature and random number stream

Richard Senington Functional Abstractions for Simulated Annealing

slide-8
SLIDE 8

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Simulated Annealing

1 Initialise Seed Solution, Temperature & Random Numbers 3 Permute Current Solution 4 Choose between old and new

depends on temperature and random number stream

7 Update Temperature using Strategy Richard Senington Functional Abstractions for Simulated Annealing

slide-9
SLIDE 9

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Simulated Annealing

1 Initialise Seed Solution, Temperature & Random Numbers 3 Permute Current Solution 4 Choose between old and new

depends on temperature and random number stream

7 Update Temperature using Strategy 8 Choose to End or return to 3 Richard Senington Functional Abstractions for Simulated Annealing

slide-10
SLIDE 10

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Simulated Annealing

1 Initialise Seed Solution, Temperature & Random Numbers 2 Initialise Monitoring Data 3 Permute Current Solution 4 Choose between old and new

depends on temperature and random number stream

7 Update Temperature using Strategy 8 Choose to End or return to 3 Richard Senington Functional Abstractions for Simulated Annealing

slide-11
SLIDE 11

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Simulated Annealing

1 Initialise Seed Solution, Temperature & Random Numbers 2 Initialise Monitoring Data 3 Permute Current Solution 4 Choose between old and new

depends on temperature and random number stream

5 Update Monitoring Data 7 Update Temperature using Strategy 8 Choose to End or return to 3 Richard Senington Functional Abstractions for Simulated Annealing

slide-12
SLIDE 12

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Local Search Meta-Heuristics Simulated Annealing

Simulated Annealing

1 Initialise Seed Solution, Temperature & Random Numbers 2 Initialise Monitoring Data 3 Permute Current Solution 4 Choose between old and new

depends on temperature and random number stream

5 Update Monitoring Data 6 Should Temperature Strategy be Changed

if so change it

7 Update Temperature using Strategy 8 Choose to End or return to 3 Richard Senington Functional Abstractions for Simulated Annealing

slide-13
SLIDE 13

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Naive Type Classes & Records Functions For SA Descriptions Of SA

Monolithic State Type

lots of pattern matching Very bad in terms of flexibility Few ways to generalise f (a, b, c, d) = ..... g (a, b, c, d) = ..... h (a, b, c, d) = .....

Richard Senington Functional Abstractions for Simulated Annealing

slide-14
SLIDE 14

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Naive Type Classes & Records Functions For SA Descriptions Of SA

Type Classes & Records

Getter and Setter functions in a Type class Functions can operate if data type has right accessor class class Ac a b where get :: a → b set :: a → b → a

Richard Senington Functional Abstractions for Simulated Annealing

slide-15
SLIDE 15

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Naive Type Classes & Records Functions For SA Descriptions Of SA

Type Classes & Records

Can build these quickly using record access. instance Ac D Temp where get = getTemp set x t = x {getTemp = t }

Richard Senington Functional Abstractions for Simulated Annealing

slide-16
SLIDE 16

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Naive Type Classes & Records Functions For SA Descriptions Of SA

Some Useful Functions

tempChange :: Ac a Temp ⇒ (Temp → Temp) → a → a varTempChange :: Ac a (a → a) ⇒ a → a updateTempStrat :: (Ac a (a → a), Ac a Monitor) ⇒ (Monitor → Bool) → a → a updateMonitor :: (Ac a Monitor, Ac a [Sol ]) ⇒ (Monitor → Sol → Monitor) → a → a permute :: Ac a [Sol ] ⇒ a → a choose :: (Ac a [Sol ], Ac a Temp) ⇒ a → a

Richard Senington Functional Abstractions for Simulated Annealing

slide-17
SLIDE 17

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Naive Type Classes & Records Functions For SA Descriptions Of SA

Function Composition

sa :: (Ac a Temp, Ac a [Sol ]) ⇒ (Temp → Temp) → a → a sa f = (tempChange f ) ◦ choose ◦ permute sa :: (Ac a (a → a), Ac a Monitor, Ac a Temp, Ac a [Sol ]) ⇒ (Monitor → Bool) → (Monitor → Sol → Monitor) → a → a sa f g = varTempChange ◦ (updateTempStrat f ) ◦ (updateMonitor g) ◦ choose ◦ permute

Richard Senington Functional Abstractions for Simulated Annealing

slide-18
SLIDE 18

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Naive Type Classes & Records Functions For SA Descriptions Of SA

Other Descriptions

Arrow sa f = permute > > choose > > (tempChange f )

Richard Senington Functional Abstractions for Simulated Annealing

slide-19
SLIDE 19

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Naive Type Classes & Records Functions For SA Descriptions Of SA

Other Descriptions

Arrow sa f = permute > > choose > > (tempChange f ) No reason a monadic version could not be created

Richard Senington Functional Abstractions for Simulated Annealing

slide-20
SLIDE 20

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Naive Type Classes & Records Functions For SA Descriptions Of SA

Issues

Lots of boiler plate code For each description function you choose, you often must;

add data component explicitly to larger data structure implement accessors (the boiler plate)

Quite imperative style Explain when and how to do things, not just what or relationships Feels like the data structure should be quite implicit in the code Where is the separation of search logic?

Richard Senington Functional Abstractions for Simulated Annealing

slide-21
SLIDE 21

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

and then David said....FRP

Processes modelled as behaviours in time Time → a Events that can be detected and reacted to (Time, a) Standard combinators for creating and processing these

detect :: Beh a → (a → Bool) → Event () until :: Beh a → Event (Beh a) → Beh a

Richard Senington Functional Abstractions for Simulated Annealing

slide-22
SLIDE 22

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Simulated Annealing as Behaviours

r(i) = randomSeed if i 0 next(r(i − 1))

  • therwise

t(i) = tempSeed if i 0 t(i − 1) ∗ geoDrop

  • therwise

sa(i) =    seedSolution if i 0 sa(i − 1) if not accepted permute(sa(i − 1), r(i ∗ 2))

  • therwise

accepted = accept(permute(sa(i − 1), r(i ∗ 2)), t(i), r(i ∗ 2 + 1))

Richard Senington Functional Abstractions for Simulated Annealing

slide-23
SLIDE 23

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Issues Very slow Due to it causing lots of re-computation

Richard Senington Functional Abstractions for Simulated Annealing

slide-24
SLIDE 24

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Issues Very slow Due to it causing lots of re-computation Solutions Global Memoization, moves the issue to ever increasing memory load

Richard Senington Functional Abstractions for Simulated Annealing

slide-25
SLIDE 25

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Issues Very slow Due to it causing lots of re-computation Solutions Global Memoization, moves the issue to ever increasing memory load Local Windowed Memoization (recent states only)

Richard Senington Functional Abstractions for Simulated Annealing

slide-26
SLIDE 26

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Data Dependency Patterns

Richard Senington Functional Abstractions for Simulated Annealing

slide-27
SLIDE 27

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Simulated Annealing, Basic

sa g s = let (g1, g2) = split g sStream = simulatedAnnealing saChoice tStream (randoms g1) s pStream pStream = zipWith permute (randoms g2) sStream tStream = geoCooling 0.99 80000 in sStream

Richard Senington Functional Abstractions for Simulated Annealing

slide-28
SLIDE 28

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Simulated Annealing, Heating

sa g s = let (g1, g2) = split g sStream = simulatedAnnealing saChoice tStream′ (randoms g1) s pStream pStream = zipWith permute (randoms g2) sStream cStream = False : zipWith (λa b → (solutionValue b − solutionValue a) < 0) (delay 1 sStream) sStream tStream cS = (geoCooling 3 1) ‘until ‘ (zip′ cS (tail $ tails cS)) (λcS′ → tStream cS′) tStream′ = tStream conStream in sStream

Richard Senington Functional Abstractions for Simulated Annealing

slide-29
SLIDE 29

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Simulated Annealing, Restart

sa g s = let (g1, g2) = split g sStream = simulatedAnnealing saChoice tStream (randoms g1) s pStream pStream = zipWith permute (randoms g2) sStream dStream = delay 170 bStream

  • verRide = take 175 $ repeat False

conStream = False : zipWith (λa b → (solutionValue b − solutionValue a) ≡ 0) dStream bStream tempStream cS = until (geoCooling 0.99 80000) (zip′ cS (tail $ tails cS)) (λcS′ → tempStream (replace overRide cS′)) tStream = tempStream conStream bStream = theBest sStream in sStream

Richard Senington Functional Abstractions for Simulated Annealing

slide-30
SLIDE 30

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Simulated Annealing, Nested Iterative Improver

sa g s = let (g1, g2) = split g sStream = simulatedAnnealing saChoice tStream (randoms g1) s pStream pStream = map (ffIterativeImprover permute) $ zipWith permute (randoms g2) sStream tStream = geoCooling 0.99 80000 in sStream

Richard Senington Functional Abstractions for Simulated Annealing

slide-31
SLIDE 31

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Simple Cooling Strategies

These are found in the original Simulated Annealing paper by Kirkpatrick et al, 1983 linearCooling :: Floating b ⇒ b → b → [b] linearCooling tempChange startTemp = iterate (+tempChange) startTemp geoCooling :: Floating b ⇒ b → b → [b] geoCooling tempChange startTemp = iterate (∗tempChange) startTemp

Richard Senington Functional Abstractions for Simulated Annealing

slide-32
SLIDE 32

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Simulated Annealing as Behaviours Data Dependency & Lazy Lists Simulated Annealing using Lists Simulated Annealing, Components

Generic Components

class Value a b where getValue :: a → b saChoice :: (Value a b, Floating b, Ord b) ⇒ b → b → a → a → a simulatedAnnealing :: (b → b → a → a → a) → [b] → [b] → a → [a] → [a] simulatedAnnealing accept ts rs x xs = scanl (λc (t, r, x′) → accept r t c x′) x (zip3 ts rs xs) theBest :: Ord a ⇒ [a] → [a] theBest xs = scanl min (head xs) (tail xs)

Richard Senington Functional Abstractions for Simulated Annealing

slide-33
SLIDE 33

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Related Work Summary Current & Future Work

Related Work (from Hackage)

SimpleEA, 0.1.1 (Jan 2011) Pure Uses simple lists genprog, 0.1 (Nov 2010) Monadic Explicit Population concurrent-sa, 1.0.1 (Apr 2010) Captures operations within the IO monad

Richard Senington Functional Abstractions for Simulated Annealing

slide-34
SLIDE 34

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Related Work Summary Current & Future Work

Related Work cont..

hgalib, 0.2 (2008) Monolithic State Monadic Uses Parallel Arrays Etage, 0.1.8 (Apr 2011) A library for modelling data flow abstractions Heavily focused upon neural networks

Richard Senington Functional Abstractions for Simulated Annealing

slide-35
SLIDE 35

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Related Work Summary Current & Future Work

Summary

We have looked at the operation of classic SA Looked at how it could be described using state data types and classes Seen how it can be described using lists Seen how lists are more flexible

Richard Senington Functional Abstractions for Simulated Annealing

slide-36
SLIDE 36

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Related Work Summary Current & Future Work

Current Work

I need to continue testing the representation of families in the list formulation I need to find better combinators for describing the data dependency patterns

Richard Senington Functional Abstractions for Simulated Annealing

slide-37
SLIDE 37

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Related Work Summary Current & Future Work

Future Work

How to make this f :: Int → [Int ] → [Int ] f s∼(x : xs) = s : (f (s + x) xs) g :: Int → [Int ] → [Int ] g t∼(x : xs) = t : (g (t ∗ x) xs) h a b = let g′ = g a f ′ f ′ = f b g′ in f ′

Richard Senington Functional Abstractions for Simulated Annealing

slide-38
SLIDE 38

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Related Work Summary Current & Future Work

Future Work

into this h :: (Int, Int) → [Int ] h (s, t) = s : h (s + t, s ∗ t)

Thanks to Peter for the example

Richard Senington Functional Abstractions for Simulated Annealing

slide-39
SLIDE 39

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Related Work Summary Current & Future Work

Thankyou for listening

Richard Senington

sc06r2s@leeds.ac.uk School of Computing Faculty of Engineering University Of Leeds

Are there any questions, thoughts or comments please?

Richard Senington Functional Abstractions for Simulated Annealing

slide-40
SLIDE 40

Introduction Monolithic State Implementations Functional Reactive Programming Conclusion Related Work Summary Current & Future Work

Genetic Algorithm (recombination aspect)

ga ts g = let (g1, g2) = split g sols = ts + + newSols pops = drop popSize ◦ blocks popSize ◦ map sort ◦ windowOn popSize $ sols newSols = zipWith recombine (zipWith choose2 pops (chunkStream popSize $ randoms g1)) (randoms g2) popSize = length ts

  • - presumed

choose2 xs rs = let as = [a | (a, b) ← zip xs rs, b < 0.5] bs = [a | (a, b) ← zip xs rs, b > 0.5] a = if null as then last xs else head as b = if null bs then last xs else head bs in (a, b) in sols

Richard Senington Functional Abstractions for Simulated Annealing