Design and Architectures for Design and Architectures for Embedded - - PowerPoint PPT Presentation

1 ESII: SimAnneal

Design and Architectures for Design and Architectures for Embedded Systems (ESII) Embedded Systems (ESII)

Prof Dr J Henkel M Shafique Prof Dr J Henkel M Shafique

• Prof. Dr. J. Henkel, M. Shafique
• Prof. Dr. J. Henkel, M. Shafique

CES CES -

• Chair for Embedded Systems

Chair for Embedded Systems Karlsruhe Institute of Technology, Germany Karlsruhe Institute of Technology, Germany

Add Add-

• on Slides: Simulated Annealing
• n Slides: Simulated Annealing
• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu

2 ESII: SimAnneal

Simulated Annealing (SA) Simulated Annealing (SA)

SA belongs to group of simple optimization algorithms: SA belongs to group of simple optimization algorithms: SA belongs to group of simple optimization algorithms: SA belongs to group of simple optimization algorithms:

Steepest Ascent: start with feasible solution and choose best Steepest Ascent: start with feasible solution and choose best neighbor to continue from neighbor to continue from Next Ascent: continue if there is Next Ascent: continue if there is any any better solution in neighborhood better solution in neighborhood Stochastic Gradient Approach: Stochastic Gradient Approach: Next neighbor selected with a certain probability Next neighbor selected with a certain probability Next neighbor selected with a certain probability Next neighbor selected with a certain probability Probability function is given Probability function is given Enables to escape local optima Enables to escape local optima ( Hill Cli bi ) Hill Cli bi ) (-> Hill Climbing) > Hill Climbing) Iterative SA Iterative SA Extension of stochastic gradient Extension of stochastic gradient

cost

x g approach approach Acceptance function depends on Acceptance function depends on t l t t l t S1 x x

• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu

control parameter control parameter S2

3 ESII: SimAnneal

Basic idea of SA Basic idea of SA

Simulates the annealing process of steel: Simulates the annealing process of steel: A t l l d A t l l d l l l l l tti it lf t bt i l l tti it lf t bt i l As steel cools down As steel cools down slowly slowly, lattice arranges itself to obtain lower , lattice arranges itself to obtain lower energy states (= better solution) energy states (= better solution) If temperature decrease is too fast, lattice order is If temperature decrease is too fast, lattice order is frozen frozen in a high in a high-

• p

, p , g energy state (= bad solution) energy state (= bad solution) Therefore: cooling process i.e. temperature T needs to be Therefore: cooling process i.e. temperature T needs to be controlled controlled -> cooling schedule > cooling schedule controlled controlled -> cooling schedule > cooling schedule If a solution ‘a’ is worse than a solution ‘b’, accept a with probability If a solution ‘a’ is worse than a solution ‘b’, accept a with probability

e-[E(S1)

[E(S1)-

• E(S2)]/(kb x T)

E(S2)]/(kb x T) – the metropolis condition

the metropolis condition p In the beginning (high temperature T) worse solutions are more In the beginning (high temperature T) worse solutions are more likely to get accepted; at the end (low T) it is less likely likely to get accepted; at the end (low T) it is less likely SA for optimization developed by Kirkpatrick/ SA for optimization developed by Kirkpatrick/Gelatt/Vecchi Gelatt/Vecchi (IBM) (IBM) When decreasing T infinitely slow, it can be shown that SA always finds When decreasing T infinitely slow, it can be shown that SA always finds the best solution the best solution

• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu

the best solution the best solution

4 ESII: SimAnneal

SA: Metropolis Loop SA: Metropolis Loop

Begin Choose some random initial configuration S; Repeat S’ := Some random initial configuration S;

( ) ( )

); ( ' S E S E E − = Δ ); , 1 min( :

T kB

e y Probabilit

Δ −

=

Until false Then If

y Probabilit random ≤ ) 1 , ( ; ' : S S =

End

• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu

5 ESII: SimAnneal

Generic SA Algorithm Generic SA Algorithm

Begin Begin S := Initial solution S0; T := Initial temperature T0; While stop criteria is not satisfied Do While stop criteria is not satisfied Do Begin While not yet in equilibrium Do Begin

( )

); ( ' S C S C C − = Δ ); 1 min( :

T

e y Probabilit

Δ −

=

Begin S’ := Some random neighboring solution of S; End U d t T Then If

y Probabilit random ≤ ) 1 , ( ; ' : S S = ); , 1 min( : e y Probabilit

Update T; End Output best solution;

• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu End

6 ESII: SimAnneal

Using SA with various constraints Using SA with various constraints

Fast temperature decrease, sufficient number of iterations Fast temperature decrease, sufficient number of iterations

1.2 0.6 0.8 1 ality 0.2 0.4 0 6 Qua 1 Time

• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu

7 ESII: SimAnneal

Using SA with various g constraints (cont’d)

Slow temperature decrease, not sufficient number of iterations (no Slow temperature decrease, not sufficient number of iterations (no equilibrium) equilibrium)

1.2 0 6 0.8 1 ality 0.2 0.4 0.6 Qua 1 Time

• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu

8 ESII: SimAnneal

Summary SA Summary SA

Simulates the annealing process of steel and interprets it Simulates the annealing process of steel and interprets it as an optimization process as an optimization process Features: Features:

Neighbouring solutions are explored in random order Neighbouring solutions are explored in random order New solutions can worsen or improve to previous one New solutions can worsen or improve to previous one New solutions can worsen or improve to previous one New solutions can worsen or improve to previous one Statistically controls acceptance of worse solutions Statistically controls acceptance of worse solutions Displays asymptotic convergence to the optimum Displays asymptotic convergence to the optimum

• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu

9 ESII: SimAnneal

References and Sources References and Sources

• [Jan03] The Electronic Design Automation Handbook, D. Jansen (Ed.), Kluwer 2003.
• [SpeC01] System Design: A practical guide with SpecC, A. Gerstlauer, R. Doemer, J. Peng, D. Gajski,

Kluwer, 2001.

• [Cai04] Estimation and Exploration Automation of System Level Design L Cai Ph D Dissertation UC
• [Cai04] Estimation and Exploration Automation of System Level Design, L. Cai, Ph.D Dissertation, UC

Irvine, 2004.

• [Gajski03], Transaction Level Modeling: An Overview, D. Gajski, L. Cai, Presented at IEEE/ACM

Codes-ISSS Conference, Newport Beach, CA, 2003.

• [Wolf01], Computers as Components, W. Wolf, Morgan Kaufmann Publishers, 2001.
• J. Henkel, M. Shafique, KIT, WS1011

http://ces.itec.kit.edu