Optimization Algorithms
- asst. prof. dr. Aleˇ
s Zamuda
Room B029, 14:00
- asst. prof. dr. Aleˇ
s Zamuda Optimization Algorithms 1/33
Optimization Algorithms asst. prof. dr. Ale s Zamuda Room B029, - - PowerPoint PPT Presentation
Optimization Algorithms asst. prof. dr. Ale s Zamuda Room B029, 14:00 asst. prof. dr. Ale s Zamuda Optimization Algorithms 1/33 Biography and Introduction: Organizations IEEE (Institute of Electrical and Electronics Engineers)
Room B029, 14:00
s Zamuda Optimization Algorithms 1/33
◮ IEEE (Institute of Electrical and Electronics Engineers)
◮ IEEE Computational Intelligence Society (CIS), member ◮ IEEE Young Professionals Slovenia, chair ◮ IEEE Senior Member
◮ Assistant Professor
s Zamuda, at University of Maribor, Slovenia
◮ Continuous research programme funded by Slovenian Research Agency,
P2-0041: Computer Systems, Methodologies, and Intelligent Services ◮ Associate Editor: Swarm and Evolutionary Computation (SWEVO) ◮ Slovenian Artificial Intelligence Society (part of EurAI) ◮ Co-operation in Science and Techology (COST) Association
Management Committee, member:
◮ CA COST Action CA15140: Improving Applicability of
Nature-Inspired Optimisation by Joining Theory and Practice (ImAppNIO)
◮ ICT COST Action IC1406 High-Performance Modelling and
Simulation for Big Data Applications (cHiPSet); SI-HPC
s Zamuda Optimization Algorithms 2/33
◮
differential evolution. Swarm and Evolutionary Computation, 2015, vol. 25C, pp. 72-99. DOI 10.1016/j.swevo.2015.10.007.
◮
andez Sosa, L. Adler. Constrained Differential Evolution Optimization for Underwater Glider Path Planning in Sub-mesoscale Eddy Sampling. Applied Soft Computing, 2016,
◮
andez Sosa. Differential Evolution and Underwater Glider Path Planning Applied to the Short-Term Opportunistic Sampling of Dynamic Mesoscale Ocean Structures. Applied Soft Computing, vol. 24, November 2014, pp. 95-108. DOI 10.1016/j.asoc.2014.06.048.
◮
Differential Evolution. Information Sciences, vol. 278, pp. 1-21, 2014. DOI 10.1016/j.ins.2014.04.037.
◮
Information Sciences, vol. 220, pp. 522-540, 2013. DOI 10.1016/j.ins.2012.07.031.
◮
c, A. Zamuda. Short-term combined economic and emission hydrothermal optimization by surrogate differential evolution. Applied Energy, 1 March 2015, vol. 141, pp. 42-56. DOI 10.1016/j.apenergy.2014.12.020.
◮
Mostaghim, R. Redpath, J. Timmis, F. Veenstra, M. Wahby and A. Zamuda. Hybrid Societies: Challenges and Perspectives in the Design of Collective Behavior in Self-organizing Systems. Frontiers in Robotics and AI, 2016, vol. 3, no. 14. DOI 10.3389/frobt.2016.00014.
◮
Silc, A. Zamuda. Special Issue on ”Bioinspired Optimization” (guest editors). Informatica - An International Journal of Computing and Informatics, 2015, vol. 39, no. 2, pp. 1-122.
s Zamuda Optimization Algorithms 3/33
Biography and Introduction: Reviewing Journals (Top 25 listed)
1. Applied energy (2015, 2016). London: Applied Science Publishers, 1975-. ISSN 0306-2619. 2. Artificial intelligence (2017). Amsterdam: Elsevier, 1970-. ISSN 0004-3702. 3. Applied mathematics and computation (2014). New York: Elsevier, 1975-. ISSN 0096-3003. 4. Applied soft computing (2013-2018). Amsterdam: Elsevier, 2001-. ISSN 1568-4946. 5. Computational optimization and applications (2013-2014). Dordrecht: Kluwer Academic Publishers, 1992-. ISSN 0926-6003. 6. Computational intelligence and neuroscience (2017). New York: Hindawi Publishing Corporation, 2006-. ISSN 1687-5265. 7. Engineering applications of artificial intelligence (2010, 2011, 2015-2016). Swansea: Pineridge Press, 1988-. ISSN 0952-1976. 8. European journal of operational research (2013, 2016). Amsterdam: North-Holland, 1977-. ISSN 0377-2217. 9. IEEE Journal of oceanic engineering (2017). New York: IEEE, Council on Oceanic Engineering, 1976-. ISSN 0364-9059. 10. IEEE Computational intelligence magazine (2012). New York (NY): IEEE, 2006-. ISSN 1556-603X. 11. IEEE Transactions on automation science and engineering (2016). New York (NY): Institute of Electrical and Electronics Engineers, 2004-. ISSN 1545-5955. 12. Expert systems with applications (2017). Oxford: Pergamon, 1990-. ISSN 0957-4174. 13. IEEE Transactions on Cybernetics (2013-2018). New York (NY): Institute of Electrical and Electronics Engineers, 2013-. ISSN 2168-2267. 14. IEEE Transactions on evolutionary computation (2007-2018). New York: Institute of Electrical and Electronics Engineers, 1997-. ISSN 1089-778X. 15. IEEE Transactions on industrial electronics (2009, 2013, 2015). New York: Institute of Electrical and Electronics Engineers, 1982-. ISSN 0278-0046. 16. IEEE Transactions on systems, man and cybernetics. Part B. Cybernetics (2009, 2010, 2012, 2013). New York (NY): Institute
17. Information sciences (2011-2016). New York: North-Holland, 1968-. ISSN 0020-0255. 18. International Journal of Systems Science (2010, 2011). London: Taylor & Francis. ISSN 0020-7721. 19. Memetic computing (2012, 2018). Heidelberg; Berlin: Springer. ISSN 1865-9292. 20. Natural computing (2015-2016, 2018). Dordrecht; Boston; London: Kluwer Academic Publishers. ISSN 1567-7818. 21. Neural computing & applications (2014, 2016-2018). London: Springer, 1993-. ISSN 0941-0643. 22. Neurocomputing (2016). Amsterdam: Elsevier, 1989-. ISSN 0925-2312. 23. Remote sensing (2015). [Online ed.]. Basel: Molecular Diversity Preservation International, 2009-. ISSN 2072-4292. 24. Soft computing (2013, 2015-2017). Heidelberg: Springer, 1997-. ISSN 1432-7643. 25. Swarm and evolutionary computation (2011-2018). Amsterdam: Elsevier. ISSN 2210-6502.
s Zamuda Optimization Algorithms 4/33
Biography and Introduction: Conferences Committees (Top 25 listed)
1. AR4MET (Bali: 2015; Batam 2016): Advanced Research in Material Sciences, Manufacturing, Mechanical and Mechatronic Engineering Technology International Conference 2. BIOMA (Ljubljana: 2014, 2016; Paris: 2018): International Conference on Bioinspired Optimization Methods and their Applications 3. CEC (Singapore: 2007; Hong Kong: 2008; Trondheim: 2009; Barcelona: 2010; New Orleans: 2011; Vancouver: 2016; San Sebastian 2017): IEEE Congress on Evolutionary Computation 4. CSOC (On-line: 2015, 2016): Computer Science On-line Conference 5. ERK (Portorose: 2010, 2016): Electrotechnical and Computer Science Conference 6. EUROGEN (Glasgow: 2015; Madrid: 2017): International Conference on Evolutionary and Deterministic Methods for Design, Optimization and Control with Applications to Industrial and Societal Problems 7. FUTURECOMP (Venice: 2014; Nice: 2015; Rome: 2016): Int. Conf. on Future Computational Technologies and Applications 8. GECCO (Berlin, 2017; Kyoto 2018): The Genetic and Evolutionary Computation Conference 9. IACIET (Jaipur: 2014): International Conference on “Innovations in Engineering & Technology” 10. ICACCI (Greater Noida: 2014; Aluva: 2015; Jaipur: 2016): International Conference on Advances in Computing, Communications and Informatics 11. ICAISC (Zakopane: 2014, 2015, 2016): International Conference on Artificial Intelligence and Soft Computing 12. ICEM (Rome: 2010; Marseille: 2012): International Conference on Electrical Machines 13. ICIA (Pillaichavadi: 2016): International Conference on Informatics and Analytics 14. ICIT (Dubai: 2014): [ScienceOne International Conference on Information Technology] 15. ICOA (2015): International Conference on Oceanic and Atmospheric 16. IPIC (Orlando: 2015): Symposium on Enabling an Intelligent Planet via Informatics and Cybernetics 17. ISNN (Saint Petersburg: 2016): International Symposium on Neural Networks 18. IWSSIP (Maribor: 2018): International Conference on Systems, Signals and Image Processing 19. PPSN (Ljubljana: 2014; Edinburgh: 2016; Coimbra: 2018): International Conference on Parallel Problem Solving From Nature 20. SASO FAS*W SOCO (Augsburg, 2016): IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO), including International Workshops on Foundations and Applications of Self-* Systems (FAS*W), including International Workshop on Self-Organising Construction (SOCO) 21. SEMCCO (Chennai: 2010, 2011): International Conference on Swarm, Evolutionary and Memetic Computing 22. SIRS (Trivandrum: 2015): nternational Symposium on Signal Processing and Intelligent Recognition Systems 23. SPICES (2015): IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems 24. SSCI SDE (Paris: 2011; Singapore: 2013; Orlando: 2014; Cape Town: 2015; Athens 2016): IEEE Symposium Series on Computational Intelligence, Symposium on Differential Evolution 25. WMSCI (Orlando: 2015, 2016, 2018): World Multi-Conference on Systemics, Cybernetics And Informatics
s Zamuda Optimization Algorithms 5/33
Awards: IEEE R8SPC 2007, SIIF Seoul 2012, ESA, Danubius Young Scientist
DYS awarded throughout all scientific disciplines, nation-wide to one scientist from 12 IDM countries, each Other Top 30 Awards won listed at: http://aleszamuda.si
s Zamuda Optimization Algorithms 6/33
Intelligence
s Zamuda Optimization Algorithms 7/33
Biography and Introduction OPTIMIZATION ALGORITHMS: DIFFERENTIAL EVOLUTION CHALLENGE 1 TREE MODEL RECONSTRUCTION CHALLENGE 2 HYDRO AND THERMAL POWER PLANT SCHEDULING AND MORE REAL-WORLD CHALLENGES CHALLENGE 3 UNDERWATER GLIDER PATH PLANNING Lesson 5: MORE REAL WORLD INDUSTRY CHALLENGES Conclusion
s Zamuda Optimization Algorithms 8/33
1: algorithm canonical algorithm DE/rand/1/bin (Storn, 1997)
Require: f(x) – fitness function; D, NP, G – DE control parameters Ensure: xbest – includes optimized parameters for the fitness function
2: Uniform randomly initialize the population (xi,0, i = 1..NP); 3: for DE generation loop g (until g < G) do 4:
for DE iteration loop i (for all vectors xi,g in current population) do
5:
DE trial vector computation xi,g (mutation, crossover):
6:
vi,g+1 = xr1,g + F × (xr2,g − xr3,g);
7:
ui,j,g+1 =
if rand(0, 1) ≤ CR or j = jrand xi,j,g
;
8:
DE selection using fitness evaluation f (ui,G+1):
9:
xi,g+1 =
if f (ui,g+1) < f (xi,g) xi,g
;
10:
end for
11: end for 12: return best obtained vector (xbest);
s Zamuda Optimization Algorithms 9/33
◮ Randomization frequency
influences performance (SPSRDEMMS on right)
◮ Suggesting values for
different problems
◮ 0.1 to 0.8 for τF,
0.05 to 0.25 for τCR
◮ Empirical insight into
randomization mechanism
s Zamuda Optimization Algorithms 10/33
Biography and Introduction OPTIMIZATION ALGORITHMS: DIFFERENTIAL EVOLUTION CHALLENGE 1 TREE MODEL RECONSTRUCTION CHALLENGE 2 HYDRO AND THERMAL POWER PLANT SCHEDULING AND MORE REAL-WORLD CHALLENGES CHALLENGE 3 UNDERWATER GLIDER PATH PLANNING Lesson 5: MORE REAL WORLD INDUSTRY CHALLENGES Conclusion
s Zamuda Optimization Algorithms 11/33
s Zamuda Optimization Algorithms 12/33
1: procedure MO reconstruction(z∗) Require: S0 - maximum number of strands in base branch; also, other default parameters for MOjDE and EcoMod Ensure: Pareto set of reconstructed parameterized procedural 3D woody plant models 2: uniform randomly generate DE initial population xi,0 ∈ [0, 1] for i = 1..NP; 3: for DE generation loop g (while FEs < 10000) do 4: for DE iteration loop i (for all individuals xi,g of a population) do 5: DE individual xi,g creation (adaptation, mutation, crossover): 6: Fi,G+1 =
if rand2 < τ1, Fi,G
; CRi,G+1 =
if rand4 < τ2, CRi,G
; 8: vi,G+1 = xr1,G + Fi,G+1(xr2,G − xr3,G ); 9: ui,j,G+1 =
if rand(0, 1) ≤ CRi,G+1 or j = jrand xi,j,G
; 10: DE fitness evaluation (genotype-phenotype mapping, rendering, and comparison): 11: z1 = g(ui,g , β1), z2 = g(ui,g , β2) {Execute Algorithm branchsegment twice} 12: h1(z1) =
x,y m1(z1 x,y , z∗ x,y ) + x,y m1(z∗ x,y , z1 x,y ); {First difference metric, at 0◦}
13: h1(z2) =
x,y m1(z2 x,y , z∗ x,y ) + x,y m1(z∗ x,y , z2 x,y ); {First difference metric, at 90◦}
14: f1(x) = f (g(x, β1), g(x, β2)) = h1(z1) + h1(z2); {Fitness evaluation, 1st criterion} 15: h2(z1) =
x,y w(z1 x,y , z∗ x,y ) + x,y w(z∗ x,y , z1 x,y ); {Second difference metric, 0◦}
16: h2(z2) =
x,y w(z2 x,y , z∗ x,y ) + x,y w(z∗ x,y , z2 x,y ); {Second difference metric, 90◦}
17: f2(x) = f (g(x, β1), g(x, β2)) = h2(z1) + h2(z2); {Fitness evaluation, 2nd criterion} 18: f(x) = {f1(x), f2(x)}; {Fitness evaluation, all criteria combined done} 19: DE selection: 20: xi,G+1 =
if f(ui,G+1) f(xi,G ) xi,G
; {Multi-objective comparison operator} 21: if not (ui,G+1 xi,G or xi,G ui,G+1 ) then add ui,G+1 to population archive; 22: end for 23: Truncate DE population archive to a size of NP using SPEA2 mechanism. 24: end for 25: return the best individuals obtained;
s Zamuda Optimization Algorithms 13/33
1: procedure branchsegment(g, w, S0, L0, l0, M0, M−1
m;0)
Require: g, w - Gravelius and Weibull index of base branch; S0 - number of strands in base branch; L0, l0 - base branch relative and actual length; M0 - base branch coordinate system; M−1
m;0 - inverse matrix of rotations
for gravimorphism in coordinate system for base branch; global (i.e. part of breeder) kd , kc , ltype, kg,w
s
, Mg,w , mg,w , kg,w
l
, αg,w
m
, αg,w , t, kf , ws, wg Ensure: rendered tree image 2: d := kd
3: render base branch(M0, l0, d); 4: if S0 = 1 then 5: render leaves(ltype); return; 6: end if 7: S1 :=
s
(S0 − 2)
8: r1 := max
S0 , Mg,w
9: r2 := max
S0 , Mg,w
10: L1 := r1L0, L2 := r2L0; {relative lengths of subbranches} 11: l1 := kg,w
l
L1, l2 := kg,w
l
L2; {active subbranch lengths} 12: α1 := kc
S0 αg,w , α2 := αg,w − α1; {branching angles}
13: M1 := Rz (α1)Ry (αp)Ry×ym (αg,w
m
)Ty (l0)M0; {transform} 14: M2 := Rz (α2)Ry (αp)Ry×ym (αg,w
m
)Ty (l0)M0; 15: M−1
m;1 := Ry×ym (−αg,w m
)Ry (−αp)Rx (−αx (t))Rz (−α1 − αz (t))M−1
m;0; {refreshing inverse matrix}
16: M−1
m;2 := Ry×ym (−αg,w m
)Ry (−αp)Rx (−αx (t))Rz (−α2 − αz (t))M−1
m;0;
17: branchsegment(g + 1, w + 1, S2, L2, l2, M2, M−1
m;2); {minor branch development}
18: branchsegment(g, w + 1, S1, L1, l1, M1, M−1
m;1); {major branch development}
19: return; {from recursive procedure call}
s Zamuda Optimization Algorithms 14/33
◮ 3D tree models are compactly represented using a procedure
◮ our EcoMod framework uses a numerically coded procedural model
with fixed dimensionality
◮ suitable for parameter estimation using DE/MOjDE.
◮ Parameterized procedural model builds a 3D structure of a tree and
all its building parts:
◮ by recursively executing a fixed procedure, ◮ over a set of numerically coded input parameters,
◮ e.g. branch thickness, relative length, and branching.
s Zamuda Optimization Algorithms 15/33
Biography and Introduction OPTIMIZATION ALGORITHMS: DIFFERENTIAL EVOLUTION CHALLENGE 1 TREE MODEL RECONSTRUCTION CHALLENGE 2 HYDRO AND THERMAL POWER PLANT SCHEDULING AND MORE REAL-WORLD CHALLENGES CHALLENGE 3 UNDERWATER GLIDER PATH PLANNING Lesson 5: MORE REAL WORLD INDUSTRY CHALLENGES Conclusion
s Zamuda Optimization Algorithms 16/33
c, A. Zamuda. Short-term combined economic and emission hydrothermal optimization by surrogate differential evolution. Applied Energy, 1 March 2015, vol. 141, pp. 42-56. DOI 10.1016/j.apenergy.2014.12.020. IF=5.613
s Zamuda Optimization Algorithms 17/33
Fitness Function and Practical Accuracy (discretized)
s Zamuda Optimization Algorithms 18/33
Surrogate Matrix – Input and Output
s Zamuda Optimization Algorithms 19/33
Biography and Introduction OPTIMIZATION ALGORITHMS: DIFFERENTIAL EVOLUTION CHALLENGE 1 TREE MODEL RECONSTRUCTION CHALLENGE 2 HYDRO AND THERMAL POWER PLANT SCHEDULING AND MORE REAL-WORLD CHALLENGES CHALLENGE 3 UNDERWATER GLIDER PATH PLANNING Lesson 5: MORE REAL WORLD INDUSTRY CHALLENGES Conclusion
s Zamuda Optimization Algorithms 20/33
◮ High durability: 25 to 365 days, ◮ long range 600–1500 km (alk. batt.), 4000–6000 km (Li+)
◮ buoyancy-driven: horizontal 0.35m/s (0.68 knots), ◮ 2 knots using propeller.
◮ Dive to depth 1000 meters, long range, modular, ◮ integrates sensors of physical and bio/chemical parameters2
◮ temperature, salinity, dissolved oxygen, turbidity, chlorophyl
and sea currents - possible rapid replacement of sensors.
1 lh6.googleusercontent.com/-Mq308aI1s2g/UHVf4k3uoiI/AAAAAAAACbw/LeiYHXMQRbs/s640/PA060013.JPG 2 http://www.webbresearch.com/pdf/Slocum_Glider_Data_Sheet.pdf
s Zamuda Optimization Algorithms 21/33
◮ Driving ”yoyo” uses little energy, most only on descent and
rise (pump); also for maintaining direction little power is consumed. + Use: improving ocean models with real data, + the real data at the point of capture, + sampling flow of oil discharges, + monitoring cable lines, and + real-time monitoring of different sensor data.
1 http://www.i-cool.org/wp-content/uploads/2009/11/google-earth-glider-path.jpg 2 http://spectrum.ieee.org/image/1523708
s Zamuda Optimization Algorithms 22/33
https://www.google.si/maps/@28.059806,-15.998355,650054m/data=!3m1!1e3
s Zamuda Optimization Algorithms 23/33
◮ Corridor-constrained optimization:
eddy border region sampling
◮ new challenge for UGPP & DE
◮ Feasible path area is constrained
◮ trajectory in corridor around the
border of an ocean eddy The objective of the glider here is to sample the oceanographic variables more efficiently, while keeping a bounded trajectory
s Zamuda Optimization Algorithms 24/33
s Zamuda Optimization Algorithms 25/33
s Zamuda Optimization Algorithms 26/33
Biography and Introduction OPTIMIZATION ALGORITHMS: DIFFERENTIAL EVOLUTION CHALLENGE 1 TREE MODEL RECONSTRUCTION CHALLENGE 2 HYDRO AND THERMAL POWER PLANT SCHEDULING AND MORE REAL-WORLD CHALLENGES CHALLENGE 3 UNDERWATER GLIDER PATH PLANNING Lesson 5: MORE REAL WORLD INDUSTRY CHALLENGES Conclusion
s Zamuda Optimization Algorithms 27/33
◮ Storn, Rainer, and Kenneth V. Price. ”Minimizing the Real Functions of
the ICEC’96 Contest by Differential Evolution.” International Conference on Evolutionary Computation. 1996.
◮ ... ◮ CEC 2005 Special Session / Competition on
Evolutionary Real Parameter single objective optimization
◮ CEC 2006 Special Session / Competition on
Evolutionary Constrained Real Parameter single objective optimization
◮ CEC 2007 Special Session / Competition on
Performance Assessment of real-parameter MOEAs
◮ CEC 2008 Special Session / Competition on
large scale single objective global optimization with bound constraints
◮ CEC 2008 Scale-Invariant Optimisation Competition ”Mountains or Molehills” ◮ CEC 2009 Special Session / Competition on
Dynamic Optimization (Primarily composition functions were used)
◮ CEC 2009 Special Session / Competition on
Performance Assessment of real-parameter MOEAs
s Zamuda Optimization Algorithms 28/33
◮ CEC 2010 Special Session / Competition on
large-scale single objective global optimization with bound constraints
◮ CEC 2010 Special Session / Competition on
Evolutionary Constrained Real Parameter single objective optimization
◮ CEC 2010 Special Session on
Niching Introduces novel scalable test problems
◮ CEC 2011 Competition on Testing Evolutionary Algorithms on Real-world
Numerical Optimization Problems
◮ CEC 2013 Special Session / Competition on
Real Parameter Single Objective Optimization
◮ CEC 2014 Special Session / Competition on
Real Parameter Single Objective Optimization (incorporates expensive functions)
◮ CEC 2014: Dynamic MOEA Benchmark Problems
s Zamuda Optimization Algorithms 29/33
◮ CEC 2015 Special Session / Competition on
Real Parameter Single Objective Optimization (incorporates 3 scenarios)
◮ CEC 2015 Black Box Optimization Competition ◮ CEC 2015 Dynamic Multi-Objective Optimization ◮ CEC 2015 Optimization of Big Data ◮ CEC 2015 Large Scale Global Optimization ◮ CEC 2015 Bound Constrained Single-Objective Numerical Optimization ◮ CEC 2015
Optimisation of Problems with Multiple Interdependent Components
◮ CEC 2015 Niching Methods for Multimodal Optimization
s Zamuda Optimization Algorithms 30/33
◮ CEC 2016 Special Session / Competition on
Real Parameter Single Objective Optimization (incorporates 4 scenarios)
◮ CEC 2016 Big Optimization (BigOpt2016) ◮ CEC 2016 Niching Methods for Multimodal Optimization ◮ CEC 2016 Special Session Associated with Competition on
Bound Constrained Single Objective Numerical Optimization
s Zamuda Optimization Algorithms 31/33
Biography and Introduction OPTIMIZATION ALGORITHMS: DIFFERENTIAL EVOLUTION CHALLENGE 1 TREE MODEL RECONSTRUCTION CHALLENGE 2 HYDRO AND THERMAL POWER PLANT SCHEDULING AND MORE REAL-WORLD CHALLENGES CHALLENGE 3 UNDERWATER GLIDER PATH PLANNING Lesson 5: MORE REAL WORLD INDUSTRY CHALLENGES Conclusion
s Zamuda Optimization Algorithms 32/33
s Zamuda Optimization Algorithms 33/33