From Quarks, Neutrinos and Neutron Stars to Evolutionary Algorithms - - PowerPoint PPT Presentation

From Quarks, Neutrinos and Neutron Stars to Evolutionary Algorithms

Stephen Friess

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 1

Outline

Part 1: Personal Background

• 1. Short Introduction
• 2. Optimization Problems in Theoretical Physics
• 3. Research Activities at the IKP TU Darmstadt
• 4. Summary and Personal Outlook

Part 2: Paper Presentation

• 1. Introduction to Evolutionary Algorithms
• 2. Negatively Correlated Search
• 3. Computational Studies with NCS-C
• 4. Discussion and Outlook

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 2

Short Introduction

Biographical Information:

◮ Name: Stephen Friess ◮ Born: 05.05.1990 ◮ Residence: Germany ◮ Degree: Master of Science (2016) ◮ Field: Theoretical Physics

Previous Fields of Research:

◮ Strong QCD and Nuclear Astrophysics.

Current Occupation:

◮ Software Developer and Consultant (GIP AG) ◮ Sector: Telecommunications (OSS systems)

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 3

Short Introduction

Working in Telecommunications:

◮ Focus: Operations Support Systems. ◮ IT Systems for Processing and Technical

Production of Telecommunication Services.

◮ Specifically: Fulfillment and Assurance

Processes for Layer-2/3 based Services.

◮ Also: Backend for a Conferencing System.

Further Education, Experiences and Achievements:

◮ 2017/18: Machine Learning MOOCs (Stanford University, etc.). ◮ 2017: IIoT Quest 2017 with startup idatase GmbH (1st place). ◮ 2015: Ludum Dare 33 & 34 Hackathons (1st and 2nd place @ KOM Lab). ◮ 2015: Teaching Assisstant for Computational Physics. ◮ Since 2013: Member of the German Physical Society (DPG eV).

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 4

Optimization Problems in Theoretical Physics

Motion of a Single Particle in Classical Mechanics:

◮ Task: Determine trajectory

x = x(t) from Newton’s Second Law: m ¨

• x =

F.

◮ Most forces

F can be derived from a potential energy V such that:

• F = −

∇V(

x).

◮ In this case follows the Law of Energy Conservation:

H = T + V = const. with kinetic energy T = 1/2m ˙

• x2

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 5

Optimization Problems in Theoretical Physics

◮ In Theoretical Physics we take the Legendre transform of H

L = p · ˙

• x − H with

p = 1 m

• ∇˙

xH

◮ We call L the Lagrangian. It can be rewritten as:

L = T − V.

◮ I.e.: The difference of kinetic and potential energy. ◮ In Theoretical Physics we introduce the scalar action S:

S[ x] =

t1

t0

dt L[ x, ˙

• x, t]

◮ Interpretation: Sum of energy differences from t0 → t1.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 6

Optimization Problems in Theoretical Physics

◮ Principle of Least Action: δS[

x] = 0

◮ Reformulation of particle motion as optimization

problem.

◮ We want to ”learn”

x(t) such that the action S[ x] is minimized.

◮ Equivalent Euler-Lagrange Equation: d

dt

∂ ∂ ˙

xi

− ∂ ∂xi

• L[

x, ˙

• x, t] = 0

◮ The Principle of Least Action and the Lagrange Formalism form the basis

• f Modern Theoretical Physics.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 7

Research Activities at the IKP TU Darmstadt

Research Focus of the Theory Center:

◮ Nuclear Physics and Nuclear Astrophysics ◮ I.e.: Matter under Extreme Conditions.

What do we learn from it:

◮ The Nature of the Nuclear Forces and

Interactions.

◮ The Astrophysical Effects of Nuclear Reaction

Theories. Previously associated Research Groups:

◮ Theoretical Nuclear Astrophysics Group (Master’s Thesis) ◮ Nuclei, Hadrons & Quarks Group (Bachelor’s Thesis)

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 8

Research Activities at the IKP TU Darmstadt

Bachelor’s Thesis at the NHQ Group:

◮ Focus: Investigation of Thermodynamic

Properties of the Quark-Gluon Plasma.

◮ Problem: Exact Lagrangian LQCD is

computationally difficult to access.

◮ Use of approximation schemes,

so called ’Effective Field Theories’.

LQCD ≈ ψ(i / ∂ − m)ψ + GS[(ψψ)2 + (ψiγ5 τψ)2]+GV(ψγµψ)2

My Research:

◮ Testing the performance of novel contributions to an existing theory, in

contrast to data from computationally ’exact’ calculations.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 9

Research Activities at the IKP TU Darmstadt

Master’s Thesis at the Theoretical Nuclear Astrophysics Group:

◮ Focus: Nucleosynthesis, Effects of Nuclear

Physics on Stellar Evolution and vice versa the Effects of Astrophysical Conditions on Nuclear Reactions.

◮ My Area of Research: Neutrino Reactions

in Proto-Neutron Stars.

◮ Neutrino reaction rates are specifically governed by matrix elements Mif

Mif ∼ Lweak = GF 2 HµLµ

◮ Thus, dependent upon a Lagrangian Lweak describing the reaction.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 10

Summary

A brief summary:

◮ The Principle of Least Action

reformulates Modern Theoretical Physics as an Optimization Problem.

◮ Consider a marble rolling down a hilly

landscape from point A to B. Naturally it takes the path of least action.

◮ Euler-Lagrange Equations arise as a direct consequence from this principle. ◮ Modern Theoretical Physics is concerned with the study of Lagrangians.

Especially their mathematical properties and observables arising from their structure.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 11

Personal Outlook

◮ An exciting time to be a scientist:

Progresses in mathematical theories and methods lead to technological innovations faster than ever.

◮ As a theoretical physicist:

I combine broad mathematical expertise and insight with an application-oriented mindset.

◮ My personal outlook:

By becoming a PhD student, I look forward to further advance my diverse set of skills and interdisciplinary interests and contribute in cutting-edge research with real-world applications.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 12

Outline

Part 1: Personal Background

• 1. Short Introduction
• 2. Optimization Problems in Theoretical Physics
• 3. Research Activities at the IKP TU Darmstadt
• 4. Personal Outlook

Part 2: Paper Presentation

• 1. Introduction to Evolutionary Algorithms
• 2. Negatively Correlated Search
• 3. Computational Studies with NCS-C
• 4. Discussion and Outlook

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 13

Introduction to Evolutionary Algorithms

General Concept (Simon 2013): An algorithm that evolves a problem solution over many iterations. There exists not a more precise and generally agreed upon definition. However,

• ne finds important and recurring ingredients (Eiben & Smith 2015):

◮ An objective function

which is minimized.

◮ A population of candidate

solutions with fixed size.

◮ A parent selection mechanism

and variation operators.

◮ A survivor selection

mechanism.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 14

Introduction to Evolutionary Algorithms

General scheme of an EA in pseudo code:

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 15

Introduction to Evolutionary Algorithms

Some important remarks:

◮ Evolutionary Algorithms are

non-deterministic and thus stochastic.

◮ Evolutionary Algorithms can

also be non-nature inspired.

◮ Natural evolution is in fact not an

• ptimizing process (e.g. genetic drifts).

Application Areas for Evolutionary Algorithms:

◮ Data Science: Data Analysis,

training of DNNs, tuning of SVMs.

◮ Technology & Engineering: Structural

Engineering, Antenna Design, etc.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 16

Negatively Correlated Search

Proposed by Ke Tang, Peng Yang and Xin Yao in 2016:

◮ Population-based search method utilizing

multiple Random Local Searches which are run in parallel. Distinguishing Feature of NCS:

◮ Inspired by cooperation in human behaviour. ◮ Information is shared among searching

’agents’ to encourage different searching behaviours which focus on uncovered regions.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 17

Negatively Correlated Search

General Framework:

◮ Each search process i is considered to be a RLS, which creates candidate

solutions using a probability distribution pi(x) as a generator of variance.

◮ To ensure variety in exploration, we need a measure

• f similarity between distributions pi and pj. Thus, we

use the Bhattacharyya distance as defined by: DB(pi, pj) = −ln

• dx
• pi(x)pj(x)
• ◮ Eventually, we want the Bhattacharyya distance from

pi to all pj to be maximal, or equivalently the term: Corr(pi) = min

j {DB(pi, pj)|j = i}

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 18

Negatively Correlated Search

◮ Further counting in the objective function f(x), we want to see for new

candidate solutions xi′: f(x′

i) minimized

Corr(p′

i ) maximized

◮ To facilitate a comparison between the quality and variety of a progenitor

solution xi and a successor solution x′

i of a search process, we normalize

f(x′

i) and Corr(p′ i ) such that:

f(xi) + f(x′

i ) = 1

Corr(pi) + Corr(p′

i ) = 1

◮ This allows us to introduce a parameter λ to control the exploration-

exploitation trade-off:

• discard xi,

if

f(x′

i )

Corr(pi) < λ

discard x′

i,

• therwise

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 19

Computational Studies with NCS-C

What is NCS-C:

◮ Implementation of NCS for Continuous Optimization.

Implementation Details of NCS-C:

◮ Candidate solutions are xi ∈ RD and new ones x′

i are generated from a

Gaussian mutation operator such that they are given by: x′

i,d = xi,d + N (0, σi)

◮ All σi are initialized with the same value across all dimensions and

dynamically updated according to:

σi =      σi/r,

if c/epoch > 0.2

σi · r,

if c/epoch < 0.2

σi,

if c/epoch = 0.2

◮ With r < 1, c being previous replacements and 0.2 chosen by convention.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 20

Computational Studies with NCS-C

◮ Further, the variance of the trade-off parameter λ is dynamically lowered to

accommodate for f(x′

i)/Corr(p′ i ) converging to 1:

λt = N (1, 0.1 − 0.1 ·

t Tmax ) Performance on CEC 2005 benchmark set:

◮ NCS-C was tested with 8 further evolutionary algorithms in the CEC 2005

benchmark set containing 20 multimodal continuous optimization problems.

◮ Results: No algorithm seemingly dominates the benchmark.

Performance on the statistical Friedman and Wilcoxon tests:

◮ Friedman-Test: NCS-C’s performance is however statistically the best. ◮ Wilcoxon-Test: Also statistically distinguishable better to individual algorithms.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 21

Computational Studies with NCS-C

◮ The performance-advantage can be visualized

by plotting the cumulative count of the K-th best ranking for each algorithm (right plot).

◮ NCS-C achieves dominance in the early onset,

• nly being competed by SaDE.

◮ Further inspection reveals: NCS-C performs

better on problems with many far apart local

• ptima, which require more exploration.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 22

Computational Studies with NCS-C

Case Study: Synthesis of Unequally Spaced Antenna Arrays

◮ NCS-C was further tested for the problem of

array pattern synthesis.

◮ Objective: Minimize the Peak Side-Lobe Levels, by

finding element positions x and excitation phases φ that minimize the maximum PSLL.

◮ Result: NCS-C leads to improvements for the synthesis

• f 32- and 37-element symmetric linear arrays.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 23

Discussion and Outlook

In Summary:

◮ NCS-C proved to be advantageous over traditional EAs in the CEC 2005

benchmark set and for a real-world problem.

◮ The advantage of the exploratory behaviour could be further demonstrated

and verified in a visualziation of search trajectories for F19. Possible NCS studies as suggested by Tang, Yang and Yao (2016):

◮ Further theoretical investigations of behaviours of NCS. ◮ Applying NCS to set-oriented optimization problems. ◮ Development of efficient techniques for estimating the Bhattacharyya distance.

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 24

Outline

Part 1: Personal Background

• 1. Short Introduction
• 2. Optimization Problems in Theoretical Physics
• 3. Research Activities at the IKP TU Darmstadt
• 4. Personal Outlook

Part 2: Paper Presentation

• 1. Introduction to Evolutionary Algorithms
• 2. Negatively Correlated Search
• 3. Computational Studies with NCS-C
• 4. Discussion and Outlook

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 25

Thank you for your attention!

June 26, 2018 | TU Darmstadt, IKP , Theoretical Nuclear Astrophysics Group | Stephen Friess | 26