Getting Lost or Getting Trapped: On the Effect of Moves to - - PowerPoint PPT Presentation

Introduction Analysis and Tempering Summary/Conclusions/What next?

Getting Lost or Getting Trapped: On the Effect

• f Moves to Incomparable Points in

Multiobjective Hillclimbing

Workshop on Theoretical Aspects of Evolutionary Multiobjective Optimization - Current Status and Future Trends

GECCO

Michael Emmerich, Andr´ e Deutz, Johannes Kruisselbrink, Rui Li July 8, 2010

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Getting Lost or Getting Trapped: On the Effect

• f Moves to Incomparable Points in

Multiobjective Hillclimbing

Workshop on Theoretical Aspects of Evolutionary Multiobjective Optimization - Current Status and Future Trends

GECCO

Michael Emmerich, Andr´ e Deutz, Johannes Kruisselbrink, Rui Li July 8, 2010

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Table of Contents

Introduction: EA Moves to Incomparable Solutions Analysis of Algorithms and Taming Single Point Schemes

(1+1)-IMEA restricting incomparibility, tempering divergence (1+1)-LIMEA

Population Based Schemes

divergent behavior (2+2)-IMEA Restricting Incomparability, Tempering Divergence for Population Based EAs commonly used EAs

Summary/Conclusions/What next?

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Search Algorithms on (Strict) Partial Orders

◮ In approximating Pareto Fronts EAs very often allow moves to

incomparable solutions

◮ Incomparability relationship between solutions is not transitive ◮ Study questions associated with moves to incomparable

solutions

◮ Divergent or cycling behavior of algorithms with incomparable

moves

◮ Can occur in elitist schemes which disallow moves to

dominated solutions

◮ Measures to counteract this?

◮ Exploitation versus exploration; tension between divergence

and exploitation

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Edgeworth-Pareto Dominance

◮ (Minimization) The point p (Pareto) dominates any green

point I.e., it is strictly better in at least one of the two

• bjectives

◮ Aka: p strictly dominates any point in the green area. ◮ Notation: ≺. ◮ Is a strict partial order (irreflexive, asymmetric and transitive).

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Weak Dominance

◮ p weakly dominates any green point ◮ I.e., ((p strictly dominates any green point except itself ) and

(p dominates itself))

◮ I.e., p is better or equal to any green point in all objectives. ◮ Notation: . ◮ Is a partial order (reflexive, antisymmetric, and transitive

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Incomparibility

The point p and any blue point are incomparable – is not transitive. Notation:

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Dominated

Any grey point (and the point p) (weakly) dominates p

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Summary

◮ p weakly dominates any green point and itself ◮ Any grey point (and the point p) (weakly) dominates p ◮ Weak dominance is a p.o. ◮ Any blue point and p are incomparable (is intransitive)

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Remark on terminology

Aside: Strict dominance and weak dominance are terms used by J.Bader, D.Brockhoff, S.Welten, and E.Zitzler in On Using Populations of Sets in Multiobjective Optimization, EMO 2009, LNCS5467

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

dominance induced by a map and partial order on the codomain of the map

f : A ⊆ Rn → Rm then f induces an important binary relation which is reflexive and transitive (and in general not antisymmetric)

• n Rn as follows x, x′ ∈ A:

x x′ ⇔ f(x) f(x) (For that matter any partial order on a set B and a map f : A → B, give rise to a binary relation which is reflexive and transitive on A. (In general this induced order is not a partial order (no anti-symmetry).)) Can define in the usual way Pareto Front, Efficient Set etc etc

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Getting worse in case of moves to incomparable solutions I

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Getting worse in case of moves to incomparable solutions II

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Getting worse in case of moves to incomparable solutions III

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Zooming in on Algorithms and Taming Divergence

◮ Single point analysis:

◮ (1+1)-IMEA (Incomparable Move Evolutionary Algorithm) ◮ (1+1)-LIMEA (only certain incomparable moves are allowed

via utility function)

◮ Population based methods:

◮ (2+2)-IMEA ◮ Tempering influence of incomparability for population based

schemes

◮ NSGA-II ◮ SMS-EMOA Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

(The) Example Problem, Model Landscape

We will use the following 2D multiobjective problem to study the behavior of (1+1)-IMEA, (1+1)-LIMEA, (2+2)-IMEA

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

Algorithm 1 (1 + 1)-IMEA input: x0, output: xt, t = 1, 2, . . . t = 1 while t < Tmax do q = mutate(xt−1) For Selection See Right Column

• utput xt

t = t + 1 end while if q ≺ xt−1 then xt = q else if xt−1 ≺ q then xt = xt−1 else if xt−1 q or xt−1 ∼ q then xt = UniformRandom{ q, xt−1 } end if end if end if

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

Transition Matrix, K,of the MC of the Example Problem and the (1+1)-IMEA

x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 x12 x1

2 3 1 6 1 6

x2

1 8 5 8 1 8 1 8

x3

1 8 5 8 1 8 1 8

x4

1 6 2 3 1 6

x5

1 6 2 3 1 6

x6

1 10 1 10 3 5 1 10 1 10

x7

1 8 1 8 5 8 1 8

x8

1 8 1 8 5 8 1 8

x9

1 10 1 10 3 5 1 10 1 10

x10

1 6 2 3 1 6

x11

1 6 2 3 1 6

x12

1 6 2 3 1 6

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

limit behavior of the (1+1)-IMEA on the example problem

◮ The MC specified by the transition matrix K is regular ◮ Fixed row tuple w of K (i.e, w such that w = wK) is a

strictly positive probability vector and theory tells you wj gives the probability of being in xj in the long run (where wj is the j-th entry of w), and it is independent of the starting state.

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

Hence, ending up in the Pareto Front has a chance of roughly 21%

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

How to deal with incomparability?

◮ We have two extremes: disallow moves to incomparable

solutions (if the child dominates the parent, it is chosen,

• therwise choose the parent) and on the other hand if parent

and child are incomparable can possibly choose the child.

◮ Of course, the first extreme entails that you cannot diverge

from optimal solutions, but it has the well-known disadvantage of needing to overcome traps.

◮ Suggestion: middle of the road; allow for exploration, that is,

allow for some moves to incomparable solutions

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

How to deal with incomparability? I

◮ Work with a utility function with constant positive weights,

wi, i = 1, . . . , m; wi > 0, that is, x ≺w x′ iff

m

• i=1

wifi(x) <

m

• i=1

wifi(x′)

◮ Respects the Pareto dominance

x ≺ x′ ⇒ x ≺w x′

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

How to deal with incomparability? II

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

Invariance Property

Can select any linear or non-linear utility function u that obeys the monotonicity property: x ≺ x′ ⇒ u(x) < u(x′) (1) Lemma 1: Given an alternative selection scheme, that disallows moves to solutions q when u(q) > u(xt−1) (in (1+1)-IMEA) for a monotone u, it cannot occur that xt ≺ xt+i for i > 0.

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Single Point Schemes

(1+1)-LIMEA

Modify (1+1)-IMEA by using a linear utility function with uw with w1 = w2 = 1. Selection scheme: accept a child if and only if it is better or equal in uw to its parent. What is the dynamics on our model landscape?

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Population Based Schemes

Population Based Scheme

Algorithm 2 (µ + λ)-EA input: P0, with |P0| = µ

• utput: Pt, t = 1, 2, . . .

t = 1 while t < Tmax do Q = variate(Pt−1), |Q| = λ Pt = select-best (Pt−1 ∪ Q), |Pt| = µ

• utput Pt

t = t + 1 end while

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Population Based Schemes

Divergence for Population Based EAs

To define divergence for a series of populations, let us first define the following (strict) partial order on populations (see also Zitzler et al, Proceedings EMO 2007): Definition: Let min(P) denote the minimal subset of a population P with respect to the Pareto order. Then P ⊳ P′ (or P is better than P′), if and only if ∀x′ ∈ P′ : ∃x ∈ P : x x′ and ∃x′ ∈ min(P′) : ∃x ∈ P : x ≺ x′. Divergent behavior in population based EA occurs, when a population Pt at a time step t evolves to a population Pt+i at time step t + i for some i > 0, such that Pt ⊳ Pt+i.

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Population Based Schemes

Population Based Scheme: (2+2)-IMEA

◮ In Algorithm 2 selection is not specified yet. By doing this and

setting λ = µ = 2 we get (2+2)-IMEA.

◮ First variation: each parent uniformly randomly moves to one

• f its neighbors (thus no mutation to itself).

◮ Selection: from the (multi-)set of two off-spring and two

parents choose the two best with non-dominationg sorting. In all undecided cases uniformly randomly one of the candidates with equal dominance rank.

◮ In the associated Markov Chain represent states conceptually

as multi-sets and for computations of the transition matrix by a program as ordered pairs.

◮ Dynamics on the model problem?

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Population Based Schemes

Dynamics of the (2+2)-IMEA

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Population Based Schemes

Not disallowing all moves to incomparable solutions to tame divergence

◮ Only certain moves to incomparable solutions can prevent

divergent behavior?

◮ One way: use a set utility function U on the population level

with the monotonicity property P′ ⊳ P ⇒ U(P′) > U(P) (2)

◮ Example: hypervolume indicator ◮ Invariance property: Algorithms that possess the invariance

property U(Pt) ≤ U(Pt+i) for a monotone population-level utility function U, cannot exhibit divergent behaviour (as defined earlier).

◮ SMS-EMOA has this invariance. What about NSGA-II?

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next? Population Based Schemes

Possibly a divergent scenario for NSGA-II

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Summary I

◮ Divergent behavior can happen in existing algorithms.

Moreover one may expect that in a plus strategy, such as the NSGA-II, this cannot occur

◮ Maybe maintain a true elitist archive for NSGA-II (e.g. the

hypervolume based archive by Knowles et al Bounded Archiving Using the Lesbesgue Measure, CEC 2003) though

• n first glance one may not think this is not needed?

◮ Of course, divergent behavior is not necessarily bad. Need to

study how it relates to exploration power. In our first example we saw that divergence was harmful but in general divergence is likely to support exploration.

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Summary II

◮ Results on our model problem suggests that the search may

not even favor optimal(non-dominated) regions, in case of moves to incomparable offspring

◮ Enlarging the population size seems counteract this

◮ Disallowing moves to incomparable children avoids divergence

but now you have to deal with traps

◮ Using monotone (set-based) utitility functions on individual

(and population) level in the selection to allow for certain incomparable moves, prevents divergent behavior

◮ This seems to be achieved at a decrease of exploration power

◮ Common elitist approaches such as the NSGA-II exhibit

divergent behavior

◮ Open questions: how likely is this to occur? And what is the

effect of population size on divergence?

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

Summary III

◮ Future work: look at relation between landscapes and

algorithms’ tendency to diverge or get trapped; additional ways to prevent divergence such as elitist schemes of G. Rudolph and A. Agapie Convergence Properties of Some Multi-Objective Evolutionary Algorithms, CEC 2000; possibly working on the level of sets will give methods to control divergence (Bader, Brockhoff, Welten and Zitzler On Using Populations of Sets in Multiobjective Optimization in LNCS 5467)

Natural Computing Leiden University, The Netherlands

Introduction Analysis and Tempering Summary/Conclusions/What next?

The End Thank you! ¿Questions?

Natural Computing Leiden University, The Netherlands