Model-Based Evolutionary Algorithms Part 2: Linkage Tree Genetic - - PowerPoint PPT Presentation

1/29

Model-Based Evolutionary Algorithms Part 2: Linkage Tree Genetic Algorithm

Dirk Thierens

Universiteit Utrecht The Netherlands Joint work: Peter Bosman, CWI Amsterdam

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 1 / 29

2/29

MBEA

Evolutionary Algorithms

Population-based, stochastic search algorithms Exploitation: selection Exploration: mutation & crossover

Model-Based Evolutionary Algorithms

Population-based, stochastic search algorithms Exploitation: selection Exploration:

1

Learn a (probabilistic) model from selected solutions

2

Generate new solutions from the model (& population)

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 2 / 29

3/29

GOMEA

Gene-pool Optimal Mixing Evolutionary Algorithm

Population-based, stochastic search algorithms Exploitation: selection (by replacement) Exploration:

1

Learn a Family-Of-Subsets model

2

Generate new solutions through optimal mixing

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 3 / 29

4/29

GOMEA: design objectives

1

Be able to efficiently learn dependency information (= linkage) between variables

2

Be able to efficiently decide between competing building blocks

3

Transfer all optimal building blocks from the parents to the

• ffspring solution

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 4 / 29

5/29

Family Of Subsets (FOS) model

Key idea is to identify groups of problem variables that together make an important contribution to the quality of solutions. These variable groups are interacting in a non-linear way and should be processed as a block = building block

FOS F

Dependency structure generally called a Family Of Subsets (FOS). Let there be ℓ problem variables x0, x1, . . . , xℓ−1. Let S be a set of all variable indices {0, 1, . . . , ℓ − 1}. A FOS F is a set of subsets of the set S. FOS F is a subset of the powerset of S (F ⊆ P(S)).

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 5 / 29

6/29

Example Family Of Subsets (FOS) models:

Univariate FOS structure F = {{0}, {1}, {2}, {3}, {4}, {5}, {6}, {7}, {8}, {9}} Marginal Product FOS Structure F = {{0, 1, 2}, {3}, {4, 5}, {6, 7, 8, 9}} Linkage Tree FOS Structure F = {{7, 5, 8, 6, 9, 0, 3, 2, 4, 1}, {7, 5, 8, 6, 9}, {0, 3, 2, 4, 1}, {7}, {5, 8, 6, 9}, {0, 3, 2, 4}, {1}, {5, 8, 6}, {9}, {0, 3}, {2, 4}, {5, 8}, {6}, {0}, {3}, {2}, {4}, {5}, {8}}

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 6 / 29

7/29

Linkage Tree

Problem variables in subset are considered to be dependent on each other but become independent in a child subset. ≈ Path through dependency space, from univariate to joint. Linkage tree has ℓ leaf nodes (= single problem variables) and ℓ − 1 internal nodes.

x0 x1 x4 x3 x2 x5 x6 x8 x7 x9 x x

1

x x

5 6

x x

7 9

x x x

1 4 x x x 2 5 6

x x x

3 7 9

x x x x x x

1 2 4 5 6

x x x x

3 7 8 9

x x x x x x

1 2 3 4 5 6 7 8 9

x x x x

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 7 / 29

8/29

Linkage Tree Learning

Start from univariate structure. Build linkage tree using bottom-up hierarchical clustering algorithm. Similarity measure:

1

Between individual variables X and Y: mutual information I(X, Y). I(X, Y) = H(X) + H(Y) − H(X, Y)

2

Between cluster groups XFi and XFj: average pairwise linkage clustering (= unweighted pair group method with a arithmetic mean: UPGMA).

IUPGMA(XFi, XFj) = 1 |XFi||XFj|

• X∈XFi
• Y∈XFj

I(X, Y).

(H(X), H(Y), H(X, Y) are the marginal and joint entropies)

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 8 / 29

9/29

Linkage Tree Learning

This agglomerative hierarchical clustering algorithm is computationally efficient. Only the mutual information between pairs of variables needs to be computed once, which is a O(ℓ2) operation. The bottom-up hierarchical clustering can also be done in O(ℓ2) computation by using the reciprocal nearest neighbor chain algorithm. note: commonly used bottom-up hierarchical clustering algorithms (hclust and agnes in R) have O(ℓ3) complexity.

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 9 / 29

10/29

Optimal Mixing EA

FOS linkage models specify the linked variables. A subset of the FOS is used as crossover mask Crossover is greedy: only improvements (or equal) are accepted. Each generation a new FOS model is build from selected solutions. For each solution in the population, all subsets of the FOS are tried with a donor solution randomly picked from the population Recombinative OM (ROM) and Gene-pool OM (GOM)

◮ ROM is GA-like: select single donor solution to perform OM with ◮ GOM is EDA-like: select new donor solution for each subset in OM Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 10 / 29

11/29

Gene-pool Optimal Mixing EA

GOMEA() Pop ← InitPopulation() while NotTerminated(Pop) FOS ← BuildFOS(Pop) forall Sol ∈ Pop forall SubSet ∈ FOS Donor ← Random(Pop) Sol ← GreedyRecomb(Sol,Donor,Subset,Pop) return Sol GreedyRecomb(Sol,Donor,SubSet,Pop) NewSol ← ReplaceSubSetValues(Sol,SubSet,Donor) if ImprovementOrEqual(NewSol,Sol) then Sol ← NewSol return Sol

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 11 / 29

12/29

Recombinative Optimal Mixing EA

ROMEA() Pop ← InitPopulation() while NotTerminated(Pop) FOS ← BuildFOS(Pop) forall Sol ∈ Pop Donor ← Random(Pop) forall SubSet ∈ FOS Sol ← GreedyRecomb(Sol,Donor,Subset,Pop) return Sol GreedyRecomb(Sol,Donor,SubSet,Pop) NewSol ← ReplaceSubSetValues(Sol,SubSet,Donor) if ImprovementOrEqual(NewSol,Sol) then Sol ← NewSol return Sol

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 12 / 29

13/29

Optimal Mixing

Characteristic of Optimal Mixing is the use of intermediate function evaluations (inside variation) Can be regarded as greedy improvement of existing solutions Coined Optimal Mixing because better instances for substructures are immediately accepted and not dependent on noise coming from other parts of the solution Building block competition no longer a stochastic decision making problem that requires a sizable minimal population size Population sizes in GOMEA much smaller than in GAs or EDAs.

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 13 / 29

14/29

Linkage Tree Genetic Algorithm

The LTGA is an instance of GOMEA that uses a Linkage Tree as FOS model Each generation a new hierarchical cluster tree is build. For each solution in population, traverse tree starting at the top. Nodes (= clusters) in the linkage tree used as crossover masks. Select random donor solution, and its values at the crossover mask replace the variable values from the current solution. Evaluate new solution and accept if better/equal, otherwise reject.

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 14 / 29

15/29

Convergence model

Univariate FOS model on onemax problem

ℓ: string length n: population size p(t): proportion bit ’1’ at generation t q(t): proportion bit ’0’ at generation t Bit ’0’ only survive if parent and donor both have a ’0’ at that index: q(t + 1) = q2(t) p(t) = 1 − [1 − p(0)]2t

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 15 / 29

16/29

Number of function evaluations FE:

In 1 generation: FE = 2 p(t)([1 − p(t)] × ℓ × n After g generations: FE =

g

• t=0

2 p(t)([1 − p(t)] × ℓ × n After convergence gconv: FE = 2 [1 − p(0)] × ℓ × n Initial random population (p(0) = 0.5): FE = ℓ × n ⇒ O(ℓ log ℓ)

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 16 / 29

17/29

g

• t=0

p(t)([1 − p(t)] =

g

• t=0

q(t)([1 − q(t)] = q(0)[1 − q(0)] + q(1)[1 − q(1)] + · · · + q(g)[(1 − q(g)] = q(0) − q(1) + q(1) − q(2) + · · · − q(g) + q(g) − q2(g) = q(0) − [q(0)]2(g+1) gconv : → q(0)

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 17 / 29

18/29

Minimal population size

Need to have at least one bit ’1’ at each index: Prob[success] = [1 − (1 − p(0))n]ℓ ≈ 1 − ℓ [1 − p(0)]n 1 − 0.01 = 1 − ℓ [1 − 1 2]n n = log2(100ℓ) n = O(log ℓ)

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 18 / 29

19/29

Deceptive Trap Function

Interacting, non-overlapping, deceptive groups of variables. fDT(x) =

l−k

• i=0

f sub

DT

• x(i,...,i+k−1)
• Dirk Thierens (Universiteit Utrecht)

Model-Based Evolutionary Algorithms 19 / 29

20/29

Nearest-neighbor NK-landscape

Overlapping, neighboring random subfunctions fNK-S1(x) =

l−k

• i=0

f sub

NK

• x(i,...,i+k−1)
• with f sub

NK

• x(i,...,i+k−1)
• ∈ [0..1]
• eg. 16 subsfcts, length k = 5, overlap o = 4 ⇒ stringlength ℓ = 20

0 1 0 1 0 1 0 0 1 0 1 0 0 1 0 0 1 1 1 1 Global optimum computed by dynamic programming Benchmark function: structural information is not known ! ⇒ Randomly shuffled variable indices.

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 20 / 29

21/29

Experiments

Compare GA, EDA, and GOMEA while each are learning the Marginal Product (MP) FOS structure, and GOMEA learning the Linkage Tree (LT) as FOS structure. Note:

◮ EDA using MP = Extended Compact GA (ECGA). ◮ GOMEA using LT = Linkage Tree Genetic Algorithm (LTGA). ◮ hBOA = EDA learning a Bayesian network each generation. Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 21 / 29

22/29

Experiments - Onemax

0.001 0.01 0.1 1 10 100 1000 10000 25 50 100 200 400 800 1600 3200 6400 12800 25600 GA-UX EDA-UF GOMEA-U GA-MPMX EDA-MPM GOMEA-MPM GOMEA-LT hBOA

Time to optimum (seconds) Number of variables

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 22 / 29

23/29

Experiments - Deceptive trap

0.001 0.01 0.1 1 10 100 1000 10000 25 50 100 200 400 800 1600 3200 6400 GA-UX EDA-UF GOMEA-U GA-MPMX EDA-MPM GOMEA-MPM GOMEA-LT hBOA

Time to optimum (seconds) Number of variables

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 23 / 29

24/29

Experiments - Overlapping NK

0.001 0.01 0.1 1 10 100 1000 10000 25 50 100 200 400 800 1600 3200 GA-UX EDA-UF GOMEA-U GA-MPMX EDA-MPM GOMEA-MPM GOMEA-LT hBOA

Time to optimum (seconds) Number of variables

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 24 / 29

25/29

Experiments

20 40 60 80 100 10000 100000 1e+06 Successful runs (out of 100) Function evaluations [2..10] N=200 N=500

Figure: LTGA vs. ILS: 100 NK problems

Iterated Local Search: perturbation size each time randomly picked between 2 and 10 bits (= better than any fixed value).

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 25 / 29

26/29

Experiments - HIFF

0.001 0.01 0.1 1 10 100 1000 10000 32 64 128 256 512 1024 2048 4096 GA-UX EDA-UF GOMEA-U GA-MPMX EDA-MPM GOMEA-MPM GOMEA-LT hBOA

Time to optimum (seconds) Number of variables

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 26 / 29

27/29

Experiments - HTrap

0.001 0.01 0.1 1 10 100 1000 10000 27 81 243 729 2187 GA-UX EDA-UF GOMEA-U GA-MPMX EDA-MPM GOMEA-MPM GOMEA-LT hBOA

Time to optimum (seconds) Number of variables

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 27 / 29

28/29

Experiments - MAX-CUT 2D square grid

0.001 0.01 0.1 1 10 100 1000 10000 9 16 25 49 100 196 400 784 1600 GA-UX EDA-UF GOMEA-U GA-MPMX EDA-MPM GOMEA-MPM GOMEA-LT hBOA

Time to optimum (seconds) Number of variables

Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 28 / 29

29/29

Conclusions1

“Blind” Evolutionary Algorithms are limited in their capability to detect and exploit partial solutions (building blocks). Optimal Mixing Evolutionary Algorithms efficiently learn important building blocks and efficiently decide between competing building blocks Linkage Tree appears to be good compromise between FOS model complexity and search efficiency.

1http://homepages.cwi.nl/˜bosman/source code.php Dirk Thierens (Universiteit Utrecht) Model-Based Evolutionary Algorithms 29 / 29