Evolving Grammars: A Structured Point of View Nuno Loureno - - PowerPoint PPT Presentation

▶
evolving grammars a structured point of view
SMART_READER_LITE
LIVE PREVIEW

Evolving Grammars: A Structured Point of View Nuno Loureno - - PowerPoint PPT Presentation

Sep 22, 2023 453 likes •889 views

1 Evolving Grammars: A Structured Point of View Nuno Loureno University of Coimbra, Portugal naml@dei.uc.pt Evolving Grammars: A Structured Point of View 2 evolutionary algorithms Genetic Algorithm Evolutionary Strategies

slide-1
SLIDE 1

1

Evolving Grammars: A Structured Point of View

Nuno Lourenço University of Coimbra, Portugal naml@dei.uc.pt

slide-2
SLIDE 2

Evolving Grammars: A Structured Point of View

evolutionary algorithms

  • Genetic Algorithm
  • Evolutionary Strategies
  • Genetic Programming

2

slide-3
SLIDE 3

Evolving Grammars: A Structured Point of View

evolutionary algorithms

Evolutionary Cycle

3

Parents' Selection Variation Operators Survivors' Selection Population Population Population

Initial Best

slide-4
SLIDE 4

Evolving Grammars: A Structured Point of View

genetic programming

  • Tree-Based
  • Graph-Based
  • Linear
  • Grammar-Based

4

CFG-GP Grammatical Evolution

slide-5
SLIDE 5

Evolving Grammars: A Structured Point of View

grammatical evolution

5

Genotype Phenotype

slide-6
SLIDE 6

Evolving Grammars: A Structured Point of View

grammatical evolution

Mapping Process

6

Binary String Integer String Context Free Grammar Program Genotype Phenotype

slide-7
SLIDE 7

Evolving Grammars: A Structured Point of View

grammatical evolution

Mapping Example

7

<movement> := left(0) | back (1) | right (2) | front (3)

00100110 00101110 00100111 00000110 38 46 39 6

38 % 4 = 2 Mapping Rule: Codon Value % Number of Possible Derivations

slide-8
SLIDE 8

Evolving Grammars: A Structured Point of View

grammatical evolution

  • On the Locality of Grammatical Evolution, Rothlauf et al.
  • Examining the “Best of Both Worlds” of Grammatical

Evolution, Whigham et al.

8

slide-9
SLIDE 9

Evolving Grammars: A Structured Point of View

Can We Overcome the Grammatical Evolution Issues?

9

slide-10
SLIDE 10

Evolving Grammars: A Structured Point of View

structured grammatical evolution

  • structured genotypic representation that ensures an
  • ne-to-one mapping between genes and non-

terminals another

10

slide-11
SLIDE 11

Evolving Grammars: A Structured Point of View

structured grammatical evolution

11

Genotype

<start> <expr> [0] [0,1] <term> <op> [0,1,0,1] [0,2,1]

slide-12
SLIDE 12

Evolving Grammars: A Structured Point of View

structured grammatical evolution

12

<start> <expr> <op> <expr> <term> <op> <term> <term> <op> <term> ( ) x + 0.5 * x

  • 0.5

Genotype

<start> <expr> [0] [0,1] <term> <op> [0,1,0,1] [0,2,1]

slide-13
SLIDE 13

Evolving Grammars: A Structured Point of View

structured grammatical evolution

Recursion

13

2 Levels of Recursion

slide-14
SLIDE 14

Evolving Grammars: A Structured Point of View

structured grammatical evolution

Variation Operators

14

Crossover

Parent 2 Mask Parent 1 Offspring 1

1

Offspring 2

[0] [0,1] [0,1,0,1] [0,2,1] [1] [1,0] [0,0,0,1] [2,3,1] 1 [0] [0,1] [0,0,0,1] [2,3,1] [1] [1,0] [0,1,0,1] [0,2,1]

Mutation

Mutation

[0] [0,1] [0,1,0,1] [1,3,1] [0] [1,1] [0,1,0,1] [1,3,1]

slide-15
SLIDE 15

Evolving Grammars: A Structured Point of View

structured grammatical evolution

Crossover

15 <start> <expr> <op> <expr> <term> <op> <term> <term> <op> <term> ( ) x + 0.5 * x

  • 0.5

<start> <expr> <term> <op> <term> x * x ( )

Parent 1 Parent 2

<start> <expr> <op> <expr> <term> <op> <term> <term> <op> <term> ( ) x * x / x

  • 0.5

<start> <expr> <term> <op> <term> x + 0.5 ( )

Offspring 1 Offspring 2

slide-16
SLIDE 16

Evolving Grammars: A Structured Point of View

structured grammatical evolution

Mutation

16

<start> <expr> <op> <expr> <term> <op> <term> <term> <op> <term> ( ) x

  • 0.5

/ x

  • 0.5

( ) <start> <expr> <op> <term> <op> <term> x

  • 0.5

/

Before Mutation After Mutation

<expr> <term> <op> <term> ( ) x

  • 0.5
slide-17
SLIDE 17

Evolving Grammars: A Structured Point of View

Experimental Analysis

17

slide-18
SLIDE 18

Evolving Grammars: A Structured Point of View

problems

  • 11-Bit Boolean Multiplexer
  • 5-Bit Parity
  • Santa Fe Ant Trail
  • Quartic Polynomial
  • Boston Housing Problem

18

slide-19
SLIDE 19

Evolving Grammars: A Structured Point of View

experimental results

11-Bit Multiplexer

19

← ERROR →

200 200 600 800 1000

← EVALUATIONS (X1000) →

10 20 30 40 50 5 15 25 35 45

RND GE SGE CFG-GP

slide-20
SLIDE 20

Evolving Grammars: A Structured Point of View

experimental results

5-Bit Parity

20

8 10 12 14 16

← ERROR → ← EVALUATIONS (X1000) →

10 20 30 40 50 5 15 25 35 45

RND GE SGE CFG-GP

slide-21
SLIDE 21

Evolving Grammars: A Structured Point of View

experimental results

Santa Fe Ant Trail

21

← ERROR →

20 40 60 80

← EVALUATIONS (X1000) →

10 20 30 40 50 5 15 25 35 45

RND GE SGE CFG-GP

slide-22
SLIDE 22

Evolving Grammars: A Structured Point of View

experimental results

Quartic

22

← RRSE →

0.3 0.2 0.1 0.4 0.5 0.6

← EVALUATIONS (X1000) →

10 20 30 40 50 5 15 25 35 45

RND GE SGE CFG-GP

slide-23
SLIDE 23

Evolving Grammars: A Structured Point of View

experimental results

Boston Housing Problem

23

← RRSE → ← EVALUATIONS (X1000) → ← EVALUATIONS (X1000) →

0.70 0.95 1.00 1.15 1.30

← RRSE →

0.70 0.95 1.00 1.15 1.30 10 20 30 40 50 10 20 30 40 50

RND GE SGE CFG-GP RND GE SGE CFG-GP

Training Test

slide-24
SLIDE 24

Evolving Grammars: A Structured Point of View

sge vs ge locality

Distance between solutions after k mutations

24

Training Test

slide-25
SLIDE 25

Evolving Grammars: A Structured Point of View

SGE is good, but…

25

slide-26
SLIDE 26

Evolving Grammars: A Structured Point of View

sge criticisms

  • The input grammar has to be pre-processed
  • To remove recursion
  • Compute the maximum number of expansions

26

slide-27
SLIDE 27

Evolving Grammars: A Structured Point of View

27

Dynamic Structured 
 Grammatical Evolution 
 (DSGE)

slide-28
SLIDE 28

Evolving Grammars: A Structured Point of View

representation

  • For each non-terminal symbol there is a list of variable size

with the integers that encode the expansion possibilities;

  • Maximum recursion is defined as the maximum sub-tree

depth for each non-terminal symbol.

28

slide-29
SLIDE 29

Evolving Grammars: A Structured Point of View

initialisation

initialisation

29

<sigexpr>

<sigexpr> <node> <sum> <features> <weight> <bias> <number> <digit>

<node> <sigexpr> <node> | <node> + <sigexpr> 1 <weight> <sum> <bias> <weight> * sig(<sum> + <bias>) <number> <digit> <digit> <digit> <number> <digit>.<digit><digit> | 


  • <digit>.<digit><digit>

1, 5, 9 1

* sig( + )

  • .

1 5 9 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 +

1, 5, 9 1, 5, 9

slide-30
SLIDE 30

Evolving Grammars: A Structured Point of View

decoding procedure

  • Similar to the initialisation procedure;
  • Instead of selecting random expansion possibilities, the
  • nes encoded in the genotype are used;

30

slide-31
SLIDE 31

Evolving Grammars: A Structured Point of View

Experimental Analysis

31

slide-32
SLIDE 32

Evolving Grammars: A Structured Point of View

problems

  • 11-Bit Boolean Multiplexer
  • 5-Bit Parity
  • Santa Fe Ant Trail
  • Quartic Polynomial
  • Boston Housing Problem

32

slide-33
SLIDE 33

Evolving Grammars: A Structured Point of View

experimental results

11-Bit Multiplexer

33

← FITNESS → ← GENERATION →

450 550 500 600 650 700 750 800 10 20 30 40 50

SGE DSGE

slide-34
SLIDE 34

Evolving Grammars: A Structured Point of View

experimental results

Santa Fe Ant Trail

34

← FITNESS → ← GENERATION →

20 10 30 40 50 60 70 10 20 30 40 50

SGE DSGE

slide-35
SLIDE 35

Evolving Grammars: A Structured Point of View

experimental results

Quartic Problem

35

← FITNESS → ← GENERATION →

0.00 0.10 0.05 0.15 0.20 0.25 0.30 0.35 10 20 30 40 50

SGE DSGE

slide-36
SLIDE 36

Evolving Grammars: A Structured Point of View

experimental results

Boston Housing Problem

36

Training Test

← FITNESS → ← GENERATION →

0.75 0.85 0.80 0.90 0.95 1.00 1.05 1.10 1.15 0.75 0.85 0.80 0.90 0.95 1.00 1.05 1.10 1.15 10 20 30 40 50

← FITNESS → ← GENERATION →

10 20 30 40 50

SGE DSGE

slide-37
SLIDE 37

Evolving Grammars: A Structured Point of View

Evolving Artificial Neural Networks

37

slide-38
SLIDE 38

Evolving Grammars: A Structured Point of View

datasets

  • Flame [2];
  • Wisconsin Breast Cancer Detection (WDBC) [30];
  • Ionosphere [34];
  • Sonar [60].

38

slide-39
SLIDE 39

Evolving Grammars: A Structured Point of View

grammar

39

<sigexpr> ::= <node> | <node> + <sigexpr> <node> ::= <weight> ∗ sig(<sum> + <bias>) <sum> ::= <weight> ∗ <features> | <sum> + <sum> <features> ::= x1 | . . . | xn <weight> ::= <number> <bias> ::= <number> <number> ::= <digit>.<digit><digit> | –<digit>.<digit><digit> <digit> ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

slide-40
SLIDE 40

Evolving Grammars: A Structured Point of View

fitness evolution

40

← FITNESS → ← GENERATION →

1.1 1.5 1.3 1.7 1.9 2.1 2.3 2.5 2.7 100 200 300 400 500

← FITNESS → ← GENERATION →

1.1 1.5 1.3 1.7 1.9 2.1 2.3 2.5 2.7 100 200 300 400 500

← FITNESS → ← GENERATION →

1.1 1.5 1.3 1.7 1.9 2.1 2.3 2.5 2.7 100 200 300 400 500

← FITNESS → ← GENERATION →

1.1 1.5 1.3 1.7 1.9 2.1 2.3 2.5 2.7 100 200 300 400 500

SGE GE DSGE

Flame WDBC Ionosphere Sonar

slide-41
SLIDE 41

Evolving Grammars: A Structured Point of View

Conclusions

41

slide-42
SLIDE 42

Evolving Grammars: A Structured Point of View

Conclusions

  • New Grammatical Representation
  • Better performance in all the benchmarks used
  • More Effective
  • More Efficiente
  • Statistical significant improvements
  • SGE over GE
  • DSGE over SGE

42

slide-43
SLIDE 43

43

Nuno Lourenço University of Coimbra, Portugal naml@dei.uc.pt

Evolving Grammars: A Structured Point of View