R EPRESENTING T REES IN GEP 0123456789 +/Q*c-abde WSC9 - - PowerPoint PPT Presentation

▶
r epresenting t rees in gep
SMART_READER_LITE
LIVE PREVIEW

R EPRESENTING T REES IN GEP 0123456789 +/Q*c-abde WSC9 - - PowerPoint PPT Presentation

Feb 19, 2024 158 likes •313 views

D ESIGNING N EURAL N ETWORKS U SING G ENE E XPRESSION P ROGRAMMING Cndida Ferreira Gepsoft, UK candidaf@gepsoft.com WSC9 www.gepsoft.com P LAN 1. Representing trees in GEP 2. GEP genes 3. Multigenic chromosomes 4. Representing ANNs in

slide-1
SLIDE 1

WSC9 www.gepsoft.com

DESIGNING NEURAL NETWORKS USING GENE EXPRESSION PROGRAMMING

Cândida Ferreira Gepsoft, UK candidaf@gepsoft.com

slide-2
SLIDE 2

WSC9 www.gepsoft.com

PLAN

1. Representing trees in GEP 2. GEP genes 3. Multigenic chromosomes 4. Representing ANNs in GEP 5. Evolving the neural network architecture 6. Example: Intragenic two-point recombination 7. Fine-tuning the weights/thresholds 8. Example: Domain-specific transposition 9. XOR: Parameters and performance 10. XOR: Two perfect solutions 11. 6-Multiplexer: Parameters and performance 12. 6-Multiplexer: A perfect unigenic solution 13. 6-Multiplexer: A perfect multigenic solution

slide-3
SLIDE 3

WSC9 www.gepsoft.com

0123456789 +/Q*c-abde

REPRESENTING TREES IN GEP

slide-4
SLIDE 4

WSC9 www.gepsoft.com

t = h (n-1) + 1

0123456789012345678901234567890 /aQ/b*ab/Qa*b*-ababaababbabbbba

GEP GENES

head tail

h - head length t - tail length n - max arity

slide-5
SLIDE 5

WSC9 www.gepsoft.com

012345678901234012345678901234012345678901234 Q/*b+Qababaabaa-abQ/*+bababbab**-*bb/babaaaab

MULTIGENIC CHROMOSOMES

/ a b b b a b

Sub-ET3

a b

Sub-ET2

Q a a * b b Q /

Sub-ET1

slide-6
SLIDE 6

WSC9 www.gepsoft.com

0123456789012 DDDabab654321

REPRESENTING ANNS IN GEP

  • 1

h1 i1 i2 h2 1 3 6 4 5 2

D

1 3 4 5 6 2

D D b b a a

head tail Dw

slide-7
SLIDE 7

WSC9 www.gepsoft.com

EVOLVING THE NEURAL NETWORK ARCHITECTURE

Genetic Operators: » Mutation » Transposition » IS Transposition » RIS Transposition » Gene Transposition » Recombination » One-point Recombination » Intragenic Two-point Recombination

slide-8
SLIDE 8

WSC9 www.gepsoft.com

EXAMPLE: INTRAGENIC TWO-POINT RECOMBINATION

Q a b b a b D

1.488 0 .039

  • 0.957

1 . 8 7 4 1.488 1 . 6 7 7

Q a b b a b D

1.255

  • 1.553
  • 0.487
  • 1.625
  • 1.91
  • 0.487

a) b) c)

0123456789012345601234567890123456

  • [0]

Qaabbbabb97872192QDbabbaaa81327963-[1] TTababaab14393255QDbabbabb96369304 0123456789012345601234567890123456 aabbbabb978

  • [0]

Q 72192QDbabbaaa81327963-[1] T 93255QDbabbabb96369304 Tababaab143 W = {-0.78, -0.521, -1.224, 1.891, 0.554, 1.237, -0.444, 0.472, 1.012, 0.679} W = {-1.553, 1.425, -1.606, -0.487, 1.255, -0.253, -1.91, 1.427, -0.103, -1.625} W = {-0.148, 1.83, -0.503, -1.786, 0.313, -0.302, 0.768, -0.947, 1.487, 0.075} W = {-0.256, -0.026, 1.874, 1.488, -0.8, -0.804, 0.039, -0.957, 0.462, 1.677}

0,1 0,2 1,1 1,2

T b a L L L L Q a b a b Sub-NN0,1 Sub-NN1,1 Sub-NN0,2 Sub-NN1,2

  • 0.503

0.075 1 . 8 3

  • 0.503

T b a a

1.237 1.237

  • 1 .224

1.891 1 . 8 9 1 0.679

Q a b b Sub-NN0,2 T b a a Sub-NN0,1

1.237 1.237

  • 1.224

Q a b a Sub-NN1,1 T b a a

  • 0.503

0.075 1.83

  • 0.503
  • .

9 4 7

  • 1

. 7 8 6 0.313

Sub-NN1,2 Q a b b a b D

1.488 0.039

  • 0.957

1 . 8 7 4 1.488 1.677

a b D

1.255

  • 1.553
  • 0.487
  • 1.625
  • 1.91
  • 0.487
slide-9
SLIDE 9

WSC9 www.gepsoft.com

FINE-TUNING THE WEIGHTS/THRESHOLDS

Genetic Operators: » Domain-specific Mutation » Domain-specific Transposition » Direct Mutation of Weights/Thresholds

slide-10
SLIDE 10

WSC9 www.gepsoft.com

EXAMPLE: DOMAIN-SPECIFIC TRANSPOSITION

01234567890123456 3456 DTQaababaabbaabba 2867 7890123456789012 0571745736284668 W = {-1.64, -1.834, -0.295, 1.205, -0.807, 0.856, 1.702,

  • 1.026, -0.417, -1.061}

T = {-1.14, 1.177, -1.179, -0.74, 0.393, 1.135, -0.625, 1.643, -0.029, -1.639}

m m

D

  • 0.417
  • 0.417
  • 0.807

1.702 1.643

  • 0.625
  • 0.029

1 . 7 2 1.702 1 . 2 5

  • 1.026
  • .

2 9 5

T Q b a a a a a b

01234567890123456 3456 DTQaababaabbaabba 8466 7890123456789012 0571466827457362 W = {-1.64, -1.834, -0.295, 1.205, -0.807, 0.856, 1.702,

  • 1.026, -0.417, -1.061}

T = {-1.14, 1.177, -1.179, -0.74, 0.393, 1.135, -0.625, 1.643, -0.029, -1.639}

d d

D

  • 0.295

0.856

  • 1.026

1.702

  • 0.625
  • 0.625

0.393 1 . 2 5

  • 1.026
  • .

2 9 5

  • 0.417
  • .

8 7

T Q b a a a a a b

slide-11
SLIDE 11

WSC9 www.gepsoft.com

XOR: PARAMETERS AND PERFORMANCE

Redundant System Compact System Number of runs 100 100 Number of generations 50 50 Population size 30 30 Number of fitness cases 4 4 Function set D T Q D T Q Terminal set a b a b Weights array length 10 10 Weights range [-2, 2] [-2, 2] Head length 4 2 Number of genes 1 1 Chromosome length 33 17 Mutation rate 0.061 0.118 One-point recombination rate 0.7 0.7 IS transposition rate 0.1

  • IS elements length

1

  • RIS transposition rate

0.1

  • RIS elements length

1

  • Dw specific transposition rate

0.1 0.1 Dw specific IS elements length 2,3,5 2,3,5 Success rate 77% 30%

slide-12
SLIDE 12

WSC9 www.gepsoft.com

XOR: TWO PERFECT SOLUTIONS

012345678901234567890123456789012 TQaTaaababbbabaaa6085977238275036 W = {1.175, 0.315, -0.738, 1.694, -1.215, 1.956,

  • 0.342, 1.088, -1.694, 1.288}

a T a a Q a b

  • 0.342

1.694 1.175 1.956

  • 0.738

a b b T

1.088

  • 0.738
  • 1.694

1.694 1.088

01234567890123456 TDbabaabb88399837 W = {0.713, -0.774, -0.221, 0.773, -0.789, 1.792, -1.77, 0.443, -1.924, 1.161}

a b T D a b

  • 1.924

0.443 0.773 1.161 1.161

1. 2.

slide-13
SLIDE 13

WSC9 www.gepsoft.com

6-MULTIPLEXER: PARAMETERS AND PERFORMANCE

Unigenic System Multigenic System Number of runs 100 100 Number of generations 2000 2000 Population size 50 50 Number of fitness cases 64 64 Function set 3U 3D 3T 3U 3D 3T Terminal set a b c d d e f a b c d d e f Linking function

  • O

Weights array length 10 10 Weights range [-2, 2] [-2, 2] Head length 17 5 Number of genes 1 4 Chromosome length 103 124 Mutation rate 0.044 0.044 Intragenic two-point recombination rate 0.6 0.6 Gene recombination rate

  • 0.1

Gene transposition rate

  • 0.1

IS transposition rate 0.1 0.1 IS elements length 1,2,3 1,2,3 RIS transposition rate 0.1 0.1 RIS elements length 1,2,3 1,2,3 Weights mutation rate 0.002 0.002 Dw specific transposition rate 0.1 0.1 Dw specific IS elements length 2,3,5 2,3,5 Success rate 4% 6%

slide-14
SLIDE 14

WSC9 www.gepsoft.com

6-MULTIPLEXER: A PERFECT UNIGENIC SOLUTION

TbDTTTfTTaUDcUUTTafeefebabbdabffddfcfeeeabcabfabdcfe... ...709761631479459597193997465381760511137453583952159 W = {0.241, 1.432, 1.705, -1.95, 1.19, 1.344, 0.925, -0.163, -1.531, 1.423}

T T f f f f a a a a f e e e U U U T T D D c b b b b b d T T T T

1.423 1.344 1 . 4 3 2 1.705 1.344 1.423

  • 1.95
  • 1.531

1 . 3 4 4

  • 1.95

1.344 1.19

  • 0.

163

  • 1.95

1.432 1 .432 1.432 1 . 3 4 4 . 9 2 5

  • 0.163

1.432

  • 1.531
  • 1

. 9 5 1.344 0.925

  • .

1 6 3

  • 1.95

1 . 4 2 3 1 .423 1.19 0.241

slide-15
SLIDE 15

WSC9 www.gepsoft.com

6-MULTIPLEXER: A PERFECT MULTIGENIC SOLUTION

W = {1.126, 0.042, 1.588, -0.03, -1.91, 1.83, -0.412, 0.607, -0.294, -0.659} W = {-1.961, 1.161, 1.588, -0.03, -1.91, 1.762, -0.412, -0.121, -0.294, -0.659} W = {1.558, -0.69, 0.921, 0.134, 0.468, -1.534, 0.966, 1.399, 0.023, 0.915} W = {1.558, 0.767, 0.076, 0.071, 0.468, -1.534, 1.387, -1.857, -1.88, 0.331}

1 2 3 4

TecTDdfafabdddfa487674791701403 TDcbTbadddfceacc501702156029560 TfTTUbadbcdffdce593993321226318 TDTbaceaaeeacacd072636270049968

Note: Sub-NNs linked by OR

0.331 0.331 0.468

D D

  • 0.659
  • 1

. 8 8 1.588 1.387

a c c d f a f D

  • 0.03

1.126

T T T T T T T T e e f c c a a a Sub-NN1 Sub-NN3 Sub-NN2 Sub-NN4 d d T

  • 1.961
  • 0.412

1.762

a b b b d b b

  • 1.91

0.042 1.126

  • .

6 5 9 0.134 1.762

  • 1.857

0.607 0.921

  • .

6 9 0.134 0.966 0.921 0.921

  • .

6 9

U d

0.023

  • 1

. 9 6 1 1.558 0.042 0.134

  • 0.412

1.558