Compositional Autoconstructive Dynamics Kyle I. Harrington , - - PowerPoint PPT Presentation

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstructive Dynamics

Kyle I. Harrington1,2

1DEMO lab

Michtom School of Computer Science Brandeis University, Waltham, MA

2Co-authors:

Emma Tosch (Brandeis University) Lee Spector (Hampshire College, Amherst, MA) Jordan Pollack (Brandeis University)

June 29, 2011 - ICCS

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Outline

1

Evolutionary Computation Genetic Algorithm Genetic Programming

2

Meta-Evolution

3

Autoconstructive Evolution Push Compositional Autoconstruction

4

Conclusion

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Evolutionary Computation

Evolutionary Algorithm

Evolutionary algorithms are a set of general techniques that can be applied to many types of problems. A minimal evolutionary algorithm.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Outline

1

Evolutionary Computation Genetic Algorithm Genetic Programming

2

Meta-Evolution

3

Autoconstructive Evolution Push Compositional Autoconstruction

4

Conclusion

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Genetic Algorithm

Overview

Developed by John Holland [Holland, 1975]. Evolve a population of individuals that represent potential problem solutions. A bit string genome.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Genetic Algorithm

Evaluation

Dependent upon genotype → phenotype mapping. Ones max: common benchmark, maximize 1’s in genome f(

⃗

g) = ∑

gi∈g

gi for genome⃗ g containing gi genes.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Genetic Algorithm

Selection

Choose members of population that reproduce. Fitness proportionate roulette: bias selection to better individuals.

Generation(t) Selection Generation(t+1) Genome Score

• Rel. score

Min Max Roulette Genome Score

• Rel. score

01010101 4 1/4 0.25 0.8147 11000010 3 1/6 10001001 3 3/16 0.25 0.4375 0.9058 11000010 3 1/6 00100000 1 1/16 0.4375 0.5 0.127 01010101 4 2/9 11011010 5 5/16 0.5 0.8125 0.9134 11000010 3 1/6 11000010 3 3/16 0.8125 1.0 0.6324 11011010 5 5/18 Total 16 Total 18 Average 3.2 Averge 3.6

Example 8-bit genome for Ones Max.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Genetic Algorithm

Variation

1-point mutation. 1-point crossover. Determining mutation and crossover probability is a “dark art.”

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Outline

1

Evolutionary Computation Genetic Algorithm Genetic Programming

2

Meta-Evolution

3

Autoconstructive Evolution Push Compositional Autoconstruction

4

Conclusion

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Genetic Programming

Overview

First in [Cramer, 1985], popularized in [Koza, 1992]. Evolution of executable computer programs. Used for: symbolic regression, gene-gene interaction detection, robot design/control, circuit design, etc. String representation f(x) = 1+ X ∗(X + 1) Tree representation Mathematical representation

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Genetic Programming

Evaluation

Target expression: g(x) = X 2 + 2X + 1 Example individual:

Tree representation Mathematical representation f(x) = 1+ X ∗(X + 1)

X = 0,f(X) = 1,g(X) = 1 X = 1,f(X) = 3,g(X) = 4 X = 2,f(X) = 7,g(X) = 9 Individual’s error: 3

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Genetic Programming

Mutation

Subtree mutation. Size-fair mutation, etc.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Genetic Programming

Crossover

Subtree crossover. Size-fair crossover, semantic-aware crossover, etc.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Meta-Evolution

Overview

Evolve the evolutionary process. The original meta-evolution algorithm was presented in [Schmidhuber, 1987]. Introduces the “meta-meta-. . . hook.”

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Example: Meta-Genetic Programming

Introduced in [Edmonds, 2001]. Evolve the variation programs. A meta-evolutionary algorithm.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Meta-Evolution

Adaptive Evolution and Self-Adaptation

Population-level: adjust variation probabilities based on population performance Individual-level: individual’s have unique variation probabilities Component-level: genes/instructions have unique variation probabilities

Full review in [Angeline, 1995]

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Autoconstructive Evolution

Overview

Introduced in [Spector and Robinson, 2002]. Integrate all levels of adaptation (population, individual, component) as part of the program itself. An autoconstructive evolutionary algorithm.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Outline

1

Evolutionary Computation Genetic Algorithm Genetic Programming

2

Meta-Evolution

3

Autoconstructive Evolution Push Compositional Autoconstruction

4

Conclusion

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Overview

Stack-based, multi-typed language designed for EC [Spector and Robinson, 2002]. Code is a data type (homoiconicity). Programs are capable of self-modification via exec stack [Spector et al., 2005]. Tag-based naming for items on all stacks [Spector et al., 2011].

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Zippers

Tree-based data structure introduced in [Huet and France, 1997]. Supports movement, insertion, removal, and substitution. An example zipper and potential movement locations.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Zipper instructions

Instruction Description ZIP.DOWN Move the top zipper deeper in the tree, if possible. ZIP.LEFT Move the top zipper to the left sibling, if possible. ZIP.RIGHT Move the top zipper to the right sibling, if possible. ZIP.RAND Replace the subtree rooted at the top zipper’s location with a random subtree. ZIP.ROOT Move the top zipper to the root of the tree. ZIP.RLOC Move the top zipper to a random location in the subtree rooted at the top zipper. ZIP.RRLOC Move the top zipper to a random location in the tree. ZIP.SWAP Pop two zippers and push them back onto the stack in reverse order. ZIP.SWAPSUB Swap the subtrees rooted at the top two zippers.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

Example program: (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+)

EXEC (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER FLOAT BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

EXEC 7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+ CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER FLOAT BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

EXEC INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+ CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER 7 FLOAT BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

EXEC 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+ CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER 7 FLOAT BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

EXEC INTEGER.* 0.5 INTEGER.DUP INTEGER.+ CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER 3 7 FLOAT BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

EXEC 0.5 INTEGER.DUP INTEGER.+ CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER 21 FLOAT BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

EXEC INTEGER.DUP INTEGER.+ CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER 21 FLOAT 0.5 BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

EXEC INTEGER.+ CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER 21 21 FLOAT 0.5 BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Push

Example

EXEC CODE (7 INTEGER.+ 3 INTEGER.* 0.5 INTEGER.DUP INTEGER.+) INTEGER 42 FLOAT 0.5 BOOLEAN

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Outline

1

Evolutionary Computation Genetic Algorithm Genetic Programming

2

Meta-Evolution

3

Autoconstructive Evolution Push Compositional Autoconstruction

4

Conclusion

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

Overview

Autoconstructive variation via functional composition of programs [Harrington et al., 2011]. Related to algorithmic chemistries [Fontana, 1991]. Forms of Compositional Autoconstruction Self-composition f ′ = f(f) Asymmetric collision f ′ = f(g) Collision f ′ = f(f,g)

Parameter Value Selection method tournaments of 5 Population size 1000 Maximum program size 500 Maximum number of generations 1001 Number of runs per condition 250

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

Example: ORDER

ORDER problem models order dependencies of program semantics (i.e. true/false clauses of if statements). The fitness of an Order solution, r(f), is computed as

∀zi ∈ [1,s],

if M(zi,f) < M(−zi,f) vi = 1

• therwise

vi = 0 r(f) = ∑

i

vi (1) where s is the problem size, M(z,f) is the number of elements preceding the first occurrence of z within program f in depth-first

• rder.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

Example: ORDER

(ZIP.RRLOC (3 (5) (16 (10) ZIP.RAND 10) -8) -6 (-8 (ZIP.RIGHT 13)) (-1 ((-11 (-6 13 (9)) -15) 8 (12) 8 -15) 10) (ZIP.RLOC) (ZIP.ROOT) 12) This autoconstructive mutation program has fitness of 7 and produces a child using standard subtree replacement mutation.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

ORDER: Results

Computational effort Median success generation

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

ORDER: Results

Standard GP Self-composition Asymmetric collision Collision

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

Example: MAJORITY

MAJORITY problem models evolutionary acquisition of beneficial components. The fitness of a Majority solution, r(f), is computed as

∀zi ∈ [1,s],

if N(zi,f) ≥ N(−zi,f)∧ N(zi,f) > 0 vi = 1

• therwise

vi = 0 r(f) = ∑

i

vi (2) where s is the problem size, N(z,f) is the number of instances of integer z in program f.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

Example: MAJORITY

(ZIP.RRLOC (3 (5) (16 (10) ZIP.RAND 10) -8) -6 (-8 (ZIP.RIGHT 13)) (-1 ((-11 (-6 13 (9)) -15) 8 (12) 8 -15) 10) (ZIP.RLOC) (ZIP.ROOT) 12) This autoconstructive mutation program has fitness of 8 and produces a child using standard subtree replacement mutation.

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

MAJORITY: Results

Computational effort Median success generation

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Compositional Autoconstruction

MAJORITY: Results

Standard GP Self-composition Asymmetric collision Collision

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Conclusion

Autoconstructive evolution for semantic model problems is “practical.” Population-level dependencies can lead to larger jumps in fitness than self-composition. Autoconstruction is ready for real-world problems. . .

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Thank you!

and the Brandeis DEMO lab (past and present), the Hampshire College CI lab, Carolyn Earnest, NSF

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Angeline, P . (1995). Adaptive and self-adaptive evolutionary computations. In Computational Intelligence: A Dynamic Systems Perspective. Cramer, N. (1985). A representation for the adaptive generation of simple sequential programs. In Proceedings of the First International Conference on Genetic Algorithms, volume 183, page 187. Edmonds, B. (2001). Meta-genetic programming: Co-evolving the operators of variation. Turk J. Elec. Engin. Fontana, W. (1991). Algorithmic chemistry. Artificial life II. Harrington, K., Tosch, E., Spector, L., and Pollack, J. (2011). Compositional Autoconstructive Dynamics. In Proc. of the 8th Intl. Conf. on Complex Systems. Holland, J. (1975). Adaptation in Natural and Artificial Systems. Huet, G. and France, I. (1997). Functional pearl: The zipper. In J. Functional Programming. Koza, J. (1992). Genetic programming: on the programming of computers by means of natural selection. Schmidhuber, J. (1987).

Compositional Autoconstruc- tive Dynamics Kyle I. Harrington, Evolutionary Computation

GA GP

Meta-Evolution Autoconstructive Evolution

Push Compositional Autoconstruction

Conclusion

. . . . . .

Evolutionary principles in self-referential learning. (On learning how to learn: The meta-meta-... hook.). PhD thesis, Institut für Informatik, Technische Universität München. Spector, L., Klein, J., and Keijzer, M. (2005). The Push3 execution stack and the evolution of control. In Proc. of the 2005 Conf. on Genetic and Evolutionary Computation, pages 1689–1696. ACM. Spector, L., Martin, B., Harrington, K., and Helmuth, T. (2011). Tag-based modules in genetic programming. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2011). Spector, L. and Robinson, A. (2002). Genetic programming and autoconstructive evolution with the push programming language. Genetic Programming and Evolvable Machines, 3(1):7–40.