GENETIC PROGRAMMING John R. Koza Foresight Institute Workshop May - PowerPoint PPT Presentation

GENETIC PROGRAMMING John R. Koza Foresight Institute Workshop May 28, 2017

GENETIC PROGRAMMING

MAIN POINT No. 1 • G e n e t i c p r o g r a m m i n g n o w routinely delivers high-return human-competitive machine intelligence

MAIN POINT No. 2 • Genetic programming is an automated invention machine

INVENTION MACHINE

MAIN POINT No. 3 • Genetic programming has delivered a progression of qualitatively more substantial results in synchrony with fjve approximately order-of- magnitude increases in the expenditure of computer time

PROGRESSIVELY MORE SUBSTANTIAL RESULTS

MAIN POINT No. 1 • Genetic programming now routinely delivers high-return human-competitive machine intelligence

“HUMAN-COMPETITIVE”

CRITERIA FOR “HUMAN- COMPETITIVENESS” • T h e r e s u l t i s e q u a l o r b e t t e r than human-designed solution to the same problem

• Previously patented, an improvement over a patented invention, or patentable today • 6 more criteria

DEFINITION OF “ HIGH- RETURN ” T h e A I r a t i o (the “artifjcial-to- intelligence” ratio) of a problem- solving method as the ratio of that which is delivered by the automated operation of the artifjcial method to the amount of intelligence that is supplied by the human applying the method to a particular problem

THE “AI” RATIO

DEFINITION OF “ ROUTINE ” A problem solving method is routine if it is general and relatively little human effort is required to get the method to successfully handle new problems within a particular domain and to successfully handle new problems from a different domain.

“ROUTINE”

PROGRESSION OF QUALITATIVELY MORE SUBSTANTIAL RESULTS PRODUCED BY GP • T o y p r o b l e m s • Human-competitive non-patent results • 20 t h -century patented inventions • 21 st -century patented inventions • Patentable new inventions

GENETIC PROGRAMMING: ON THE PROGRAMMING OF COMPUTERS BY MEANS OF NATURAL SELECTION (Koza 1992)

A COMPUTER PROGRAM

GENETIC PROGRAMMING • C r e a t e i n i t i a l p o p u l a t i o n ( r a n d o m ) • Main generational loop – E x e c u t e a l l p r o g r a m s – Evaluate fjtness of all programs – Select single individuals or pairs of individuals based on fjtness to participate in the genetic operations (mutation, crossover, reproduction, architecture-altering operations) • Termination Criterion

CREATING RANDOM PROGRAMS

CREATING RANDOM PROGRAMS

DARWINIAN SELECTION

MUTATION OPERATION

CROSSOVER OPERATION

  Symbolic Regression   Intertwined Spirals   Truck Backer Upper   Broom Balancing   Wall Following   Artifjcial Ant   Box Moving   Discrete Pursuer-Evader Game   Differential Pursuer-Evader Game   Co-Evolution of Game-Playing Strategies   Inverse Kinematics   Emergent Collecting   Central Place Foraging   Block Stacking   Randomizer   Cellular Automata   Task Prioritization   Programmatic Image Compression   Econometric Exchange Equation   Optimization (Lizard)   Boolean 11-Multiplexer   11-Parity–Automatically Defjned Functions

2 MAIN POINTS FROM 1992 BOOK • Virtually all problems in artifjcial intelligence, machine learning, adaptive systems, and automated learning can be recast as a search for a computer program. • Genetic programming provides a way to successfully conduct the search for a computer program in the space of computer programs.

GENETIC PROGRAMMING II: AUTOMATIC DISCOVERY OF REUSABLE PROGRAMS (Koza 1994)

COMPUTER PROGRAMS • S u b r o u t i n e s p r o v i d e o n e w a y t o R E U S E c o d e  possibly with different instantiations of the dummy variables (formal parameters) • Loops (and iterations) provide a 2 n d way to REUSE code • Recursion provide a 3 rd way to REUSE code • Memory provides a 4 th way to REUSE the results of executing code

AUTOMATICALLY DEFINED FUNCTION volume ( p r o g n (defun volume (arg0 arg1 arg2) (values (* arg0 (* arg1 arg2)))) (values (- (volume L0 W0 H0) (volume L1 W1 H1))))

AUTOMATICALLY DEFINED FUNCTION FOR volume

AUTOMATICALLY DEFINED FUNCTIONS (SUBROUTINES) • ADFs provide a way to REUSE code • Code is typically reused with different instantiations of the dummy variables (formal parameters)

MAIN POINTS OF 1994 BOOK • Scalability is essential for solving non-trivial problems in artifjcial intelligence, machine learning, adaptive systems, and automated learning • Scalability can be achieved by reuse • Genetic programming provides a way to automatically discover and reuse subprograms in the course of automatically creating computer programs to solve problems

GENETIC PROGRAMMING III: DARWINIAN INVENTION AND PROBLEM SOLVING (Koza, Bennett, Andre, Keane 1999)

MEMORY Settable Indexed Matrix Relational memory (named) vector memory variables memory

ADL

ADR

HUMAN-COMPETITIVE RESULTS (NOT RELATED TO PATENTS) Transmembrane segment identifjcation problem for proteins Motifs for D–E–A–D box family and manganese superoxide dismutase family of proteins Cellular automata rule for Gacs-Kurdyumov-Levin (GKL) problem Quantum algorithm for the Deutsch-Jozsa “early promise” problem Quantum algorithm for Grover’s database search problem Quantum algorithm for the depth-two AND/OR query problem Quantum algorithm for the depth-one OR query problem Protocol for communicating information through a quantum gate Quantum dense coding Soccer-playing program that won its fjrst two games in the 1997 Robo Cup competition Soccer-playing program that ranked in the middle of fjeld in 1998 Robo Cup competition Antenna designed by NASA for use on spacecraft Sallen-Key fjlter

AUTOMATIC SYNTHESIS OF BOTH THE TOPOLOGY AND SIZING OF ANALOG ELECTRICAL CIRCUITS

COMPONENT-CREATING FUNCTIONS

TOPOLOGY-MODIFYING FUNCTIONS • S E R I E S d i v i s i o n • PARALLEL division • VIA • FLIP

TOPOLOGY-MODIFYING FUNCTIONS

DEVELOPMENTAL GP ( L I S T ( C ( – 0 . 9 6 3 ( – ( – - 0 . 8 7 5 - 0 . 1 1 3 ) 0.880)) (series (flip end) (series (flip end) (L -0.277 end) end) (L (– -0.640 0.749) (L -0.123 end)))) (flip (nop (L -0.657 end)))))

DEVELOPMENTAL GP

EVALUATION OF FITNESS z0 + IN OUT Embryonic Circuit P r o g r a m T r e e F u l l y D e s i g n e d C i r c u i t ( N e t G r a p h ) Circuit Netlist (ascii) Circuit Simulator (SPICE) Circuit Behavior (Output) Fitness

DESIRED BEHAVIOR OF A LOWPASS FILTER

EVOLVED CAMPBELL FILTER U . S . p a t e n t 1 , 2 2 7 , 1 1 3 George Campbell American Telephone and Telegraph 1917

EVOLVED ZOBEL FILTER U. S. patent 1,538,964 Otto Zobel American Telephone and Telegraph Company 1925

EVOLVED SALLEN-KEY FILTER

EVOLVED DARLINGTON EMITTER- FOLLOWER SECTION U. S. patent 2,663,806 Sidney Darlington Bell Telephone Laboratories 1953

NEGATIVE FEEDBACK

HAROLD BLACK ’ S RIDE ON THE LACKAWANNA FERRY Courtesy of Lucent Technologies

20 th -CENTURY PATENTS Campbell ladder topology for fjlters Zobel “ M -derived half section” and “constant K ” fjlter sections Crossover fjlter Negative feedback Cauer (elliptic) topology for fjlters PID and PID-D2 controllers Darlington emitter-follower section and voltage gain stage Sorting network for seven items using only 16 steps 60 and 96 decibel amplifjers Analog computational circuits Real-time analog circuit for time-optimal robot control Electronic thermometer Voltage reference circuit Philbrick circuit NAND circuit Simultaneous synthesis of topology, sizing, placement, and routing

SIX POST-2000 PATENTED INVENTIONS

EVOLVED HIGH CURRENT LOAD CIRCUIT

REGISTER-CONTROLLED CAPACITOR CIRCUIT

LOW-VOLTAGE CUBIC CIRCUIT

VOLTAGE-CURRENT-CONVERSION CIRCUIT

LOW-VOLTAGE BALUN CIRCUIT

TUNABLE INTEGRATED ACTIVE FILTER

21 st -CENTURY PATENTED INVENTIONS Low-voltage balun circuit Mixed analog-digital variable capacitor circuit High-current load circuit Voltage-current conversion circuit Cubic function generator Tunable integrated active fjlter

CIRCUIT SYNTHEIS PLUS LAYOUT

100%-COMPLIANT LOWPASS FILTER GENERATION 25 WITH 5 CAPACITORS AND 11 INDUCTORS  AREA OF 1775.2

100%-COMPLIANT LOWPASS FILTER BEST-OF-RUN CIRCUIT OF GENERATION 138 WITH 4 INDUCTORS AND 4 CAPACITORS  AREA OF 359.4

REVERSE ENGINEERING OF METABOLIC PATHWAYS

ANTENNA DESIGN

AUTOMATED DESIGN OF OPTICAL LENS SYSTEMS • Tackaberry-Muller lens system

EVOLVED SORTING NETWORK

GENETIC NETWORK FOR lac operon

SUBROUTINE DUPLICATION

SUBROUTINE CREATION

SUBROUTINE DELETION

ARGUMENT DUPLICATION

ARGUMENT DELETION

PARAMETERIZED TOPOLOGIES • One of the most important characteristics of computer programs is that they ordinarily contain inputs (free variables) and conditional operations