[PPT] - Dissertation Proposal: The Evolution of Robust Software Eric PowerPoint Presentation

SLIDE 1

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software

Eric Schulte

University of New Mexico

August 15, 2012

1/28

SLIDE 2

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software

Outline

1 Introduction 2 Related Work 3 Preliminary Work 4 Further Research 5 Applications 6 Conclusion

2/28

SLIDE 3

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Introduction

Natural Selection of Software

Insight

Over the past 50 years developers and engineers have selected, copied, modified, and pasted software tools, environments and code in a process resembling natural selection.

Research Questions

What effects have these processes had on software?
Can these insights lead to new tools and techniques?

3/28

SLIDE 4

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Introduction

Motivation

Need

1.3 Million employed software developers in the US in 2008.
Expect 21% increase by 2018 [Bureau of Labor Statistics, 2011].

Opportunity

Software Engineering Biology [Holland, 1992] [Beal and Sussman, 2008] [Perkins et al., 2009] [Rinard et al., 2004] [Raman and Wagner, 2010] [Weimer et al., 2009]

4/28

SLIDE 5

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Related Work

Biological Robustness

Genotype and Phenotype

Realms

genotype genetic information which specifies an organism phenotype resulting physical organism in the world

Robustness

genotypic (mutational) robustness to internal variance in genetic information phenotypic (environmental) robustness to external variance in the world

5/28

SLIDE 6

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Related Work

Biological Robustness

Mechanisms of robustness

Important amino acids
ver represented

[Knight et al., 1999]

Buffer changes

(e.g., metabolic pathways) [Wagner, 2005]

Degenerate vital functions

[Edelman and Gally, 2001]

(from http://www.raymondcheong.com/) 6/28

SLIDE 7

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Related Work

Biological Robustness

Robustness and Evolution

Evolution finds large

neutral spaces

Evolution increases

mutational robustness [Van Nimwegen et al., 1999]

Accrued genetic information

leads to evolutionary innovation [Ciliberti et al., 2007]

7/28

SLIDE 8

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Related Work

Evolutionary Computation

Digital Evolution

Computational simulation of adaptive systems [Holland, 1962].

Avida

[Ofria and Wilke, 2004]

computational models

which mimic evolved biological systems

allow time frames and

controls impossible in-situ

(used under the GNU Free Documentation License) 8/28

SLIDE 9

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Related Work

Evolutionary Computation

Genetic Programming

Natural selection as a heuristic used to program computers
Simplified programming languages [Koza, 1992]
Used to repair real-world extant software written by humans

m a i n ( i n g c , c h a r * a r g v [ ] ) i n t a ; i n t b ; i f ( a ! = b ) m a i n ( i n t a r g c , c h a r * a r g v [ ] ) i a ; i n t b ; i f ( a ! = b ) a + + ; e l s e b + ; e l s e b

9/28

SLIDE 10

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Related Work

Software Engineering

Acceptably Correct Computation

Ignore memory errors

[Rinard et al., 2004]

Hallucinate acceptable inputs

[Beal and Sussman, 2008]

Dynamically enforce learned invariants

[Perkins et al., 2009]

Drop loop executions at runtime

[Misailovic et al., 2011]

10/28

SLIDE 11

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Software Mutational Robustness: The fraction of mutants of software that are functional.

11/28

SLIDE 12

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Software Mutational Robustness: The fraction of mutants of software that are functional. software Depends upon:

The software itself

11/28

SLIDE 13

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Software Mutational Robustness: The fraction of mutants of software that are functional. mutants Depends upon:

The software itself
The mutation operators

11/28

SLIDE 14

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Software Mutational Robustness: The fraction of mutants of software that are functional. functional Depends upon:

The software itself
The mutation operators
The test of functionality

11/28

SLIDE 15

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Software Mutational Robustness: The fraction of mutants of software that are functional. Independent of:

The software itself
The mutation operators
The test of functionality

11/28

SLIDE 16

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Mutation Operators

→

(a) Insert AST

→

(b) Delete AST

→

(c) Swap AST

movq 8(%rdx), %rdi xorl %eax, %eax movq -80(%rbp), %rdx addl $1, %r14d call atoi movq -80(%rbp), %rdx movl %eax, (%r15) addq $4, %r15

→

movq 8(%rdx), %rdi xorl %eax, %eax movq -80(%rbp), %rdx addl $1, %r14d call atoi movq %rdx, -80(%rbp) movq -80(%rbp), %rdx movl %eax, (%r15) addq $4, %r15

(d) Insert ASM

movq 8(%rdx), %rdi xorl %eax, %eax movq %rdx, -80(%rbp) addl $1, %r14d call atoi movq -80(%rbp), %rdx movl %eax, (%r15) addq $4, %r15

→

movq 8(%rdx), %rdi xorl %eax, %eax addl $1, %r14d call atoi movq %rdx, -80(%rbp) movl %eax, (%r15) addq $4, %r15

(e) Delete ASM

movq 8(%rdx), %rdi xorl %eax, %eax movq %rdx, -80(%rbp) addl $1, %r14d call atoi movq -80(%rbp), %rdx movl %eax, (%r15) addq $4, %r15

→

movq 8(%rdx), %rdi xorl %eax, %eax movq -80(%rbp), %rdx addl $1, %r14d call atoi movq %rdx, -80(%rbp) movl %eax, (%r15) addq $4, %r15

(f) Swap ASM

(ELF level mutation operators not shown)

12/28

SLIDE 17

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Semantic Space & Mutation Testing

Specification Program Test Suite

13/28

SLIDE 18

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Semantic Space & Mutation Testing

Specification Program Test Suite 1 equivalent mutant

13/28

SLIDE 19

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Semantic Space & Mutation Testing

Specification Program Test Suite 1 equivalent mutant killed mutant 2

13/28

SLIDE 20

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Semantic Space & Mutation Testing

Specification Program Test Suite 1 equivalent mutant un-killed mutant 3 killed mutant 2

13/28

SLIDE 21

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Semantic Space & Mutation Testing

Specification Program Test Suite 1 equivalent mutant un-killed mutant 3 4 killed mutant 2

13/28

SLIDE 22

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Semantic Space & Mutation Testing

Specification Program Test Suite 1 equivalent mutant 5 un-killed mutant 3 4 killed mutant 2

13/28

SLIDE 23

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Semantic Space & Mutation Testing

Specification Program Test Suite 1 equivalent mutant 5 neutral mutant 3 4 killed mutant 2

13/28

SLIDE 24

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Benchmarks

Sorting Programs

bubble insertion merge quick

Siemens Benchmark

grep printtokens schedule sed space tcas

Real Programs

bzip2 1.0.2 (& alt. suite) ccrypt 1.2 (& alt. suite) imagemagick 6.5.2 jansson 1.3 leukocyte lighttpd 1.4.15 nullhttpd 0.5.0

ggenc 1.0.1 (& alt. suite)

potion 40b5f03 redis 1.3.4 tiff 3.8.2 vyquon 335426d

14/28

SLIDE 25

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Preliminary Work

Software Mutational Robustness

Preliminary Results

Average Mutational Robustness of 36.75%.

20 40 60 80 100 10 20 30 40 50 60 70 80 90 100 % Neutral Variants % Test Suite Statement or Instruction Coverage C ASM

15/28

SLIDE 26

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Further Research

Causes of Mutational Robustness

Level: Source VS. Compiled

Hypothesis: Compilation and linking increase software robustness.

Biological Base Pair Codon Gene Motif Protein Software Source Code Assembly Executable

Genotype Phenotype

16/28

SLIDE 27

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Further Research

Causes of Mutational Robustness

Level: Source VS. Compiled

Hypothesis: Compilation and linking increase software robustness.

Biological Base Pair Codon Gene Motif Protein Software Source Code Assembly Executable

Genotype Phenotype translation & compilation

16/28

SLIDE 28

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Further Research

Causes of Mutational Robustness

Level: Source VS. Compiled

Hypothesis: Compilation and linking increase software robustness.

Biological Base Pair Codon Gene Motif Protein Software Source Code Assembly Executable

Genotype Phenotype translation & transcription compilation linking

16/28

SLIDE 29

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Further Research

Causes of Mutational Robustness

Provenance: Evolved VS. Engineered

Hypothesis: Evolution increases mutational robustness and evolvability.

Compare three classes of software

1 Those programmed entirely by human engineers. 2 Those programmed initially by human engineers and then

incrementally evolved.

3 Those programmed entirely through an evolutionary process.

Test for

mutational robustness
evolvability

17/28

SLIDE 30

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Further Research

Correlates of Mutational Robustness

Evolvability

Are mutational robustness and evolvability correlated in software?

Requirements

metric of evolvability
metric of mutational robustness

Examples

what level of mutational robustness is desirable in software
when beneficial (e.g., maintaining critical behavior)
when detrimental (e.g., rapidly changing specification)

18/28

SLIDE 31

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Further Research

Correlates of Mutational Robustness

Environmental Robustness

Are mutational and environmental robustness correlated in software?

Definitions

mutational robustness robustness to genotypic variation environmental robustness robustness to environmental variation

Requirements

metric of mutational robustness
metric of environmental robustness (fuzz testing)

19/28

SLIDE 32

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Applications

Software Diversity

Compilers and Linkers

Software Diversity:

increases security and reliability

[Williams et al., 2009]

proactively fixes bugs

(Zak Fry in Schulte et al. [2012]) Multiple Compilation:

Jump-start population diversity
Increase library of genetic material
Mix and match optimization

between compilers Source Population

gcc -Ofast llvm gcc -Os

Evolutionary Computation

20/28

SLIDE 33

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Applications

New Program Representation

Program Atavism using Version Control

Version Control Repository

HEAD 8496cd 898b98 9d2bd0 fcd88f 8ae460

Individual

HEAD 8496cd 898b98 9d2bd0 fcd88f 8ae460

Population Evolutionary Computation

21/28

SLIDE 34

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Applications

Software Optimization

Components

extant program with existing test suite
system emulator
multi-objective fitness function combining
existing test suite
performance metrics

Benefits

flexibility through unsound transformations
explicit priorities through multi-objective fitness functions
optimization of hard-a-priori features
hardware specific and multi-threaded issues
performance trade-offs (e.g., memory/cpu), etc. . .

22/28

SLIDE 35

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Applications

Software Husbandry

Program 1 Program 2 Program 3 Neutral Space

Benefits

transfer optimization between projects
transfer functionality between projects
source of implementation diversity

23/28

SLIDE 36

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

24/28

SLIDE 37

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog

24/28

SLIDE 38

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog
ASM Rep.

bubble.s

.section .bss .lcomm buf , 500 .lcomm lst , 500 .section .text .globl _start _start: popl %eax popl %eax movl $0 , %esi movl $10 , %edx jmp read_arg store_arg: movl %ecx , lst(,%esi ,4) incl %esi read_arg: popl %eax cmpl $0 , %eax je sort movl $0 , %edi movl $0 , %ecx read_ch: movb 0(%eax ,% edi), %bl cmpl $0 , %ebx je store_arg imul %edx , %ecx subl $48 , %ebx addl %ebx , %ecx incl %edi jmp read_ch sort: movl $0 , %ecx movl $0 , %edi movl lst(,%edi ,4), %eax main_loop: movl %eax , %ebx movl %edi , %edx incl %edi cmpl %edi , %esi je cond_restart movl lst(,%edi ,4), %eax cmpl %ebx , %eax jge main_loop incl %ecx movl %ebx , lst(,%edi ,4) movl %eax , lst(,%edx ,4) jmp sort cond_restart : cmpl $0 , %ecx jne sort movl $0 , %edi movl $0 , %edi print_loop : pushl %edi movl lst(,%edi ,4), %eax pushl $0 movl $10 , %ecx cnv_int_down : movl $0 , %edx idiv %ecx addl $48 , %edx pushl %edx cmpl $0 , %eax jne cnv_int_down movl $0 , %edi cnv_int_up : popl %eax cmpl $0 , %eax je do_print movb %al , buf(,%edi ,1) incl %edi jmp cnv_int_up do_print: movl $4 , %eax movl $buf , %ecx movl $1 , %ebx movl %edi , %edx int $0x80 movl $32 , buf movl $buf , %ecx movl $4 , %eax movl $1 , %ebx movl $1 , %edx int $0x80 popl %edi incl %edi cmpl %edi , %esi jg print_loop movl $10 , buf movl $buf , %ecx movl $4 , %eax movl $1 , %ebx int $0x80 movl $1 , %eax movl $0 , %ebx int $0x80

24/28

SLIDE 39

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog
ASM Rep.
ELF Rep.
elfa – Lisp
rw-elfb– C

a http://gitweb.adaptive.cs.unm.edu/elf.git b http://gitweb.adaptive.cs.unm.edu/rw-elf.git

Structure of an ELF Binary

^?ELF ELF Header Program Header Table . . . . . . Sections . . . Section Header Table . . .

24/28

SLIDE 40

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog
ASM Rep.
ELF Rep.
elfa – Lisp
rw-elfb– C
Fault Localization

a http://gitweb.adaptive.cs.unm.edu/elf.git b http://gitweb.adaptive.cs.unm.edu/rw-elf.git

Fault Localization Overview

memory addr. to instruction

movq 8(%rdx), %rdi xorl %eax, %eax movl %eax, (%r15) addl $1, %r14d call atoi movq -80(%rbp), %rdx movq %rdx, -80(%rbp) addq $4, %r15 movq 8(%rdx), %rdi xorl %eax, %eax movl %eax, (%r15) addl $1, %r14d

Sample Program Counter Raw Sample Counts Smoothed Sample Counts CPU Machine-code Instructions

24/28

SLIDE 41

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog
ASM Rep.
ELF Rep.
elfa – Lisp
rw-elfb– C
Fault Localization
Mutational Robustness

a http://gitweb.adaptive.cs.unm.edu/elf.git b http://gitweb.adaptive.cs.unm.edu/rw-elf.git

24/28

SLIDE 42

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog
ASM Rep.
ELF Rep.
elfa – Lisp
rw-elfb– C
Fault Localization
Mutational Robustness
Experimentation

a http://gitweb.adaptive.cs.unm.edu/elf.git b http://gitweb.adaptive.cs.unm.edu/rw-elf.git

Software Mutational Robustness

20 40 60 80 100 10 20 30 40 50 60 70 80 90 100 % Neutral Variants % Test Suite Statement or Instruction Coverage C ASM

24/28

SLIDE 43

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog
ASM Rep.
ELF Rep.
elfa – Lisp
rw-elfb– C
Fault Localization
Mutational Robustness
Experimentation
Random walks

a http://gitweb.adaptive.cs.unm.edu/elf.git b http://gitweb.adaptive.cs.unm.edu/rw-elf.git

Neutral Variants of Insertion Sort

172.5 173 173.5 174 174.5 175 50 100 150 200 250 4 6 8 10 12 14 16 18 20

Mutational Robustness Number of Applied Mutations

Avg. LOC

Mutational Robustness

24/28

SLIDE 44

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog
ASM Rep.
ELF Rep.
elfa – Lisp
rw-elfb– C
Fault Localization
Mutational Robustness
Experimentation
Random walks
Multiple languages

a http://gitweb.adaptive.cs.unm.edu/elf.git b http://gitweb.adaptive.cs.unm.edu/rw-elf.git

Robustness Across Multiple Languages

C C++ Haskell OCaml Avg. bubble 25.7 28.2 27.6 16.7 24.6 insertion 26.0 42.0 35.6 23.7 31.8 merge 21.2 46.0 24.9 22.7 28.7 quick 25.5 42.0 26.3 11.4 26.3 Avg. 24.6 39.5 28.6 18.6 Std.Dev. 2.3 7.8 4.8 5.7 24/28

SLIDE 45

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Summary of Preliminary Work

Genprog
ASM Rep.
ELF Rep.
elfa – Lisp
rw-elfb– C
Fault Localization
Mutational Robustness
Experimentation
Random walks
Multiple languages
Robust Hardware

a http://gitweb.adaptive.cs.unm.edu/elf.git b http://gitweb.adaptive.cs.unm.edu/rw-elf.git

Physical Evolutionary Computation

25 ′latitude ✹✳✼✺❝♠ ❙✉❜♣♦♣✉❧❛t✐♦♥

✭✻✹ ✐♥❞✐✈✐❞✉❛❧s✮ ✳✳✳✼✾✾✻✺✰✯ ✳✳✳✾✾✾✻✺✰✯ ✳✳✳✾✽✵✹✰✰✯ ✳✳✳✲✸✽✯✽✯✰ ✳✳✳①✼✺✺✽✯✯ ✳✳✳✽✻✼✾✰✰✯ ✳✳✳②✽✾✹✯✯✺

❘P◆ ❊✈❛❧✉❛t✐♦♥

✾✼✷✽✯✯✯ ✸✹ ✶✷ st❛❝❦

❚♦✉r♥❛♠❡♥t

✯✻✯②✲✽✯

❈r♦ss♦✈❡r

② ✰ ① ✻ ✼ ✹ ✯ ✻ ✯ ② ✲ ✽ ✯ ✶ ✴ ✾ ✲ ✹ ✷ ✾ ✾ ✻ ✺ ✰ ✯ ✯ ②✰①✻✼✹✾✾✻✺✰✯✯ ✶✴✾✲✹✷✯✻✯②✲✽✯

▼✉t❛t✐♦♥

✵✲✺✶①✼②✺✽✯✯✯ ✵✲✺✶①✼✺✺✽✯✯✯

❍❛r❞✇❛r❡ ❚✐❧❡

❼ ✼✵▼❍③ ❆❘▼✼❚❉▼■ ▼❈❯ ❼ ✸✷❑❇ ❘❆▼✱ ✶✻❑❇ ❊❊P❘❖▼ ❼ ✶✷✽❑❇ ♣r♦❜❧❡♠ ❞❛t❛ st♦r❛❣❡ ❼ ❤❛r❞✇❛r❡ t✐♠❡rs ❼ ♣♦✇❡r s❤❛r✐♥❣✱ ✹ s❡r✐❛❧ ♣♦rts

❙♦❢t✇❛r❡ Pr♦❝❡ss

∀ ❡✈♦❧✉t✐♦♥❛r② t✐❧❡ r❛♥❞♦♠❧② ❝r❡❛t❡ ❛♥ ✐♥✐t✐❛❧ ♣♦♣✉❧❛t✐♦♥ ❢♦r ❡✈❡r✱ ❝♦♥❝✉rr❡♥t❧② ✇✐t❤ ❢r❡q✉❡♥❝② fm s❡❧❡❝t ♦♥❡ ✐♥❞✐✈✐❞✉❛❧ ❛♥❞ ♠✉t❛t❡ ✇✐t❤ ❢r❡q✉❡♥❝② fc s❡❧❡❝t t✇♦ ✐♥❞✐✈✐❞✉❛❧s ❛♥❞ ❝r♦ss♦✈❡r ✇✐t❤ ❢r❡q✉❡♥❝② fs s❡❧❡❝t ♦♥❡ ✐♥❞✐✈✐❞✉❛❧ ❛♥❞ s❤❛r❡ ✇✐t❤ ❢r❡q✉❡♥❝② fd ✐❢ ❝♦❧❧❡❝t✐♥❣❴❞❛t❛ r❡♣♦rt st❛t✐st✐❝s t♦ ❝❡♥tr❛❧ ❝♦❧❧❡❝t♦r

❘❡❛❧ ❚✐♠❡ ●❡♥❡t✐❝ ❖♣❡r❛t♦r ❚✐♠❡❧✐♥❡s ✷✺ ♠s

♠✉t❛t✐♦♥ ✭fm = 20Hz✮ ❝r♦ss♦✈❡r ✭fc = 10Hz✮ s❤❛r✐♥❣ ✭fs = 0.1Hz✮

❈♦♠♣✉t❛t✐♦♥❛❧ ❙♣❛❝❡

❙❛♠♣❧❡ ❣❡♦♠❡tr②✿ ❙✐①t❡❡♥ 4.75cm × 4.75cm t✐❧❡s ❛rr❛♥❣❡❞ ✐♥ ❛ 4 × 4 sq✉❛r❡✳

Pr♦❜❧❡♠ ❙♣❛❝❡

❙❛♠♣❧❡ ✜t♥❡ss ❢✉♥❝t✐♦♥✿ ●r❛✈✐t② ❞❛t❛ ❢r♦♠ t❤❡ st❛t❡ ♦❢ ▼♦♥t❛♥❛✳ ❊❛❝❤ ❤❛r❞✲ ✇❛r❡ t✐❧❡ ❝♦✈❡rs 25′ latitude×25′ longitude ♠❛♣♣❡❞ ✉s✐♥❣ t❤❡ ❵❧♦❝❛❧ ❞✐s❥♦✐♥t✬ s♣❛t✐❛❧ ♠❛♣♣✐♥❣✳ ❚❤❡ s✐③❡ ❛♥❞ s❤❛♣❡ ♦❢ t❤❡ t♦t❛❧ ❝♦✈❡r❡❞ ❛r❡❛ ✐s ❞❡t❡r♠✐♥❡❞ ❜② t❤❡ ♥✉♠❜❡r ❛♥❞ ❝♦♥✜❣✉r❛t✐♦♥ ♦❢ t❤❡ ❤❛r❞✇❛r❡ t✐❧❡s✱ ❛♥❞ ❜② t❤❡ s❝❛❧❡ ❛♥❞ s❝❤❡♠❡ ✉s❡❞ t♦ ♠❛♣ t❤❡ ❤❛r❞✇❛r❡ s♣❛❝❡ t♦ t❤❡ ♣r♦❜❧❡♠ s♣❛❝❡✳

24/28

SLIDE 46

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Work-plan and Timeline

TODAY Dissertation Work Plan 2010 2011 2012 2013 Assembly Repair ASE Short Paper ELF Representation Mutational Robustness Embedded Repair Representation Comparison Engineered/Evolved Rb. Sources of Robustness Environmental/Mutational Rb. Optimization Diversity Optimal Robustness

25/28

SLIDE 47

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Conclusion

Software is a product of natural selection
Software is not fragile
Software robustness can be used to build new tools
Software may provide general insight into evolution and

robustness.

26/28

SLIDE 48

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Thank You

Eric Schulte

ffice

FEC309 email eschulte@cs.unm.edu homepage http://cs.unm.edu/~eschulte

27/28

SLIDE 49

m a i n ( a r g c , c h a r * a r g v [ ] ) i n t a ; i n t m a i n ( i n t a r g c , c h a r * [ ] ) i n t a ; i n t b ; i f ( a ! = b i f ( a ! = b

Dissertation Proposal: The Evolution of Robust Software Conclusion

Bibliography

J. Beal and G. J. Sussman. Engineered robustness by controlled hallucination. November 2008.
U. D. o. L. Bureau of Labor Statistics. Computer software engineers and computer programmers. On the internet,

November 2011. http://www.bls.gov/oco/ocos303.htm.

S. Ciliberti, O. Martin, and A. Wagner. Innovation and robustness in complex regulatory gene networks.

Proceedings of the National Academy of Sciences, 104(34):13591, 2007.

G. M. Edelman and J. Gally. Degeneracy and complexity in biological systems. Proceedings of the National

Academy of Sciences, 98(24):13763, 2001.

J. Holland. Outline for a logical theory of adaptive systems. Journal of the ACM (JACM), 9(3):297–314, 1962.
J. H. Holland. Adaptation in natural and artificial systems: an introductory analysis with applications to biology,

control, and artificial intelligence. The MIT press, 1992.

R. Knight, S. Freeland, and L. Landweber. Selection, history and chemistry: the three faces of the genetic code.

Trends in biochemical sciences, 24(6):241–247, 1999.

J. Koza. Genetic programming: On the programming of computers by means of natural selection, 1992. See

http://miriad. Iip6. fr/microbes Modeling Adaptive Multi-Agent Systems Inspired by Developmental Biology, 229, 1992.

S. Misailovic, D. Roy, and M. Rinard. Probabilistically accurate program transformations. Static Analysis, pages

316–333, 2011.

C. Ofria and C. O. Wilke. Avida: A software platform for research in computational evolutionary biology. Artificial

Life, 10(2):191–229, 2004.

J. Perkins, S. Kim, S. Larsen, S. Amarasinghe, J. Bachrach, M. Carbin, C. Pacheco, F. Sherwood, S. Sidiroglou,
G. Sullivan, et al. Automatically patching errors in deployed software. 2009.
K. Raman and A. Wagner. The evolvability of programmable hardware. Journal of The Royal Society Interface, jun

2010.

M. Rinard, C. Cadar, D. Dumitran, D. Roy, T. Leu, and W. Beebee Jr. Enhancing server availability and security

through failure-oblivious computing. In Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation-Volume 6, pages 21–21. USENIX Association, 2004.

E. Schulte, Z. P. Fry, E. Fast, S. Forrest, and W. Weimer. Software mutational robustness: Bridging the gap

between mutation testing and evolutionary biology. http://arxiv.org/abs/1204.4224, April 2012.

E. Van Nimwegen, J. Crutchfield, and M. Huynen. Neutral evolution of mutational robustness. Proceedings of the

National Academy of Sciences, 96(17):9716, 1999.

A. Wagner. Robustness and evolvability in living systems. 2005.
W. Weimer, T. Nguyen, C. Le Goues, and S. Forrest. Automatically finding patches using genetic programming. In

Proceedings of the 31st International Conference on Software Engineering, pages 364–374. IEEE Computer Society, 2009. 28/28