Genetic Improvement Justyna Petke C entre for R esearch in E - - PowerPoint PPT Presentation

▶
genetic improvement
SMART_READER_LITE
LIVE PREVIEW

Genetic Improvement Justyna Petke C entre for R esearch in E - - PowerPoint PPT Presentation

Oct 03, 2022 15 likes •1.42k views

Genetic Improvement Justyna Petke C entre for R esearch in E volution, S earch and T esting University College London CREST Thank you Yue Jia Alexandru Marginean Mark Harman CREST Genetic Improvement Justyna Petke What does the word

slide-1
SLIDE 1

CREST

Genetic Improvement

Justyna Petke

Centre for Research in Evolution, Search and Testing University College London

slide-2
SLIDE 2

CREST

Justyna Petke Genetic Improvement

Thank you

Mark Harman Yue Jia Alexandru Marginean

slide-3
SLIDE 3

CREST

Justyna Petke Genetic Improvement

What does the word “Computer” mean?

Oxford Dictionary

“a person who makes calculations, especially with a calculating machine.”

Wikipedia

“The term "computer", in use from the mid 17th century, meant "one who computes": a person performing mathematical calculations.”

slide-4
SLIDE 4

CREST

Justyna Petke Genetic Improvement

What does the word “Computer” mean?

Oxford Dictionary

“a person who makes calculations, especially with a calculating machine.”

Wikipedia

“The term "computer", in use from the mid 17th century, meant "one who computes": a person performing mathematical calculations.”

slide-5
SLIDE 5

CREST

Justyna Petke Genetic Improvement

in the beginning ...

slide-6
SLIDE 6

CREST

Justyna Petke Genetic Improvement

The First Computer?

slide-7
SLIDE 7

Different People have different opinions

slide-8
SLIDE 8
slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13
slide-14
SLIDE 14
slide-15
SLIDE 15
slide-16
SLIDE 16

CREST

Justyna Petke Genetic Improvement

Who are the programers

slide-17
SLIDE 17

CREST

Justyna Petke Genetic Improvement

Who are the programers

slide-18
SLIDE 18

CREST

Justyna Petke Genetic Improvement

Who are the programers

slide-19
SLIDE 19

CREST

Justyna Petke Genetic Improvement

Who are the programers

it’s always been people

slide-20
SLIDE 20

CREST

Justyna Petke Genetic Improvement

Who are the programers

it’s always been people

slide-21
SLIDE 21

CREST

Justyna Petke Genetic Improvement

Who are the programers

lots of people

slide-22
SLIDE 22

CREST

Justyna Petke Genetic Improvement

why people?

human computers seem quaint today will human programmers seem quaint tomorrow ?

slide-23
SLIDE 23

CREST

Justyna Petke Genetic Improvement

programming is changing

slide-24
SLIDE 24

CREST

Justyna Petke Genetic Improvement

Requirements Requirements

Functional Requirements Non-Functional Requirements

slide-25
SLIDE 25

CREST

Justyna Petke Genetic Improvement

Functional Requirements Non-Functional Requirements

Memory Execution Time Battery Size Bandwidth functionality of the Program

slide-26
SLIDE 26

CREST

Justyna Petke Genetic Improvement

Software Design Process

slide-27
SLIDE 27

CREST

Justyna Petke Genetic Improvement

Software Design Process

slide-28
SLIDE 28

CREST

Justyna Petke Genetic Improvement

Software Design Process

slide-29
SLIDE 29

CREST

Justyna Petke Genetic Improvement

Software Design Process

slide-30
SLIDE 30

CREST

Justyna Petke Genetic Improvement

Software Design Process

slide-31
SLIDE 31

CREST

Justyna Petke Genetic Improvement

Multiplicity

Multiple Devices Conflicting Objectives Multiple Platforms

slide-32
SLIDE 32

CREST

Justyna Petke Genetic Improvement

Functional Requirements Non-Functional Requirements

Which requirements must be human coded ?

humans have to define these

slide-33
SLIDE 33

CREST

Justyna Petke Genetic Improvement

Functional Requirements Non-Functional Requirements

Which requirements are essential to human ?

humans have to define these we can optimise these

slide-34
SLIDE 34

CREST

Justyna Petke Genetic Improvement

slide-35
SLIDE 35

CREST

Justyna Petke Genetic Improvement

Pareto Front

slide-36
SLIDE 36

CREST

Justyna Petke Genetic Improvement

Pareto Front

each circle is a program found by a machine

slide-37
SLIDE 37

CREST

Justyna Petke Genetic Improvement

Pareto Front

different non functional properties have different pareto program fronts

slide-38
SLIDE 38

CREST

Justyna Petke Genetic Improvement

Failed Test Cases

slide-39
SLIDE 39

CREST

Justyna Petke Genetic Improvement

Why can’t functional properties be optimisation objectives ?

slide-40
SLIDE 40

CREST

Justyna Petke Genetic Improvement

slide-41
SLIDE 41

CREST

Justyna Petke Genetic Improvement

Optimisation

slide-42
SLIDE 42

CREST

Justyna Petke Genetic Improvement

Optimisation

2.5 times faster but failed 1 test case?

slide-43
SLIDE 43

CREST

Justyna Petke Genetic Improvement

slide-44
SLIDE 44

CREST

Justyna Petke Genetic Improvement

Optimisation

double the battery life but failed 2 test cases?

slide-45
SLIDE 45

CREST

Justyna Petke Genetic Improvement

Conformant GI VS. Heretical GI functional correctness is king conforms to traditional views

slide-46
SLIDE 46

CREST

Justyna Petke Genetic Improvement

Conformant GI VS. Heretical GI functional correctness is king conforms to traditional views correctness means having sufficient resources for computation conforms to a compelling new

  • rthodoxy
slide-47
SLIDE 47

CREST

Justyna Petke Genetic Improvement

Conformant GI VS. Heretical GI functional correctness is king conforms to traditional views correctness means having sufficient resources for computation conforms to a compelling new

  • rthodoxy

there’s nothing correct about a flat battery

slide-48
SLIDE 48

CREST

Justyna Petke Genetic Improvement

can it work ?

slide-49
SLIDE 49

CREST

Justyna Petke Genetic Improvement

Software Uniqueness

500,000,000 LoC

  • ne has to write approximately 6 statements

before one is writing unique code

slide-50
SLIDE 50

CREST

Justyna Petke Genetic Improvement

Software Uniqueness

500,000,000 LoC

  • ne has to write approximately 6 statements

before one is writing unique code

  • M. Gabel and Z. Su.

A study of the uniqueness of source code. (FSE 2010)

The space of candidate programs is far smaller than we might suppose.

“ ”

slide-51
SLIDE 51

CREST

Justyna Petke Genetic Improvement

Software Robustness

after one line changes up to 89% of programs that compile run without error

slide-52
SLIDE 52

CREST

Justyna Petke Genetic Improvement

Software Robustness

after one line changes up to 89% of programs that compile run without error

  • W. B .Langdon and J. Petke

Software is Not Fragile. (CS-DC 2015)

Software engineering artefacts are more robust than is often assumed.

“ ”

slide-53
SLIDE 53

CREST

Justyna Petke

Genetic Improvement for Software Specialisation

slide-54
SLIDE 54

CREST

Justyna Petke

slide-55
SLIDE 55

Question

Can we improve the efficiency of an already highly-optimised piece of software using genetic programming?

Genetic Improvement Justyna Petke

slide-56
SLIDE 56

Contributions

Introduction of multi-donor software transplantation Use of genetic improvement as means to specialise software

Genetic Improvement Justyna Petke

slide-57
SLIDE 57

Genetic Improvement

Genetic Improvement Justyna Petke

slide-58
SLIDE 58

Program Representation

Changes at the level of lines of source code Each individual is composed of a list of changes Specialised grammar used to preserve syntax

Genetic Improvement Justyna Petke

slide-59
SLIDE 59

Example

  • Genetic Improvement

Justyna Petke

slide-60
SLIDE 60

Code Transplants

GP has access to both:

  • the host program to be evolved
  • the donor program(s)

code bank contains all lines of source code GP has access to

Genetic Improvement Justyna Petke

slide-61
SLIDE 61

Mutation

Addition of one of the following operations: delete copy replace

Genetic Improvement Justyna Petke

slide-62
SLIDE 62

Example

  • Genetic Improvement

Justyna Petke

slide-63
SLIDE 63

Crossover

Concatenation of two individuals by appending two lists of mutations

  • Genetic Improvement

Justyna Petke

slide-64
SLIDE 64

Fitness

Based on solution quality and Efficiency in terms of lines of source code Avoids environmental bias

Genetic Improvement Justyna Petke

slide-65
SLIDE 65

Fitness

T est cases are sorted into groups One test case is sampled uniformly from each group Avoids overfitting

Genetic Improvement Justyna Petke

slide-66
SLIDE 66

Selection

Fixed number of generations Fixed population size Initial population contains single-mutation individuals

Genetic Improvement Justyna Petke

slide-67
SLIDE 67

Selection

T

  • p-half of population selected

Based on a threshold fitness value Mutation and Crossover applied

Genetic Improvement Justyna Petke

slide-68
SLIDE 68

Genetic Improvement

Genetic Improvement Justyna Petke

slide-69
SLIDE 69

Filtering

Mutations in best individuals are often independent Greedy approach used to combine best individuals

Genetic Improvement Justyna Petke

slide-70
SLIDE 70

Question

Can we improve the efficiency of an already highly-optimised piece of software using genetic programming?

Genetic Improvement Justyna Petke

slide-71
SLIDE 71

Motivation for choosing a SAT solver

Boolean satisfiability (SAT) example: x1 ∨ x2 ∨ ¬x4 ¬x2 ∨ ¬x3

  • xi : a Boolean variable
  • xi, ¬xi : a literal
  • ¬x2 ∨ ¬x3 : a clause

Genetic Improvement Justyna Petke

slide-72
SLIDE 72

Motivation for choosing a SAT solver

Bounded Model Checking Planning Software Verification Automatic Test Pattern Generation Combinational Equivalence Checking Combinatorial Interaction Testing and many other applications..

Genetic Improvement Justyna Petke

slide-73
SLIDE 73

Motivation for choosing a SAT solver

MiniSAT

  • hack track in SAT solver competitions
  • good source for software transplants

Genetic Improvement Justyna Petke

slide-74
SLIDE 74

Question

Can we evolve a version of the MiniSAT solver that is faster than any

  • f the human-improved versions of the solver?

Genetic Improvement Justyna Petke

slide-75
SLIDE 75

Experiments: Setup

Solvers used: MiniSAT2-070721 T est cases used: ∼ 2.5% improvement for general benchmarks (SSBSE’13)

Genetic Improvement Justyna Petke

slide-76
SLIDE 76

Motivation for choosing a SAT solver

MiniSAT

  • hack track in SAT solver competitions
  • good source for software transplants

Genetic Improvement Justyna Petke

slide-77
SLIDE 77

Question

Can we evolve a version of the MiniSAT solver that is faster than any

  • f the human-improved versions of the solver for a particular

problem class?

Genetic Improvement Justyna Petke

slide-78
SLIDE 78

Experiments: Setup

Solvers used: MiniSAT2-070721 T est cases used: from Combinatorial Interaction Testing field

Genetic Improvement Justyna Petke

slide-79
SLIDE 79

Combinatorial Interaction Testing

Use of SAT

  • solvers limited due to poor scalability

SAT benchmarks containing millions of clauses It takes hours to days to generate a CIT test suite using SAT

Genetic Improvement Justyna Petke

slide-80
SLIDE 80

Experiments: Setup

Host program: MiniSAT2-070721 (478 lines in main algorithm) Donor programs: MiniSAT

  • best09 (winner of ’09 MiniSAT
  • hack competition)

MiniSAT

  • bestCIT (best for CIT from ’09 competition)
  • total of 104 new lines

Genetic Improvement Justyna Petke

slide-81
SLIDE 81

Results

Solver Donor Lines Seconds MiniSAT (original) — 1.00 1.00 MiniSAT

  • best09

— 1.46 1.76 MiniSAT

  • bestCIT

— 0.72 0.87 MiniSAT

  • best09+bestCIT

— 1.26 1.63

Genetic Improvement Justyna Petke

slide-82
SLIDE 82

Question

How much runtime improvement can we achieve?

Genetic Improvement Justyna Petke

slide-83
SLIDE 83

Results

Solver Donor Lines Seconds MiniSAT (original) — 1.00 1.00 MiniSAT

  • best09

— 1.46 1.76 MiniSAT

  • bestCIT

— 0.72 0.87 MiniSAT

  • best09+bestCIT

— 1.26 1.63 MiniSAT

  • gp

best09 0.93 0.95

Genetic Improvement Justyna Petke

slide-84
SLIDE 84

Results

Donor: best09 13 delete, 9 replace, 1 copy Among changes: 3 assertions removed 1 deletion on variable used for statistics

Genetic Improvement Justyna Petke

slide-85
SLIDE 85

Results

Mainly if and for statements switched off Decreased iteration count in for loops

Genetic Improvement Justyna Petke

slide-86
SLIDE 86

Results

Solver Donor Lines Seconds MiniSAT (original) — 1.00 1.00 MiniSAT

  • best09

— 1.46 1.76 MiniSAT

  • bestCIT

— 0.72 0.87 MiniSAT

  • best09+bestCIT

— 1.26 1.63 MiniSAT

  • gp

best09 0.93 0.95 MiniSAT

  • gp

bestCIT 0.72 0.87

Genetic Improvement Justyna Petke

slide-87
SLIDE 87

Results

Donor: bestCIT 1 delete, 1 replace Among changes: 1 assertion deletion 1 replace operation triggers 95% of donor code

Genetic Improvement Justyna Petke

slide-88
SLIDE 88

Results

Solver Donor Lines Seconds MiniSAT (original) — 1.00 1.00 MiniSAT

  • best09

— 1.46 1.76 MiniSAT

  • bestCIT

— 0.72 0.87 MiniSAT

  • best09+bestCIT

— 1.26 1.63 MiniSAT

  • gp

best09 0.93 0.95 MiniSAT

  • gp

bestCIT 0.72 0.87 MiniSAT

  • gp

best09+bestCIT 0.94 0.96

Genetic Improvement Justyna Petke

slide-89
SLIDE 89

Results

Donor: best09+bestCIT 50 delete, 20 replace, 5 copy Among changes: 5 assertions removed 4 semantically equivalent replacements 3 operations used for statistics removed ∼ half of the mutations remove dead code

Genetic Improvement Justyna Petke

slide-90
SLIDE 90

Results

Solver Donor Lines Seconds MiniSAT (original) — 1.00 1.00 MiniSAT

  • best09

— 1.46 1.76 MiniSAT

  • bestCIT

— 0.72 0.87 MiniSAT

  • best09+bestCIT

— 1.26 1.63 MiniSAT

  • gp

best09 0.93 0.95 MiniSAT

  • gp

bestCIT 0.72 0.87 MiniSAT

  • gp

best09+bestCIT 0.94 0.96 MiniSAT

  • gp-combined

best09+bestCIT 0.54 0.83

Genetic Improvement Justyna Petke

slide-91
SLIDE 91

Results

Combining results: 37 delete, 15 replace, 4 copy 56 out of 100 mutations used Among changes: 8 assertion removed 95% of the bestCIT donor code executed

Genetic Improvement Justyna Petke

slide-92
SLIDE 92

Conclusions

Introduced multi-donor software transplantation Used genetic improvement as means to specialise software Achieved 17% runtime improvement on MiniSAT for the Combinatorial Interaction Testing domain by combining best individuals

Genetic Improvement Justyna Petke

slide-93
SLIDE 93

CREST

Justyna Petke Genetic Improvement GP Progra ms Progra ms Progra ms MiniSat Improv Non-functional property Test Fitne Test Sensitivity Analysis

Justyna Petke, Mark Harman, William B. Langdon and Westley Weimer Using Genetic Improvement & Code Transplants to Specialise a C++ program to a Problem Class (EuroGP’14)

Multi-doner transplant Specialized for CIT 17% faster

MiniSat MiniSat MiniSat

v1 v2 vn

G E C C O H u m i e s i l v e r m e d a l

Inter version transplantation

slide-94
SLIDE 94

CREST

Justyna Petke Genetic Improvement Bowtie 2 GP Progra ms Progra ms Progra ms Bowtie 2 Non-functional property Test Fitne Test Sensitivity Analysis

Genetic Improvement of Programs

  • W. B. Langdon and M. Harman

Optimising Existing Software with Genetic Programming. TEC 2015

70 times faster 30+ interventions HC clean up: 7 slight semantic improvement

slide-95
SLIDE 95

CREST

Justyna Petke Genetic Improvement Cuda GP Progra ms Progra ms Progra ms Cuda Improv Non-functional property Test Fitne Test Sensitivity Analysis

Genetic Improvement of Programs

  • W. B. Langdon and M. Harman

Genetically Improved CUDA C++ Software, EuroGP 2014

7 times faster updated for new hardware automated updating

slide-96
SLIDE 96

CREST

Justyna Petke Genetic Improvement GP Non-functional property Test Fitne Test Sensitivity Analysis

Memory speed trade offs

System malloc System

  • ptimised

malloc

Fan Wu, Westley Weimer, Mark Harman, Yue Jia and Jens Krinke
 Deep Parameter Optimisation
 Conference on Genetic and Evolutionary Computation (GECCO 2015)

Improve execution time by 12% or achieve a 21% memory consumption reduction

slide-97
SLIDE 97

CREST

Justyna Petke Genetic Improvement GP Non-functional property Test Fitne Test Sensitivity Analysis

Reducing energy consumption

Ensemble AProVE MiniSat

CIT

MiniSat MiniSat Ensemble AProVE Improved MiniSat

CIT

Improved MiniSat Improved MiniSat

Bobby R. Bruce Justyna Petke Mark Harman Reducing Energy Consumption Using Genetic Improvement Conference on Genetic and Evolutionary Computation (GECCO 2015)

Energy consumption can be reduced by as much as 25%

slide-98
SLIDE 98

CREST

Justyna Petke Genetic Improvement GP Non-functional property Test Fitne Test

Grow and graft new functionality

GP Non-functional property Test Fitne Test Sensitivi ty

Feature

Grow Graft

Human Knowledge

Feature

Host System

Mark Harman, Yue Jia and Bill Langdon, 
 Babel Pidgin: SBSE can grow and graft entirely new functionality into a real world system 
 Symposium on Search-Based Software Engineering SSBSE 2014. (Challenge track)

C h a l l e n g e T r a c k A w a r d

slide-99
SLIDE 99

CREST

Justyna Petke Genetic Improvement GP Non-functional property Test Fitne Test Sensitivity Analysis

Real world cross system transplantation

Earl T. Barr, Mark Harman, Yue Jia, Alexandru Marginean, and Justyna Petke 
 Automated Software Transplantation (ISSTA 2015)

Donor Host feature Host’ feature

Successfully autotransplanted new functionality and passed all regression tests for 12 out of 15 real world systems

slide-100
SLIDE 100

CREST

Justyna Petke Genetic Improvement

Automated Software Transplantation

E.T. Barr, M. Harman,

  • Y. Jia, A. Marginean & J. Petke

ACM Distinguished Paper Award at ISSTA 2015 coverage in article in 2647 shares of

slide-101
SLIDE 101

Video Player Start from scratch

Why Autotransplantation?

Check open source repositories Why not handle H.264? ~100 players

CREST

Justyna Petke Genetic Improvement

slide-102
SLIDE 102

Human Organ Transplantation

CREST

Justyna Petke Genetic Improvement

slide-103
SLIDE 103

Automated Software Transplantation

Host Donor Organ Organ ENTRY

V

Organ Test Suite

Manual Work:

Organ Entry Organ’s Test Suite Implantation Point

CREST

Justyna Petke Genetic Improvement

slide-104
SLIDE 104

μTrans

Host Donor Stage 1: Static Analysis Host Beneficiary Stage 2: Genetic Programming Stage 3: Organ Implantation Organ Test Suite

Implantation Point Organ Entry

CREST

Justyna Petke Genetic Improvement

slide-105
SLIDE 105

Stage 1 — Static Analysis

Donor OE ENTRY

Vein Organ

Matching Table Dependency Graph Host Implantation Point Stm: x = 10; -> Decl: int x; Donor: int X -> Host: int A, B, C

CREST

Justyna Petke Genetic Improvement

slide-106
SLIDE 106

Stage 2 — GP

S1 S2 S3 S4 S5 … Sn

Matching Table V3H V4H

Donor Variable ID Host Variable ID (set)

V1D V2D … V1H V2H V5H Individual

Var Matching Stateme nts

V1D V1H V2D V4H S1 S7 S73 M1: M2: … Genetic Programming

fitness(i) =

  • 1

3(1 + |T Xi| |T |

+ |T Pi|

|T | )

i ∈ IC i / ∈ IC

Weak Proxies: Does it execute test cases without crashing? Does it compile? Strong Proxies: Does it produce the correct output?

CREST

Justyna Petke Genetic Improvement

slide-107
SLIDE 107

Host Organ Donor Do we break the initial functionality? Have we really added new functionality? How about the computational effort? Is autotransplantation useful?

Research Questions

Regression Tests Acceptance Tests

CREST

Justyna Petke Genetic Improvement

slide-108
SLIDE 108

Research Questions

Do we break the initial functionality? How about the computational effort? Is autotransplantation useful? Have we really added new functionality? Empirical Study 15 Transplantations 300 Runs 5 Donors 3 Hosts Case Study: H.264 Encoding Transplantation

CREST

Justyna Petke Genetic Improvement

slide-109
SLIDE 109

Validation

Regression Tests Augmented Regression Tests Donor Acceptance Tests Acceptance Tests Manual Validation Host Beneficiary

CREST

Justyna Petke Genetic Improvement

slide-110
SLIDE 110

Subjects

Minimal size: 0.4k Max size: 422k Average Donor:16k Average Host: 213k

Subjects Type Size KLOC Idct Donor 2.3 Mytar Donor 0.4 Cflow Donor 25 Webserver Donor 1.7 TuxCrypt Donor 2.7 Pidgin Host 363 Cflow Host 25 SoX Host 43 Case Study x264 Donor 63 VLC Host 422

C
  • n
s i s t e n t * C
  • m
p l e t e * W e l l D
  • c
u m e n t e d * E a s y t
  • R
e u s e *

*

E v a l u a t e d

* I S S T A *

A r t i f a c t

* A E C

CREST

Justyna Petke Genetic Improvement

slide-111
SLIDE 111

Experimental Methodology and Setup

μSCALPEL Host Implantation Point Donor OE Organ Test Suite

Host Beneficiary

Implantation Point Organ

64 bit Ubuntu 14.10 16 GB RAM 8 threads

CREST

Justyna Petke Genetic Improvement

slide-112
SLIDE 112

Donor Host All Passed Regression Regression++ Acceptance Idct Pidgin 16 20 17 16 Mytar Pidgin 16 20 18 20 Web Pidgin 20 18 Cflow Pidgin 15 20 15 16 Tux Pidgin 15 20 17 16 Idct Cflow 16 17 16 16 Mytar Cflow 17 17 17 20 Web Cflow 17 Cflow Cflow 20 20 20 20 Tux Cflow 14 15 14 16 Idct SoX 15 18 17 16 Mytar SoX 17 17 17 20 Web SoX 17 Cflow SoX 14 16 15 14 Tux SoX 13 13 13 14 TOTAL 188/300 233/300 196/300 256/300

Empirical Study

C
  • n
s i s t e n t * C
  • m
p l e t e * W e l l D
  • c
u m e n t e d * E a s y t
  • R
e u s e *

*

E v a l u a t e d

* I S S T A *

A r t i f a c t

* A E C

CREST

Justyna Petke Genetic Improvement

in 12 out of 15 experiments we successfully autotransplanted new functionality

slide-113
SLIDE 113

Execution Time (minutes) Donor Host Idct Pidgin 5 7 97 Mytar Pidgin 3 1 65 Web Pidgin 8 5 160 Cflow Pidgin 58 16 1151 Tux Pidgin 29 10 574 Idct Cflow 3 5 59 Mytar Cflow 3 1 53 Web Cflow 5 2 102 Cflow Cflow 44 9 872 Tux Cflow 31 11 623 Idct SoX 12 17 233 Mytar SoX 3 1 60 Web SoX 7 3 132 Cflow SoX 89 53 74 Tux SoX 34 13 94 Total

Empirical Study

Average 334 (min)

  • Std. Dev.

Total 72 (hours) 10 (Average)

C
  • n
s i s t e n t * C
  • m
p l e t e * W e l l D
  • c
u m e n t e d * E a s y t
  • R
e u s e *

*

E v a l u a t e d

* I S S T A *

A r t i f a c t

* A E C

CREST

Justyna Petke Genetic Improvement

slide-114
SLIDE 114

Case Study

Transplant Time & Test Suites Time (hours) Regression Regression++ Acceptance H.264 26 100% 100% 100%

C
  • n
s i s t e n t * C
  • m
p l e t e * W e l l D
  • c
u m e n t e d * E a s y t
  • R
e u s e *

*

E v a l u a t e d

* I S S T A *

A r t i f a c t

* A E C

VLC H264

CREST

Justyna Petke Genetic Improvement

within 26 hours performed a task that took developers an avg of 20 days of elapsed time

slide-115
SLIDE 115

Automated Software Transplantation

H D O ENTRY

V

Organ’s Test Suite

Manual Work:

Organ Entry Organ’s Test Suite Implantation Point

Alexandru Marginean — Automated Software Transplantation

μTrans

Alexandru Marginean — Automated Software Transplantation

Host Donor Stage 1: Static Analysis Host Beneficiary Stage 2: Genetic Programming Stage 3: Organ Implantation Organ’s Test Suite

Validation

Alexandru Marginean — Automated Software Transplantation

Regression Tests Augmented Regression Tests Host Beneficiary Donor Acceptance Tests Acceptance Tests Manual Validation RQ1.a RQ1.b RQ2

CREST

Justyna Petke Genetic Improvement

slide-116
SLIDE 116

CREST

Justyna Petke Genetic Improvement

* http://crest.cs.ucl.ac.uk/autotransplantation/MuScalpel.html

slide-117
SLIDE 117

CREST

Justyna Petke Genetic Improvement

GI Applications

Bug fixing

slide-118
SLIDE 118

CREST

Justyna Petke Genetic Improvement

* http://dijkstra.cs.virginia.edu/genprog/ * http://people.csail.mit.edu/fanl/

Kali, SPR, ClearView

(from MIT)

and other …

Claire Le Goues, Stephanie Forrest, Westley Weimer: Current challenges in automatic software repair.

Software Quality Journal 21(3): 421-443 (2013)

slide-119
SLIDE 119

CREST

Justyna Petke Genetic Improvement

GI Applications

Bug fixing Improving energy consumption Porting old code to new hardware Grafting new functionality into an existing system Specialising software for a particular problem class Other

slide-120
SLIDE 120

CREST

Justyna Petke Genetic Improvement

What if fitness is expensive to compute ?

GI4GI: Improving Genetic Improvement Fitness Functions Mark Harman & Justyna Petke (Genetic Improvement Workshop 2015)

slide-121
SLIDE 121

CREST

Justyna Petke Genetic Improvement

GI4GI: Energy Optimisation Example

many factors affecting energy consumption, including: screen behaviour memory access device communications CPU utilisation

slide-122
SLIDE 122

CREST

Justyna Petke Genetic Improvement

GI4GI: Energy Optimisation Example

a hardware-dependent linear energy model for GI:

Post-compiler software optimization for reducing energy (ASPLOS’14) Schulte et al.

slide-123
SLIDE 123

CREST

Justyna Petke Genetic Improvement

GI4GI: Energy Optimisation Example

Idea: Use GI to evolve a fitness function f for energy consumption. Use f to improve energy consumption of software.

slide-124
SLIDE 124

CREST

Justyna Petke Genetic Improvement

GI4GI

slide-125
SLIDE 125

CREST

Justyna Petke

GI Growth

slide-126
SLIDE 126

CREST

Justyna Petke Genetic Improvement

Growing Area

1st International Genetic Improvement Workshop

at GECCO 2015, Madrid, Spain www.geneticimprovementofsoftware.com

slide-127
SLIDE 127

CREST

Justyna Petke Genetic Improvement

GI Growth

Special Issue

  • n GI

Special Session on GI *http://www.wcci2016.org/

slide-128
SLIDE 128

CREST

Justyna Petke Genetic Improvement

Functional Requirements Non-Functional Requirements

Conclusions

humans have to define these we can optimise these

slide-129
SLIDE 129

CREST

Justyna Petke Genetic Improvement

Summary

Search Based Optimisation Software Engineering

S B S E

Genetic Improvement Combinatorial Interaction Testing

slide-130
SLIDE 130

CREST

Justyna Petke

Research Opportunities

CREST

Justyna Petke

slide-131
SLIDE 131

CREST

Justyna Petke Contact me if you want to visit CREST: j.petke at ucl.ac.uk Centre for Research on Evolution, Search and Testing University College London

slide-132
SLIDE 132

CREST

Justyna Petke

20 mins walk

National Gallery Nelson’s Column Eros RoyalCourts

  • f Justice
  • St. Paul’s

Tate ModernGlobe Theatre Covent Garden Market Westminster Abbey House of Parliament London Eye British Museum Madame Tussaud’s Sherlock Holmes Museum Marble Arch National History Museum

slide-133
SLIDE 133

CREST

Justyna Petke

COWs

CREST Open Workshop Roughly one per month Discussion based Recorded and archived

http://crest.cs.ucl.ac.uk/cow/

slide-134
SLIDE 134

CREST

Justyna Petke

COWs

http://crest.cs.ucl.ac.uk/cow/

slide-135
SLIDE 135

CREST

Justyna Petke

COWs

http://crest.cs.ucl.ac.uk/cow/ #Total Registrations 1512 #Unique Attendees 667 #Unique Institutions 244 #Countries 43 #Talks 421 (Last updated on November 4, 2015)

slide-136
SLIDE 136

CREST

Justyna Petke

CREST Open Workshop (COW)

slide-137
SLIDE 137

CREST

Justyna Petke

http://crest.cs.ucl.ac.uk/cow/

Genetic Improvement

25-26 January 2016

45th COW

slide-138
SLIDE 138

CREST

Justyna Petke

Dynamic Adaptive Search Based Software Engineering

EPSRC Grant

Stirling Birmingham York UCL

DAASE

slide-139
SLIDE 139

CREST

Justyna Petke Genetic Improvement

Summary

Search Based Optimisation Software Engineering

S B S E

Genetic Improvement Combinatorial Interaction Testing COWs Visitor Scheme Open positions

slide-140
SLIDE 140

CREST

Justyna Petke Genetic Improvement

Pictures used with thanks from these sources

Pickering's Harem: [Public domain], via Wikimedia Commons IBM 026 Card Punch: By Ben Franske (Own work) [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY-SA-3.0-2.5-2.0-1.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons BBC_Micro: [Public domain], via Wikimedia Commons Programmer: undesarchiv, B 145 Bild-F031434-0006 / Gathmann, Jens / CC-BY-SA [CC-BY-SA-3.0- de (http://creativecommons.org/licenses/by-sa/3.0/de/deed.en)], via Wikimedia Commons IBM PC: By Boffy B (Own work) [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0-2.5-2.0-1.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons IMac: By Matthieu Riegler, Wikimedia Commons [CC-BY-3.0 (http://creativecommons.org/ licenses/by/3.0)], via Wikimedia Commons Ada Lovelace: By Alfred Edward Chalon [Public domain], via Wikimedia Commons Stonehenge: By Yuanyuan Zhang [All right reserved] via Flickr Bath Abbey: By Yuanyuan Zhang [All right reserved] via Flickr