Automated Software Transplantation ( ISSTA 2015 ) Earl T. Mark - PowerPoint PPT Presentation

Automated Software Transplantation ( ISSTA 2015 ) Earl T. Mark Yue Alexandru Justyna Barr Harman Jia Marginean Petke CREST, University College London Justyna Petke — Automated Software Transplantation

Genetic Improvement of Software geneticimprovementofsoftware.com Functional Properties Non-functional Properties Execution Time New Feature Memory Functionality Improvement Energy Error Justyna Petke — Automated Software Transplantation

Justyna Petke — Automated Software Transplantation

Automated Software Transplantation ( ISSTA 2015 ) Thank you Alexandru Marginean CREST, University College London Justyna Petke — Automated Software Transplantation

Why Autotransplantation? ~100 players Check open source repositories Why not handle Video Player H.264? Start from scratch Justyna Petke — Automated Software Transplantation

Automated Software Transplantation Donor ENTRY Manual Work: V Organ Entry Organ Organ Organ Test Organ’s Test Suite Suite Host Implantation Point Justyna Petke — Automated Software Transplantation

μ Trans Host Implantation Point Stage 2: Stage 3: Stage 1: Genetic Organ Static Analysis Programming Implantation Donor Organ Entry Host Organ Test Beneficiary Suite Justyna Petke — Automated Software Transplantation

Stage 1 — Static Analysis Donor Organ Vein ENTRY OE Host Implantation Matching Dependency Point Table Graph Donor: int X -> Host: int A, B, C Stm: x = 10; -> Decl: int x; Justyna Petke — Automated Software Transplantation

Stage 2 — GP Individual S 1 S 2 S 3 S 4 S 5 … S n Matching V 1H M 1 : V 1D Var M 2 : V 4H V 2D Stateme Genetic nts S 1 S 7 S 73 … Weak Proxies: Programming Does it compile? Strong Proxies: Does it Does it execute test cases without produce the correct output? crashing? Matching Table Donor Host Variable Variable ID ID (set) V 1H V 2H V 1D V 3H V 4H V 5H ( 3 (1 + | T X i | + | T P i | 1 V 2D | T | ) i ∈ I C | T | fitness ( i ) = 0 i / ∈ I C … Justyna Petke — Automated Software Transplantation

Research Questions Acceptance Tests Regression Tests Donor Host Organ RQ1: Do we break the initial RQ2: Have we really added RQ4: Is autotransplantation RQ3: How about the computational effort? new functionality? functionality? useful? Justyna Petke — Automated Software Transplantation

Research Questions RQ1: Do we break the initial RQ2: Have we really added functionality? new functionality? Empirical Study Case Study: 15 Transplantations H.264 Encoding 300 Runs Transplantation 5 Donors 3 Hosts RQ3: How about the RQ4: Is autotransplantation computational effort? useful? Justyna Petke — Automated Software Transplantation

Validation Donor Regression Acceptance Tests Tests Augmented Acceptance Regression Tests Tests RQ1.1 RQ1.2 RQ2 Host Manual Beneficiary Validation Justyna Petke — Automated Software Transplantation

f a c i t t r A * C o m p * l e t t n A e e Subjects * t s * E W i s n e C o l A l C D T * o * S c e u s S m u e E e I R n * o t v t e d y s a * E a d l e u t a Size Reg. Organ Subjects Type KLOC Tests Tests Idct Donor 2.3 - 3-5 Minimal size: 0.4k Mytar Donor 0.4 - 4 Cflow Donor 25 - 6-20 Max size: 422k Webserver Donor 1.7 - 3 TuxCrypt Donor 2.7 - 4-5 Pidgin Host 363 88 - Average Donor:16k Cflow Host 25 21 - SoX Host 43 157 - Average Host: 213k Case Study x264 Donor 63 - 5 VLC Host 422 27 - Justyna Petke — Automated Software Transplantation

Experimental Methodology and Setup Host Donor Organ Test Implantation OE Suite Point 64 bit Ubuntu 14.10 Host μ S CALPEL Beneficiary 16 GB RAM Implantation Point 8 threads Organ Justyna Petke — Automated Software Transplantation

Empirical Study a c f i t t r A * C o m * p l e t n t e A e * t s * E i W s n e C o A l l C D T * RQ1,2 o * S c e u s S u m e E e I R n * o t e v t d y s a E * a d e l u t a All Passed Regression++ Acceptance Donor Host All Passed Regression Regression Regression++ Acceptance Idct Pidgin 16 20 17 16 Mytar Pidgin 16 20 18 20 Web Pidgin 0 20 0 18 Cflow Pidgin 15 20 15 16 Tux Pidgin 15 20 17 16 Idct Cflow 16 17 16 16 Mytar Cflow 17 17 17 20 Web Cflow 0 0 0 17 Cflow Cflow 20 20 20 20 Tux Cflow 14 15 14 16 Idct SoX 15 18 17 16 Mytar SoX 17 17 17 20 Web SoX 0 0 0 17 Cflow SoX 14 16 15 14 Tux SoX 13 13 13 14 233/300 TOTAL 188/300 188/300 233/300 196/300 196/300 256/300 256/300 Alexandru Marginean — Automated Software Transplantation RQ1.1 RQ1.1 RQ1.2 RQ2 RQ2 RQ1.2 Justyna Petke — Automated Software Transplantation

f a c i t t Empirical Study r A * C o m * p l e t t n e A e t * s * E i W s n e C o l A l C D T * o * S e c u s S u m e E e I R n o t * e v t y d s a * E d a RQ3 l e u t a Execution Time (minutes) Average Total Donor Host Std. Dev. Idct Pidgin 5 7 97 Mytar Pidgin 3 1 65 Web Pidgin 8 5 160 Cflow Pidgin 58 16 1151 Tux Pidgin 29 10 574 Idct Cflow 3 5 59 Mytar Cflow 3 1 53 Web Cflow 5 2 102 Cflow Cflow 44 9 872 Tux Cflow 31 11 623 Idct SoX 12 17 233 Mytar SoX 3 1 60 Web SoX 7 3 132 Cflow SoX 89 53 74 Tux SoX 34 13 94 334 (min) 10 (Average) Total 72 (hours) Justyna Petke — Automated Software Transplantation

f a c Case Study i t t r A * C o m p * l e t t n A e e * t s * E W i s n e C o l A l C D T * o * S c e u s S m u RQ4 e E e I R n * o t v t e d y s a * E a d l e u t a Transplant Time & Test Suites Time (hours) Regression Regression++ Acceptance within 26 hours performed a task H.264 that took developers 26 100% 100% 100% an avg of 20 days of elapsed time VLC H264 Justyna Petke — Automated Software Transplantation

Justyna Petke — Automated Software Transplantation

Automated Software μ Trans Transplantation D ENTRY Host Manual Work: V Stage 2: Stage 3: Organ Entry Stage 1: O Genetic Organ Static Analysis Programming Implantation Donor Organ’s Organ’s Test Suite Test Suite H Implantation Point Organ’s Host Test Suite Beneficiary Alexandru Marginean — Automated Software Transplantation f a c i t t r A * C o m * p l e t t n e A e Validation Subjects t * s * E i W s n e C o l A l C D T * o * S e c u s S m u e E e I R n * o t t e v d y s a * E a d l e u t a Donor Regression Size Reg. Organ Acceptance Subjects Type • Minimal size: Tests KLOC Tests Test Suite Tests 0.4k; Idct Donor 2.3 - 3-5 Mytar Donor 0.4 - 4 Cflow Donor 25 - 6-20 • Max size: 422k; Augmented Acceptance Webserver Donor 1.7 - 3 Regression Tests TuxCrypt Donor 2.7 - 4-5 Tests RQ1.a • Average Donor: Pidgin Host 363 88 - Cflow Host 25 21 - RQ1.b RQ2 16k; SoX Host 43 157 - Case Study Host Manual • Average Host: Beneficiary Validation x264 Donor 63 - 5 VLC Host 422 27 - 213k; Justyna Petke — Automated Software Transplantation Alexandru Marginean — Automated Software Transplantation

Genetic Improvement of Software geneticimprovementofsoftware.com Functional Properties Non-functional Properties Execution Time New Feature EuroGP 2014 ISSTA 2015 Memory GECCO 2015 Functionality Improvement Energy TEVC 2015 GECCO 2015 Error Justyna Petke — Automated Software Transplantation

Genetic Improvement 50th CREST Open Workshop 30 & 31 Jan 2017 More information : crest.cs.ucl.ac.uk/cow/50/ Justyna Petke — Automated Software Transplantation

geneticimprovementofsoftware.com Justyna Petke — Automated Software Transplantation