Dynamic Fitness Functions for Genetic Improvement in Compilers and - - PowerPoint PPT Presentation
Oliver Krauss MSc Dynamic Fitness Functions for Genetic Improvement in Compilers and Interpreters Kyoto 16.7.2018 Advisors: Prof. Dr. Dr. h.c. Hanspeter Mssenbck, Prof. (FH) Priv.-Doz. Dipl.-Ing. Dr. Michael Affenzeller Abstract Genetic
Oliver Krauss MSc
Kyoto 16.7.2018
Advisors: Prof. Dr. Dr. h.c. Hanspeter Mössenböck,
– Improve non-functional software features (performance) – Deal with large fitness landscapes – Dynamic fitness functions ... – split test suites by the complexity of the cases – Sequential or Parallel – ... don't perform well – ... show promise
Page 1 | 26
Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook
Page 2 | 26
Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook
Page 3 | 26
∗ ∗ ∗ ∗ ∗ ∗ ∗ depth width – n node types (MiniC -> 158) – depth 5, width 5
Page 4 | 26
∗ ∗ ∗ ∗ ∗ ∗ ∗ depth width – n node types (MiniC -> 158) – depth 5, width 5 – All Options -> n3901 – MiniC -> 466.638
Page 4 | 26
∗ ∗ ∗ ∗ ∗ ∗ ∗ depth width – n node types (MiniC -> 158) – depth 5, width 5 – All Options -> n3901 – MiniC -> 466.638 – depth 10, width 10
Page 4 | 26
∗ ∗ ∗ ∗ ∗ ∗ ∗ depth width – n node types (MiniC -> 158) – depth 5, width 5 – All Options -> n3901 – MiniC -> 466.638 – depth 10, width 10 – All Options -> n11.111.111.101 – MiniC -> 80.814.253.363
Page 4 | 26
Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook
Page 5 | 26
Code
void main () { int i, n, now , prev , next; prev = 0; now = 1; i = 0; n = 30; while (i < n){ if (i == 0 ) print (0); else if (i == 1) print (1); else { next = now + prev; prev = now; now = next; print(next); } i = i + 1; } }GI
Truffle Graal Coco/R Optimize Compile
Figure: Code Interpretation and Compilation with GI (Truffle and Graal images from [1])
Page 6 | 26
– Subset of C11 [2] – Coco/R -> Creates Scanner & Parser in Java – 342 node classes (103 operators, 55 operands)
Page 7 | 26
– Interpreter based on AST – Generalized and Spezialized AST - Nodes – Self optimizing by AST-Rewriting – Can run any Guest Language on the JVM – Python, Ruby, JavaScript, ...
Page 8 | 26
– Just-in-time (JIT) compiler – Agressive optimizations – Part of OpenJDK
Page 9 | 26
AST from Coco/R Select subtree for Optimization Run in Truffle Collect Testdata
IN OUT 1 1 2 1 3 2 4 3 5 5
Create Stub GI Population Evaluate Population with Truffle Hand best solution to Truffle clone & build Profile
Get Testdata
clone AST
evaluate
cross & mutate after x Generations
Figure: GP Optimization in depth
Page 10 | 26
Figure: The architecture of the optimization framework in combination with Truffle and Graal
Page 11 | 26
Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook
Page 12 | 26
fib
{}
read
n
if
ret ret
||
==
n 1
==
n
n + fib
−
n 1
fib
−
n 2 Visited Previously Visited Spezialisation
Goal: visit all nodes – n = 0 [v=14,s=0]
Page 13 | 26
fib
{}
read
n
if
ret ret
||
==
n 1
==
n
n + fib
−
n 1
fib
−
n 2 Visited Previously Visited Spezialisation
Goal: visit all nodes – n = 0 [v=14,s=0] – n = 1 [v=11,s=0]
Page 13 | 26
fib
{}
read
n
if
ret ret
||
==
n 1
==
n
n + fib
−
n 1
fib
−
n 2 Visited Previously Visited Spezialisation
Goal: visit all nodes – n = 0 [v=14,s=0] – n = 1 [v=11,s=0] – n = 2 [v=23,s=6]
Page 13 | 26
correctness(AST,tests) = tests
n=1 succeeded(AST,test)
tests (1)
complexity(AST,test) = nodes
n=1
visited(AST,test) nodes(AST) (2)
Page 14 | 26
nodes
n=1
visited(A) ∩ visited(B) max(nodes
n=1
visited(A), nodes
n=1
visited(B)) (3)
confidence(testsuite,AST) = nodes
n=1
visited(testsuite) nodes(AST) (4)
Page 15 | 26
Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook
Page 16 | 26
Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs
Page 17 | 26
Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs
Page 17 | 26
Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs
Page 17 | 26
Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs
Page 17 | 26
Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs
Page 17 | 26
Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs
Page 17 | 26
Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs
Page 17 | 26
Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs
Page 17 | 26
Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached
Page 18 | 26
Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached
Page 18 | 26
Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached
Page 18 | 26
Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached
Page 18 | 26
Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached
Page 18 | 26
Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached
Page 18 | 26
Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached
Page 18 | 26
Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached
Page 18 | 26
Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook
Page 19 | 26
– Same configuration for all – No grafting – Comparison of solved test cases Table: Use Case statistics Use Case
Tests Nodes visited x*2 8 5 5, 5, 5, 5, 5 Sqrt 60 5 14, 42, 42, 42, 53 Fibonacci 32 5 11, 15, 30, 30, 30 Constructed 31 5 11, 15, 20, 20, 21
Page 20 | 26
x*2 always solved, except Parallel FFN
Page 21 | 26
x*2 always solved, except Parallel FFN Sqrt return 2
Page 21 | 26
x*2 always solved, except Parallel FFN Sqrt return 2 Fibonacci only first 2 (Sequential FFN)
Page 21 | 26
x*2 always solved, except Parallel FFN Sqrt return 2 Fibonacci only first 2 (Sequential FFN) Constructed 3 of 5 tests solved
Page 21 | 26
x*2 always solved, except Parallel FFN Sqrt return 2 Fibonacci only first 2 (Sequential FFN) Constructed 3 of 5 tests solved
GA Baseline
Page 21 | 26
x*2 always solved, except Parallel FFN Sqrt return 2 Fibonacci only first 2 (Sequential FFN) Constructed 3 of 5 tests solved
GA Baseline Sequential FFN Solved more tests
Page 21 | 26
x*2 always solved, except Parallel FFN Sqrt return 2 Fibonacci only first 2 (Sequential FFN) Constructed 3 of 5 tests solved
GA Baseline Sequential FFN Solved more tests Parallel FFN Total failure
Page 21 | 26
Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook
Page 22 | 26
– Sequential FFN outperformed GA – Known why all solutions failed – Can be improved
– Sequential - solutions dominate from previous – Additive mutator – Parallel - one previous repeat dominates – Merge crossover
Page 23 | 26
– Increase amount of tests – Switch correctness measure from 0|1 to 0..1
– Use indicators from original AST – Operators / Operands – Types – Restrict size with loop/branch counting
Page 24 | 26
– Sequential FFN -> Age Layered Population Structure (ALPS) – Parallel FFN -> Island GA – Implement Mutator & Crossover Operators – Update FFN to also include hot path – Your Suggestion Here
Page 25 | 26
Oliver Krauss
hagenberg.at +43 (0)50804-27195
Page 26 | 26
[1]
Proceedings of the 2013 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software, ser. Onward! 2013, Indianapolis, Indiana, USA: ACM, 2013, pp. 187–204. [Online]. Available: http://doi.acm.org/10.1145/2509578.2509581. [2] ISO, ISO/IEC 9899:2011 Information technology — Programming languages — C. Geneva, Switzerland: International Organization for Standardization,
catalogue/catalogue_tc/catalogue_detail.htm?csnumber=57853.
Page 1 | 7
[3]
in Proceedings of the 3rd Annual Conference on Systems, Programming, and Applications: Software for Humanity, ser. SPLASH ’12, Tucson, Arizona, USA: ACM, 2012, pp. 13–14. [Online]. Available: http://doi.acm.org/10.1145/2384716.2384723. [4]
Available: http://openjdk.java.net/projects/graal/. [5]
“Snippets: Taking the high road to a low level”, ACM Trans. Archit. Code Optim., vol. 12, no. 2, 20:20:1–20:20:25, Jun. 2015. [Online]. Available: http://doi.acm.org/10.1145/2764907.
Page 2 | 7
[6]
queuing and graph caching for dynamic compilers”, in Proceedings of the Sixth ACM Workshop on Virtual Machines and Intermediate Languages,
Available: http://doi.acm.org/10.1145/2414740.2414750. [7]
action, https://www.youtube.com/watch?v=4pdiAneMMhU, [Last Accessed 14.07.2018], 2014.
Page 3 | 7
Crossover A single point crossover was selected Mutator A single point mutator was selected. The point to be mutated is selected randomly by the mutator. Mutation Probability was set to 57% Selector crossover and mutator use a tournament selection PopulationSize was set to 200 individuals Generations each run was 20 generations. GA, and the sequential FFN took 80 generations over 4 restarts. Parallel FFN took 60 generations initially and 20 generations after restart = 80 generations. Elitism per generation the 3 best individuals were preserved Restart-Elitism 10 best individuals. Parallel FFN had 30 individuals carried over (10 from each group). Operators and Operands all AST nodes of the MiniC language were selected as operators and operands for each test case.
Page 4 | 7
int constructed(int n) { if (n == 0) { return 1; } if (n == 1) { return 2; } if (n == 2) { return n + 1; } return n * n; }
Page 5 | 7
– Runs in same JVM influence each following run – 200.000 runs for measurement due to warm up phase
Page 6 | 7
Figure: Regular Static Fitness Landscape [7] Figure: Test-based Fitness Landscape
Page 7 | 7