Erlang Code Evolution Control David Insa 1 , Sergio Prez 1 , Josep - PowerPoint PPT Presentation

Erlang Code Evolution Control David Insa 1 , Sergio Pérez 1 , Josep Silva 1 , Salvador Tamarit 2 1 Universitat Politècnica de València 2 Universidad Politécnica de Madrid LOPSTR 2017 12/10/2017

Content • Introduction • The technique in detail • Type Analysis Phase • Test Case Generation Phase • Comparison Phase • Recordingthe trace • SecEr tool • Conclusions& Future Work

Introduction • In debugging programmers use breakpoints to observe the values of an expression during an execution • Is this feature available in testing? • It would be useful to focus the test cases on an specific point without code modifications OUR PROPOSAL Ø Introduce the ability to specify Points Of Interest (POI) in the context of testing Ø A technique to compare two equivalent POIs in different versions of the same program for Erlang

Introduction 1.- Identify a POI and a set of input functions Old Version New Version 2.- A test suite is automatically generated main(X,Y) -> main(X,Y) -> Each test case contains: A = X + Y, A = add(X,Y), • A call to an input function with specific arguments • The sequence of values the POI is evaluated to (trace) D = X – Y, D = sub(X,Y), 3.- Each test case is passed against the new A * D. A * D. version and both traces are compared add(X,Y)-> 4.- A report of the success or failure of the test main(5,4) X + Y. cases is provided OldVersionTrace: 1 NewVersionTrace: 1 sub(X,Y) -> We have implemented our approach for Erlang Success X – Y. in a tool named SecEr

Introduction 1.- Identify a POI and a set of input functions Old Version New Version 2.- A test suite is automatically generated main(X,Y) -> main(X,Y) -> Each test case contains: A = X + Y, A = add(X,Y), • A call to an input function with specific arguments • The sequence of values the POI is evaluated to (trace) D = X – Y, D = sub(X,Y), 3.- Each test case is passed against the new A * D. A * D. version and both traces are compared add(X,Y)-> 4.- A report of the success or failure of the test main(5,4) main(5,4) X + Y. cases is provided OldVersionTrace: 1 OldVersionTrace: 1 NewVersionTrace: -1 NewVersionTrace: 1 sub(Y,X) -> We have implemented our approach for Erlang Failure Success X – Y. in a tool named SecEr

Content • Introduction • The technique in detail • Type Analysis Phase • Test Case Generation Phase • Comparison Phase • Recordingthe trace • SecEr tool • Conclusions& Future Work

The technique in detail Type Test Case Comparison Final Analysis Generation Test Input Report Phase Case Phase Phase

The technique in detail Type Test Case Comparison Final Analysis Generation Test Input Report Phase Case Phase Phase

Type analysis phase Module.erl Types Input TypEr TypEr PropEr PropEr CutEr CutEr Input Function

Type analysis phase .erl TypEr 〈 Clause Input TypEr Types PropEr CutEr Inputs + new code ,Types 〉 Fun g(0,0)->… “-spec g(0|1, 0|1)->…” g(1,1)->… 1.- The infered types refer to the whole function g(0,1) Solution: Consider each clause independently and refine the result to types per clause f(A,[A,B]) “-spec f(1|2,[1|2|5|6])->…” 2.- The length of the list is unkwnown f(1,[1,2,5,6]) 3.- The repeated-variable restrictionis ignored f(1,[2,5])

Type analysis phase .erl TypEr 〈 Clause Input PropEr CutEr Inputs + new code ,Types 〉 Fun PropEr Property PropEr . . . true | false . . . gen gen test Value Types prop_identity() -> ?FORALL(X, any(), id(X) = X). id(X) -> X.

Type analysis phase .erl TypEr PropEr 〈 Clause Input PropEr CutEr Inputs + new code + new code ,Types 〉 Fun TypEr PropEr ≠ types types TypEr types PropEr PropEr TypEr to Input 〈 Clause types PropEr gen ,Types 〉 Clause Parameters f(A,[A,B]) 2.- The length of the list is unknown Solution: Traverse the list parameters of the clause element by element 3.- Repeated variable relation is lost Solution: Store the values of already treated variables

Type analysis phase .erl TypEr PropEr CutEr 〈 Clause Input CutEr Inputs + new code + new code + new code ,Types 〉 Fun CutEr Inputs concolic Input . . . Inputs gen Runtime . . . Errors

The technique in detail Type Test Case Comparison Final Analysis Generation Test Input Report Phase Case Phase Phase

Test case generation phase Continue until a limit is reached .erl 1 Store all the generated test cases (Input + Trace) code instrumentator POI 1 Test Case .erl 1 ’ T mutating input Trace 1 New Input execute New? Input gen selector Input F New random Input gen add

The technique in detail Type Test Case Comparison Final Analysis Generation Test Input Report Phase Case Phase Phase

Comparison phase .erl 2 Result Result code analyzer instrumentator POI 2 add Comparison .erl 2 ’ Final Result Report Input test case trace Test Test case Trace 2 execute selector comparator Case Trace 1

Content • Introduction • The technique in detail • Type Analysis Phase • Test Case Generation Phase • Comparison Phase • Recordingthe trace • SecEr tool • Conclusions& Future Work

Recording the trace • There are several tools for tracing executions in Erlang • None of them allows us to collect the trace of patterns • Debuggers will not provide a value for a POI if it is inside an expression whose evaluation fails {1,B,3} = {1,2,4} OUR PROPOSAL Ø Collect the traces as a side effect when executing the code Ø Approach based on message passing to a tracing server Ø The code needs to be instrumented (4 STEPS)

Recording the trace (1 & 2) 1. Obtain and annotate the Abstract Syntax Tree of the program. Annotate each node with two lists of variables: Variables beingbound in its subtree § Variables that were alreadybound when reaching the node § A 2. Find the selected POI in the AST with a top-down traversal: B C (B,1) Store the current traversed path with tuples of the § form (Node,ChildIndex) D E (E,2) The result is a path that yields directly to the POI § F G (F,1) POI (POI,1)

Recording the trace (3) 3. Analyze the location of the POI • Expressions: Add a send command to inform the tracing server • Patterns: Need special treatment {1,B,3} = {1,2,4} 2 Target expressions • Pattern-matching • List comprehension • Expressions with clauses: • if • case • functions • etc.

Recording the trace (3) Divide the AST path into two sub-paths: PathBefore: Root -> deepest target expression • PathAfter: First child of the target expression -> POI • A PathBefore (B,1) B B C (B,1),(TE,2) D TE TE (TE,2) PathAfter F F G (F,1) (F,1),(POI,1) POI (POI,1) POI

Recording the trace (4) 4. Perform the actual instrumentation • Traverse the PathBefore • Transform the code following a rule according to PathAfter • TraversePathBefore backwards to update the AST A Five exclusive rules to instrument expressions B B C • LEFT_PM (pattern-matching) D TE TE • PAT_GEN_LC (list comprehensions) New • CLAUSE_PAT (pattern in expressions with clauses) Subtree F F G • CLAUSE_GUARD (guard in expressions with clauses) POI • EXPR (expressions)

Recording the trace (4) (LEFT_PM) p = e → p = begin np = e, tracer!{add, p = e → p = begin np = e, tracer!{add, npoi}, np end npoi}, np end if (p = e, _ ) = last(PathBefore ) (p = e, _ ) = last(PathBefore ) ⋀ (_, pos(p)) = hd(PathAfter) ⋀ (_, pos(p)) = hd(PathAfter) where (_, npoi, np) = pfv(p,PathAfter) (_, npoi, np) = pfv(p,PathAfter) {1,B,3} = begin p = begin {1,POI,FV} = {1,2,4}, np = e {1,B,3} = {1,2,4} tracer ! {add,POI}, tracer!{add, npoi} {1,POI,FV} np p = e end end

Content • Introduction • The technique in detail • Type Analysis Phase • Test Case Generation Phase • Comparison Phase • Recordingthe trace • SecEr tool • Conclusions& Future Work

SecEr tool Old_Version_POI SecEr command: $ ./secer -f FILE –li LINE – var VARIABLE [-oc OCCURRENCE] -f FILE –li LINE – var VARIABLE [-oc OCCURRENCE] [-funs INPUT_FUNCTIONS] –to TIMEOUT New_Version_POI

SecEr tool happy1.erl happy0.erl $ ./secer -f happy0.erl –li 9 – var Happy -oc 1 -f happy1.erl –li 18 – var Happy -oc 1 -funs [main/2] –to 15

SecEr tool

SecEr tool

SecEr tool

Content • Introduction • The technique in detail • Type Analysis Phase • Test Case Generation Phase • Comparison Phase • Recordingthe trace • SecEr tool • Conclusions& Future Work

Conclusions Conclusions • Combination of Erlang existing tools and mutation to improve the result • New approach to automatically check the behaviour preservation between versions Test Case Type Analysis Comparison Final Generation Test Input Report Phase Phase Case Phase • New tracing process that allows for placing a POI in patterns, guards and expressions {1,B,3} = begin {1,POI,FV} = {1,2,4}, {1,B,3} = {1,2,4} tracer ! POI, {1,POI,FV} end