Testing Model Transformations in Model Driven Engineering Benoit Baudry INRIA – IRISA (currently visiting CSU)
Outline � What about model transformation testing? � Triskell’s contributions – Coverage criteria – Model synthesis � Related work � Challenges 2
Model Transformation Testing: Motivation borrow requirement 1.1 "Registering a book" the "book" becomes "registered" return after the "librarian" did "register« the "book". deliver the "book" is "available" after the "librarian" did "register« setDamaged the "book". rese rve Book User .... state : String Code 3
Model Transformation Testing: Motivation � A transformation is meant to be reused – But also has to be adapted from one project to another � A transformation is meant to hide the complexity – we woud like to trust the transformation as we trust a compiler 4
Dynamic testing process Test data Program Execution Test data Result Specification generation Oracle Oracle Localization / false Verdict Problems Debbuging true non vérifié Stopping criterion Diagnosis Test data evaluation 5
Dynamic transformtation testing process Test data / test model Model transformation Execution Test model Result / Specification generation output model Oracle Oracle Localization / false Verdict Problems Debbuging true non vérifié Stopping criterion Diagnosis Test data evaluation 6
Dynamic transformtation testing process � Specific issues � Complex data – Models are manipulated as sets of objects � Complex constraints � Lack of specific tools 7
Model Transformation Testing � Currently in Triskell – Coverage criteria – Automatic synthesis of test models (in coll. With Mc Gill) – Specific fault models 8
Model transformation Source metamodel Transformation Target metamodel structure + constraints language structure + constraints Input model Transformation Output model pre post condition condition 9
Test data generation: criteria � Several model transformation languages – Different features – Different paradigms – Different domains � We did not want to choose � We define black-box criteria – Independent of the model transformation language 10
Test data generation: criteria � Define test criteria based on the input metamodel – Intuition: a set of models is adequate for testing if every class of the input metamodel is instantiated at least once and if the properties have relevant values � A model for testing is called a test model 11
Test data generation: Example outgoingTransition source Transition AbstractState 0..* 1 ownedState event: String label: Integer incomingTransition target 0..* 0..* 1 Possibly infinite set of models => Need stopping criteria State 0..1 Composite isFinal: Boolean container isInitial: Boolean What we expect from test models evt1 4 1 3 •Every class to be instantiated evt1 evt2 1 2 1 1 •Properties to take several relevant values evt2 evt3 evt3 2 3 2 •Combine properties in a meaningful way 12
Relevant values for properties label = 0 label = 1 outgoingTransition source label > 1 Transition AbstractState 0..* 1 ownedState event: String label: Integer incomingTransition target 0..* 0..* 1 0 incomingTransition 1 incomingTransition 0..1 State >1 incominTransitions Composite isFinal: Boolean container isInitial: Boolean Adapt category partition testing to define ranges of relevant values for properties of the metamodel 13
Relevant values for properties � Define partitions for each property in the input metamodel � A partition defines a set of ranges on a domain – choose one value in each range for the property � Example – partition for AbstractState::label={[0],[1],[2..MaxInt]} – A set of test models will need to have, at least three states with three different values for label 14
Relevant values for properties outgoingTransition source Transition AbstractState 0..* 1 ownedState event: String label: Integer incomingTransition target 0..* 0..* 1 State 0..1 Composite isFinal: Boolean container isInitial: Boolean Transition:: event {''}, {'evt1'}, {'.+'} Transition::# source {1} {1} Transition::# target AbstractState:: label {0}, {1}, {2..MaxInt} {0}, {1} AbstractState::# container {0}, {1}, {2..MaxInt} AbstractState::# incomingTransition {0}, {1}, {2..MaxInt} AbstractState::# outgoingTransition State: isInitial {true}, {false} State:: isFinal {true}, {false} 15 Composite::# ownedState {0}, {1}, {2..MaxInt}
Relevant object structures Transition:: event {''}, {'evt1'}, {'.+'} Transition::# source {1} {1} Transition::# target AbstractState:: label {0}, {1}, {2..MaxInt} {0}, {1} AbstractState::# container {0}, {1}, {2..MaxInt} AbstractState::# incomingTransition {0}, {1}, {2..MaxInt} AbstractState::# outgoingTransition State: isInitial {true}, {false} State:: isFinal {true}, {false} Composite::# ownedState {0}, {1}, {2..MaxInt} We would like to constrain the models to have a State with one outgoing transition and more than one incoming transitions 16
Relevant object structures constraints constraints PropertyConstraint PropertyConstraint ObjectFragment ObjectFragment 1 1 property property 1..* 1..* property property Partition Partition org::omg::mof2::emof::Property org::omg::mof2::emof::Property 1..* 1..* object object 1 1 range range range range 1 1 default: String default: String Range Range ModelFragment ModelFragment 1..* 1..* isComposite: Boolean isComposite: Boolean isDerived: Boolean isDerived: Boolean ValuePartition ValuePartition «enumeration» «enumeration» * * ownedAttribute ownedAttribute BooleanValue BooleanValue range range StringRange StringRange IntegerRange IntegerRange false: 0 false: 0 1..* 1..* regexp: String [1..1] regexp: String [1..1] true: 1 true: 1 1..* 1..* intervals intervals MultiplicityPartition MultiplicityPartition 0..1 0..1 owningClass owningClass IntegerInterval IntegerInterval BooleanRange BooleanRange org::omg::mof2::emof::Class org::omg::mof2::emof::Class lower: Integer lower: Integer values: BooleanValue values: BooleanValue upper: Integer upper: Integer isAbstract: Boolean isAbstract: Boolean 17
Relevant object structures � Criteria define structures that must be covered by test models � These criteria combine partitions � One criterion = set of constraints – one criterion declares the set of ranges that should be covered by a set of test models � Example – Range coverage: Each range of each partition for all properties of the meta-model must be used in at least one model. 18
Test criteria � Six test criteria (different combinations of ranges) – AllRanges – AllPartitions – + 4 class criteria � object fragments constrain each property of the object � Do not consider constraints on the metamodel – Might generate insatisfiable fragments 19
Evaluating a set of models � A prototype tool: MMCC – Framework for partitions and fragments definitions � Computes a set of model fragments according to – Input metamodel – Test criterion � Checks the coverage of a set of test models – With respect to the set of model fragments 20
Automatic synthesis of test models � Automatic synthesis useful to – Limit the effort for test generation – Evaluate the test criteria � Challenges: – Combine different sources of knowldege – Expressed in different formalisms – Complex constraints 21
Automatic synthesis of test models Model Test Model Knowledge Transformation 1.Test Model Objectives Meta-model Pre-condition 2.Model Fragments specifies specifies specifies Test Models 22
The Solution(1): Combining Knowledge to Common Constraint Language OCL Requirements/ Objects and OCL Requirements/ Objects and ECORE OCL ECORE OCL Constraint Natural Language Property ranges Constraint Natural Language Property ranges Model Constraints Model Constraints expressed as expressed as Pre- Test Model expressed as Pre- Test Model expressed as condition Objectives condition Objectives Meta- Model Meta- Model model Fragments model Fragments transformed to Common Constraint Common Constraint Language: Language: Alloy Alloy 23
Model synthesis The run command: Integer scope run test_requirement1 for 1 ClassModel,5 int, exactly 5 Class, exactly 20 Attribute, exactly 4 PrimitiveDataType, exactly 5 Association Exact number of objects 1. Specify a scope 2. Specify an exact number of objects Output: Alloy model instance that satisfies meta-model + pre-condition + test_requirement1 and has the specified size 24
CARTIER: OVERALL FRAMEWORK 25
Perspectives on model synthesis � Strenghten the tool – Automate what can be � Experiments � Design experiments to test model transformations � We want to numerically estimate via mutation analysis the efficiency of test models 26
27 Mutation Analysis
Mutation Analysis � Evaluate the set of models – Producing a Mutation Score Mutant 1 2 3 4 5 6 7 8 proportion Killed 5/8 results mutants P of P killed insert the mutation yes operators oracle sufficient Test mutant set Mutant mutants results alive no suppress the equivalent mutants improve the test set 28
Recommend
More recommend
