Real Tim e TRON TRON TRON Testing using UPPAAL W ith Mariius - PowerPoint PPT Presentation

Real Tim e TRON TRON TRON Testing using UPPAAL W ith Mariius Mikucionis, Brian Nielsen, Arne Skou, Anders Hessel, Paul Pettersson

Overview � Introduction Informationsteknologi � Conformance for Real-Time System � Off-line Test Generation � Controllable Timed Automata CLASSI C CORA CLASSI C CORA CLASSI C CORA � Observable Timed Automata TI GA TI GA TI GA � On-line Test Generation TRON TRON TRON � Conclusion and Future Work

Testing Primary validation technique used in industry � • In general avg. 10-20 errors per 1000 LOC Informationsteknologi • 30-50 % of development time and cost in embedded software To find errors � To determine risk of release � Part of system development life-cycle � Output System Environ- Input Under Test m ent Expensive, error prone, time consuming (for Real-Time Systems) � UPPAAL model can be used to generate test specifications �

Real-tim e Model-Based Testing Plant Controller Program Informationsteknologi Continuous Discrete sensors actuators Conform s-to? 1 2 Test generation a inputs 1 2 (offline or 3 4 online) wrt. b c 3 4 Design Model 1 2 a outputs 1 2 a 3 4 b c b c 3 4 UPPAAL Model

Conform ance Relation Specification Implementation give? Informationsteknologi coin? coin? give? coin? give? •Timed Automata with Timed-LTS semantics • I nput actions (?) are controlled by the environment • Output actions (!) are controlled by the implementation •Implementations are input enabled • Testing hypothesis: IUT can be modeled by some (unknown) TA

? I5 I4 I8 Does I n conform -to S 1 ? I3 I2 I7 I1 I6 S1 Informationsteknologi

Tim ed Conform ance • Derived from Tretman’s IOCO Informationsteknologi • Let I , S be timed I/O LTS, P a set of states l2 • TTr (P): the set of timed traces from P •eg.: σ = coin?. 5 .req?. 2 .thinCoffee!. 9 .coin? • Out (P after σ ) = possible outputs and delays after σ •eg. out ({l2,x=1}): {thinCoffee, 0 ...2 } • I rt-ioco S = def • ∀σ ∈ TTr( S) : Out( I after σ ) ⊆ Out( S after σ ) • TTr( I ) ⊆ TTr( S) if s and I are input enabled • I ntuition • no illegal output is produced and • required output is produced ( at right tim e) See also [Krichen&Tripakis, Khoumsi]

Does I n conform -to S 1 ? S1 I1 Informationsteknologi σ =coin.give.10 σ∈ TTr (I1), σ ∉ TTr (S1) out (I1 after coin.give.3)={0... ∞ } ⊄ out (S1 after coin.give.3)={coffee,0…2}

Does I n conform -to S 1 ? S1 I3 I4 Informationsteknologi σ =coin.give.7.coffee σ =coin.give.1.coffee σ∈ TTr (I3), σ ∉ TTr(S1) σ∈ TTr (I4), σ ∉ TTr(S1) out (I3 after coin.give.7)={coffee,0} out (I4 after coin.give.1)={coffee,0...4} ⊄ ⊄ out (S1 after coin.give.7)={} out (S1 after coin.give.1)={0...4}

Does I n conform -to S 1 ? I8 S1 I7 Informationsteknologi σ =token.5.vodka σ∈ TTr (I8), σ ∉ TTr (S1) σ =coin.give.5.tea But σ was not specified σ∈ TTr (I7), σ ∉ TTr (S1) out (I7 after coin.give.5)={tea, coffee,0} ⊄ out (S1 after coin.give.5)={coffee,0}

Sam ple Cooling Controller IUT-model Env-model C r On! Informationsteknologi Off! Low? Med? High? •When T is high (low) switch on (off) cooling within r secs. •When T is medium cooling may be either on or off (impl freedom)

Environm ent Modeling Temp. � E M Any action possible at any time High! � E 1 Only realistic temperature variations Informationsteknologi � E 2 Temperature never increases when cooling Med! � E L No inputs (completely passive) Low! time E M E 1 E 2 E L E L E 2 E 1 E M

I m plem entation relation Relativized real-tim e io-conform ance ε 0 ,i 0 , ε 1 ,i 1 … E S I System Environm ent I UT ε 0’ ,o 0 , ε 1’ ,o 1 … Model Informationsteknologi assum ptions • E,S, I are input enabled Timed LTS • Let P be a set of states • TTr ( P ): the set of timed traces from states in P • P after σ = the set of states reachable after timed trace σ • Out ( P ) = possible outputs and delays from states in P •I rt-ioco E S = def ∀σ ∈ TTr(E): Out((E,I) after σ ) ⊆ Out((E,S) after σ ) •I rt-ioco E s iff TTr(I) ∩ TTr(E) ⊆ TTr(S) ∩ TTr(E) / / input enabled • I ntuition, for all assum ed environm ent behaviors, the I UT • never produces illegal output, and • alw ays produces required output in tim e

Re-use Testing Effort Given I, E, S � Assume I rt-ioco E S � Informationsteknologi 1. Given new (weaker) system specification S’ If S S’ then I rt-ioco E S’ 2. Given new (stronger) environment specification E’ If E’ E then I rt-ioco E’ S

Off-Line Test Generation Controllable Tim ed Autom ata

Model Based Conform ance Testing Model Test suite pass Informationsteknologi DBLclick! Test Test Test Test execution Test Gene- execution x>=2 Test Gene- click? tool Generator tool x:=0 rator Generator rator Event tool fail tool tool tool click? mapping x<2 Driver Selection & optimization I m p l e m e n t a t i o n U n d Implementation Relation e r T e s t Does the behavior of the ( blackbox ) implementation comply to that of the specification?

Model-Based Testing Output Informationsteknologi System Environm ent Input Model Model Σ FSM model of system and environment � Determinizable/deterministic models � Test purpose P ≈ reachability property φ P � Test-case generation ≈ witness generation � Test input sequence σ φ p = i 0 ,i 1 ,i 2 ,… � Test suite T = { σ 1 , …, σ n }, minimized by excluding � all σ i substring of some other σ j

Testing Verdict o 0 ,o 1 ,o 2 ,… Informationsteknologi System Test i 0 ,i 1 ,i 2 ,… Under Test Program Test program σ φ p = i 0 ,i 1 ,i 2 ,… � Test in/output δ φ p = i 0 ,o 0 ,i 1 ,o 1 ,i 2 ,i 3 ,… � Test Verdict: � OK, if δ φ p = i 0 ,o 0 ,i 1 ,o 0 ,i 2 ,i 3 ,… run of system model � NOK, otherwise �

Testing Real-Tim e System s ε 0’ ,o 0 , ε 1’ ,o 1 … System Informationsteknologi Environm ent ε 0 ,i 0 , ε 1 ,i 1 … Model Model Test input sequence σ φ p = ε 0 ,i 0 , ε 1 ,i 1 , ε 2 ,i 2 ,… � Test in/output δ φ p = ε 0 ,i 0 , ε 1 ,o 0 , ε 1 ,i 1 ,o 1 ,… � Test Verdict: � OK, if δ φ p = ε 0 ,i 0 , ε 1 ,o 0 , ε 1 ,i 1 ,o 1 ,… run of system model � NOK, otherwise � Timed Automata? �

This w ork Test case generation from timed automata � Informationsteknologi by reachability analysis � implementation in UPPAAL � Testing Criteria: � single test purpose � coverage criteria: location, branching, definition/use � pairs, etc. Optimality: � Test Cases: σ φ p = ε 0 ,i 0 , ε 1 ,i 1 , ε 2 ,i 2 ,… with minimum cost � e.g. min ( ε 0 + ε 1 + …+ ε n ) Test Suites: T = { σ 1 , …, σ n } with minimum cost �

Controllable Tim ed Autom ata I nput Enabled : Assumption about all inputs can always be accepted. model of SUT Informationsteknologi Output Urgent : enabled outputs will occur immediately. Determ inism : two transitions with same input/output leads to the same state. I solated Outputs : if an output is enabled, no other output is enabled.

Exam ple Light Controller Informationsteknologi

Off-Line Testing = Optim al Reachability transition covered Informationsteknologi � Specific Test Purposes � Specific Test Purposes � Model Coverage � Model Coverage � Optimal test-suites � Optimal test-suites

Off-Line Testing = Optim al Reachability Fastest Transition Coverage =12600 ms transition covered Informationsteknologi //13 out(IGrasp); //@900 // Bring dimmer PassiveDn->ActiveDN-> out(IGrasp); //touch:switch light on silence(500);//hold // ActiveUP+increase to level 10 silence(200); silence(1000); in(OSetLevel,1); out(IRelease); silence(1000); in(OSetLevel,2); in(OSetLevel,0); silence(1000); in(OSetLevel,3); silence(1000); in(OSetLevel,4); out(IGrasp); //@200 // touch: switch light off silence(1000); in(OSetLevel,5); silence(200); silence(1000); in(OSetLevel,6); out(IRelease);//touch silence(1000); in(OSetLevel,7); in(OSetLevel,0); silence(1000); in(OSetLevel,8); silence(1000); in(OSetLevel,9); //9 silence(1000); in(OSetLevel,10 out(IGrasp); //@400 //Bring dimmer from ActiveUp silence(1000); in(OSetLevel,9); //bring dimm State to ActiveDN silence(500); //hold //To Passive DN (level=0) in(OSetLevel,0); out(IRelease); //check release->grasp is ignored out(IRelease); out(IGrasp); //@12400 Page 1 out(IRelease); Page 2 silence(dfTolerance);

Off-Line Testing = Optim al Reachability transition covered Informationsteknologi 1 W 5 0 W 1 0 0 W 1 W � Specific Test Purposes � Specific Test Purposes � Model Coverage � Model Coverage � Optimal test-suites � Optimal test-suites

Tim ed Autom ata ( E) FSM+ clocks+ guards+ resets Informationsteknologi T_sw=4 T_idle=20 WANT: if touch is issued twice quickly then the light will get brighter; otherwise the light is turned off. Solution: Add real-valued clock x