UPPAAL Model Checking, Performance Analysis and Testing of Real - - PowerPoint PPT Presentation

▶
uppaal
SMART_READER_LITE
LIVE PREVIEW

UPPAAL Model Checking, Performance Analysis and Testing of Real - - PowerPoint PPT Presentation

Feb 16, 2024 46 likes •539 views

UPPAAL Model Checking, Performance Analysis and Testing of Real Time Systems Kim G. Larsen CISS Aalborg University DENMARK CISS Center For Embedded Software Systems Regional ICT Center (2002- ) 3 research groups

slide-1
SLIDE 1

UPPAAL

Model Checking, Performance Analysis and Testing of Real Time Systems

Kim G. Larsen

CISS – Aalborg University

DENMARK

slide-2
SLIDE 2

CISS –

Center For Embedded Software Systems

Regional ICT Center (2002- )

  • 3 research groups
  • Computer Science
  • Control Theory
  • Hardware
  • Wireless Communication
  • 20 Employed
  • 25 Associated
  • 20 PhD Students
  • 50 Industrial projects
  • 10 Elite-students
  • 140+ MDKK
  • ARTIST Design
  • ARTEMIS
  • ... ...

Kim G. Larsen [2]

FM Forum -- Model Checking in Action -- Kim G Larsen

Characteristica:

  • Dedicated function
  • Complex environment
  • SW/HW/Mechanics
  • Networked
  • Autonomous
  • Ressource constrained

: Energy : Bandwidth : Memory : …

  • Timing constraints
slide-3
SLIDE 3

Model Checking & Performance Analysis

slide-4
SLIDE 4

Origin of UPPAAL

TAU

CCS & Modal Transition Systems Refinements Modal Mu-Calculus Explicit State Representation Prolog

EPSILON

TCCS Timed Refinements Timed Mu-Calculus Regions Prolog< 1989 1989 1993 1993

UPPAAL

Timed Automata TCTL Zones C++ & Java 1995 1995

FM Forum -- Model Checking in Action -- Kim G Larsen Kim Larsen [4]

2007 2007

UP4ALL

2013 2013

CAV Award

slide-5
SLIDE 5

Contributors

@UPPsala

  • Wang Yi
  • Paul Pettersson
  • John Håkansson
  • Anders Hessel
  • Pavel Krcal
  • Leonid Mokrushin
  • Shi Xiaochun

@AALborg

  • Kim G Larsen
  • Alexandre David
  • Gerd Behrman
  • Arne Skou
  • Brian Nielsen
  • Jacob I. Rasmussen
  • Marius Mikucionis
  • Thomas Chatain

@Elsewhere

−

Emmanuel Fleury, Didier Lime, Johan Bengtsson, Fredrik Larsson, Kåre J Kristofgersen, T

  • bias Amnell, Thomas Hune, Oliver Möller, Elena Fersman,

Carsten Weise, David Griffjoen, Ansgar Fehnker, Frits Vandraager, Theo Ruys, Pedro D’Argenio, J-P Katoen, Jan T retmans, Judi Romijn, Ed Brinksma, Martijn Hendriks, Klaus Havelund, Franck Cassez, Magnus Lindahl, Francois Laroussinie, Patricia Bouyer, Augusto Burgueno, H. Bowmann, D. Latella, M. Massink, G. Faconti, Kristina Lundqvist, Lars Asplund, Justin Pearson...

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [5]

slide-6
SLIDE 6

UPPAAL Model Checker

Editor Simulator

Kim Larsen [6]

FM Forum -- Model Checking in Action -- Kim G Larsen

Verifier Performance Analyses

Discrete Control Concurrency Continuous Aspects Stochasticity Timing Constraints Resources

slide-7
SLIDE 7

Timed Automata

ADD a clock x

Synchronizing action Clock Guard Conjunctions of x~n x: real-valued clock Reset

[Alur & Dill’89]

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [7]

slide-8
SLIDE 8

Semantics

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [8]

Semantics in UPPAAL

slide-9
SLIDE 9

Train Crossing

Time River Bridge tracks

Safe Approaching Crossing Safe

3 – 5 20

FM Forum -- Model Checking in Action -- Kim G Larsen

[9]

slide-10
SLIDE 10

Train Crossing

Time River Bridge tracks

Safe Approaching Crossing Safe Safe Approaching Crossing Safe

Stop the train while it still stoppable! 10 3 – 5 20

FM Forum -- Model Checking in Action -- Kim G Larsen

[10]

slide-11
SLIDE 11

Train Crossing

Time River Bridge tracks

Safe Approaching Crossing Safe Safe Approaching Crossing Safe

10 3 – 5 20

Stopped Crossing Safe Restarted Stopped Crossing Safe

7 – 15

Crossing Restarted

FM Forum -- Model Checking in Action -- Kim G Larsen

[11]

slide-12
SLIDE 12

Train Crossing

Safe Approaching Crossing Safe Stopped Restarted

Add timing+ synchronization

FM Forum -- Model Checking in Action -- Kim G Larsen

[12]

slide-13
SLIDE 13

Editor

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 13]

GUI

  • Unlimited undo and redo
  • Syntax and bracket highlighting
  • Rectangular selection
  • Customization of colors
  • T
  • oltip
  • Hiding of information
  • Improved help menu with search

component Language

  • User defjned functions (C-like)
  • New types (records, type

declarations, meta variables, scalars)

  • Partial instantiation of templates
  • Select clauses on edges
  • Forall and exist quantifjers
slide-14
SLIDE 14

Concrete Simulator

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 14]

Graphical Simulator

  • visualization

and recording

  • inexpensive fault detection
  • inspection of error traces
  • Message Sequence Charts
  • Gannt Charts

Graphical Simulator

  • visualization

and recording

  • inexpensive fault detection
  • inspection of error traces
  • Message Sequence Charts
  • Gannt Charts
slide-15
SLIDE 15

Symbolic Simulator

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 15]

Graphical Simulator

  • visualization

and recording

  • inexpensive fault detection
  • inspection of error traces
  • Message Sequence Charts
  • Gannt Charts

Graphical Simulator

  • visualization

and recording

  • inexpensive fault detection
  • inspection of error traces
  • Message Sequence Charts
  • Gannt Charts
slide-16
SLIDE 16

Verifjer

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 16]

Verifier

  • Exhaustive & automatic

checking of requirements

  • .. including validating, safety, liveness,

bounded liveness and response properties

  • .. performance properties,

e.g probabilistic and expectation.

  • .. generation of debugging information

for visualisation in simulator.

  • .. plot composer

Verifier

  • Exhaustive & automatic

checking of requirements

  • .. including validating, safety, liveness,

bounded liveness and response properties

  • .. performance properties,

e.g probabilistic and expectation.

  • .. generation of debugging information

for visualisation in simulator.

  • .. plot composer
slide-17
SLIDE 17

Demo

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 17]

slide-18
SLIDE 18

Evolution of Performance

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 18]

slide-19
SLIDE 19

Evolution of Code Base

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 19]

Client-Server Architecture GUI: Java Engine: C++ Platforms: Linux, MacOS, Solaris, Windows 3 major cycles.

slide-20
SLIDE 20

THE ”secret” of UPPAAL

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 20]

slide-21
SLIDE 21

Zones & DBMs THE ”secret” UPPAAL

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 21]

  • DBM package
  • Minimal Constraint

Form [RTSS97]

  • Clock Difgerence

Diagrams [CAV99]

  • PW List

[SPIN03]

x1 x2 x3 x0

  • 4

4 2 2 5 3 3

  • 2
  • 2

1

slide-22
SLIDE 22

UPPAAL as a back-end

  • Vooduu: verifjcation of object-oriented designs using

Uppaal, 2004.

  • Moby/RT: A T
  • ol for Specifjcation and Verifjcation of

Real-Time Systems, 2000.

  • Formalising the ARTS MPSOC Model in UPPAAL, 2007
  • Marte UML  UPPAAL , 2003.
  • Yggdrasil: Statechart  UPPAAL, 2003
  • Component-Based Design and Analysis of Embedded

Systems with UPPAAL PORT, 2008

  • Verifjcation of COMDES-II Systems Using UPPAAL with

Model Transformation, 2008

  • METAMOC: Modular WCET Analysis Using UPPAAL, 2010.
  • … …

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 22]

slide-23
SLIDE 23

Industrial Usage

some examples

slide-24
SLIDE 24

Bang & Olufsen (1997)

  • Bug known to exist for 10

years

  • Ill-described:

2.800 loc + 3 fmowchart + 1 B&O eng.

  • 3 months for modeling.
  • UPPAAL detects error with

1.998 transition steps (shortest)

  • Error trace was confjrmed

in B&O laboratory.

  • Error corrected and verifjed

in UPPAAL.

  • Follow-up project.

Arne Skou, Klaus Havelund 1st RTSS’97 talk, Klaus Havelund

FM Forum -- Model Checking in Action -- Kim G Larsen Kim Larsen [ 24]

slide-25
SLIDE 25

Bang & Olufsen (2001)

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 25]

slide-26
SLIDE 26

MECEL AB (1998)

Gear Controller

Lindahl, Pettersson, Yi 1998

Network Canbus GearBox Engine Interface Clutch GearControl

Flowgraph

Paul Pettersson

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 26]

slide-27
SLIDE 27

MECEL AB (1998)

Gear Controller

Lindahl, Pettersson, Yi 1998

Network Canbus GearBox Engine Interface Clutch GearControl

Flowgraph

Paul Pettersson

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 27]

slide-28
SLIDE 28

MECEL AB (1998)

Gear Controller

Lindahl, Pettersson, Yi 1998

Network Canbus GearBox Engine Interface Clutch GearControl Paul Pettersson

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 28]

slide-29
SLIDE 29

TERMA A/S (2004)

Memory Management for Radars

Radar Video Processing Subsystem

A d v a n c e d N

  • i

s e R e d u c t i

  • n

T e c h n i q u e s

A i r p

  • r

t S u r v e i l l a n c e C

  • s

t a l S u r v e i l l a n c e

F r e q u e n c y D i v e r s i t y

combiner

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 29]

e1,2 e0,5 e0,4 e0,3 e0,2 e2,4 e2,3 e2,2 e1,5 e1,4 e1,3 e3,2 e3,4 e3,3 e3,5 e2,5

echo 9.170 GHz 9.438 GHz

Combiner (VP3)

slide-30
SLIDE 30

FM Forum -- Model Checking in Action -- Kim G Larsen Kim Larsen [ 30]

RESULTS

  • Verification that existing round-robin

scheduler with the given size of buffers is safe (no underflow, no overflow)

  • Synthesis of scheduler for drastically

minimized buffer sizes.

  • Savings not important enough for methodology to

be taken up by Terma!

Adder 1

S = A + S' - A'

Adder 2

T = B + T' - B'

Buffer 1

1 Kbytes

Buffer 2

1 Kbytes

Buffer 9

2 Kbytes

Buffer 8

2 Kbytes

Buffer 7

2 Kbytes

Buffer 6

512 bytes

Buffer 4

2 Kbytes

Buffer 3

512 bytes

Input A 8 (100MHz) A' S' 16 (100 MHz) Input B 8 (100 MHz) T' B' T S Output S Output T 256 (100 MHz) 128 (200 MHz) SDRAM Buffer 5

512 bytes

B 8 (100MHz) 8 (100MHz) 8 (100MHz) 16 (100 MHz) 16 (100 MHz) 16 (100 MHz) 256 (100 MHz) 256 (100 MHz) 256 (100 MHz) 256 (100 MHz) 256 (100 MHz) 256 (100 MHz) 256 (100 MHz) 256 (100 MHz)

Arbiter

slide-31
SLIDE 31

TERMA A/S (2011)

Herschel-Planck Scientifjc Mission at ESA

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 31]

Attitude and Orbit Control Software TERMA A/S Steen Ulrik Palm, Jan Storbank Pedersen, Poul Hougaard

slide-32
SLIDE 32

TERMA A/S (2011)

Herschel-Planck Scientifjc Mission at ESA

  • Application software (ASW)
  • built and tested by T

erma:

  • does attitude and orbit control, tele-

commanding, fault detection isolation and recovery.

  • Basic software (BSW)
  • low level communication and scheduling

periodic events.

  • Real-time operating system (RTEMS)
  • Priority Ceiling for ASW,
  • Priority Inheritance for BSW
  • Hardware
  • single processor, a few communication

buses, sensors and actuators.

Kim Larsen [ 32]

FM Forum -- Model Checking in Action -- Kim G Larsen

Requirements:

Software tasks should be schedulable. CPU utilization should not exceed 50% load

slide-33
SLIDE 33

TERMA A/S (2011)

Herschel-Planck Scientifjc Mission at ESA

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 33]

UPPAAL 4.1 Framework ISoLA 2010

slide-34
SLIDE 34

TERMA A/S (2011)

Herschel-Planck Scientifjc Mission at ESA

FM Forum -- Model Checking in Action -- Kim G Larsen

Page 34

Marius Micusionis

slide-35
SLIDE 35

TERMA A/S (2011)

Herschel-Planck Scientifjc Mission at ESA

CONCLUSION

  • Schedulability framework made available in

UPPAAL

  • Provides more exact analysis than classical

methods

  • Depending on WCET information the task set is

schedulable or not.

  • Performance:
  • 1-2 minutes: BCET=WCET or BCET/WCET < 0.5
  • 1 day: 0.5 < BCET/WCET < 0.8
  • Work on domain specifjc notation in order to

be fully taken up by company.

FM Forum -- Model Checking in Action -- Kim G Larsen

Page 35

slide-36
SLIDE 36

& NASA

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 36]

Tilt-Tray Sorters Scheduling rules & Optimization, 2008

Philips: Indoor Lighting systems, 2014 Océ Datapath, 2012

ASML, 2004: Wafer Scanners Optimization of Throughput

slide-37
SLIDE 37

UPPAAL Outside Europe

Editor Simulator

Kim Larsen [ 37]

FM Forum -- Model Checking in Action -- Kim G Larsen

Verifier Performance Analyses

Discrete Control Concurrency Continuous Aspects Stochasticity Timing Constraints Resources

slide-38
SLIDE 38

Testing

TRON & YGGDRASIL

slide-39
SLIDE 39

Online vs. Offmine

T este r IU T

A d a p t e r A d a p t e r

input

  • utput

“input” “output”

T est gen

IU T

A d a p t e r A d a p t e r

input

  • utput

“input” “output”

mode l mode l verdi ct verdi ct T est suite T est suite mode l mode l

Executio n and evaluatio n

verdi ct verdi ct nline testing: Pros: Abstract system-level behavior Realistic setup, many components Adaptive, explores only relevant states Allows concurrency, non-determinism Long and intricate interactions Automatic check against model Cons: Does not guarantee coverage Interpreting model can be slow Can be diffjcult to replicate Does not replace offmine testing Offmine testing: Real-time systems are inherently non-determinis Non-determinism yields exponentially large tes Few or no concurrent components Short and specifjc interactions Evaluation requires careful assertion programm

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 39]

slide-40
SLIDE 40

Model Interpretation

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 40]

slide-41
SLIDE 41

Yggdrasil (offmine)

MBAT Daimler Case (2014)

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 41]

G O A L : C

  • v

e r a l l a s p e c s ( l

  • c

a t i

  • n

s , t r a n s i t i

  • n

s , … )

slide-42
SLIDE 42

Test Code & Output

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 42]

slide-43
SLIDE 43

Yggdrasil Industrial Use

  • Novo Nordisk
  • Reduction in time for

testing a module 30 days 30 days  2 days

  • Skov A/S
  • TK Validate
  • Ambitios business plan
  • Evaluation at
  • Daimler
  • Infjnion Austria
  • EADS
  • Bombardier
  • Cov. Inc 40%
  • Reduced test time

20% (80% for unit test)

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 43]

slide-44
SLIDE 44

TRON (online)

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 44]

setLevel(L)

Uppaal TRON

A d a p t e r

handleGrasp() “grasp” “level(L)” “release” handleRelease() verdi ct verdi ct grasp level(L) release TCP/IP socket streams Java method calls

slide-45
SLIDE 45

TRON GUI

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 45]

slide-46
SLIDE 46

Danfoss (2008)

Cooling Control for Industrial Fridges

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 46]

  • Sequanto SeqZap test harness
  • Programmable Logic

Controllers (PLC)

Outcome 4 instances of discrepancy between model and actual behavior, also involving timing errors. Danfoss has adopted MBT in development

  • f new more complex controller!
slide-47
SLIDE 47

Advantages of MBT

  • Engineer focus on what to test at a high

level of abstraction

  • Avoids cost of making scripts
  • As much test code as production code
  • Maintenance nightmare
  • Heard of, but is still considered an

advanced technique by industry

  • Industry is very motivated, MB A&T will

give

  • 10% cost reduction
  • 20% quality improvement

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim G. Larsen [47]

slide-48
SLIDE 48

Model Checking & Testing

Testing

  • Checks the actual

implementation

  • Only few

executions checked

  • But is the most

direct method Model Checking

  • Abstract models
  • Exhaustive “proof”
  • Many mature tools
  • Early detection of

errors

  • State space expl

How to effectively combine the different model checking and testing techniques?

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim G. Larsen [48]

slide-49
SLIDE 49

www.uppaal.{org,com}

FM Forum -- Model Checking in Action -- Kim G Larsen

Kim Larsen [ 49]