Specification and Verification for distributed applications running - - PowerPoint PPT Presentation

▶
specification and verification for distributed
SMART_READER_LITE
LIVE PREVIEW

Specification and Verification for distributed applications running - - PowerPoint PPT Presentation

Feb 11, 2023 186 likes •686 views

Specification and Verification for distributed applications running on heterogeneous infrastructures E. Madelaine Oasis team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis ECNU Dept of Software Engineering October 2010 Eric

slide-1
SLIDE 1

Eric MADELAINE ---- OASIS 1

  • E. Madelaine

Oasis team

INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis ECNU Dept of Software Engineering October 2010

Specification and Verification for distributed applications running on heterogeneous infrastructures

slide-2
SLIDE 2

Eric MADELAINE ---- OASIS 2

Motivations…

Specification and Verification for distributed applications Distributed Systems Components Services Adaptation Message passing Clouds Grids Self Healing Reconfiguration Self Optimizing Load Balancing Interfaces MultiCores

slide-3
SLIDE 3

Eric MADELAINE ---- OASIS 3

Motivations… (2)

Services Adaptation Message passing Clouds Grids Self Healing Reconfiguration Self Optimizing Load Balancing Components Interfaces MultiCores Distributed Systems

  • Hard to program ?
  • Correct Behaviour ?
  • Correct Assembly ?
slide-4
SLIDE 4

Eric MADELAINE ---- OASIS 4

Do we need formal methods for developing component-based software ?

Safe COTS-based development => Behaviour Specifications

A B C ???

Safe management for complex systems (e.g. replacement at runtime)

C2 A B C

slide-5
SLIDE 5

Eric MADELAINE ---- OASIS 5

Is it more difficult for distributed, asynchronous, evolving components ?

Yes ! Asynchrony creates race-conditions, dead-locks, etc. Dynamicity creates new challenges: Correct (DYNAMIC) Adaptation, quiescent states, …

slide-6
SLIDE 6

Eric MADELAINE ---- OASIS 6

Motivations… (3)

Heterogeneous Resources for Distributed Applications

Windows CCS Cluster Storage Server GPUs Cloud Cluster 47+ linux nodes 2-proc/4-core PacaGrid cluster P2P LAN network Mobile terminals

slide-7
SLIDE 7

Eric MADELAINE ---- OASIS 7

Heterogeneous Resources for Distributed Applications

Windows CCS Cluster Storage Server GPUs Cloud Cluster 47+ linux nodes 2-proc/4-core PacaGrid cluster P2P LAN network Mobile terminals

  • Heterogeneity: OS/processor/architecture/communications
  • High performance / Dynamicity / Mobility / Safety / Security…

= ProActive = seamless programming, deployment, scheduling, and execution middelware, strong semantic guaranties The Challenge:

  • Provide optimisations that depends on the underlying platform, and

may evolve dynamically, while keeping simplicity and correctness

slide-8
SLIDE 8

Eric MADELAINE ---- OASIS 8

  • Motivation
  • Building safe applications
  • Heterogeneous Infrastructures: Multi-Cores, P2P, Grids, Clouds
  • Context
  • Oasis team, ,MCorePhP collaborative project
  • Active Objects, Distributed Components
  • Safe Distributed Components
  • Behavioural Semantics
  • Model generation
  • Checking Properties
  • Specification and Verification Tools, Case Study
  • VerCors platform
  • Case-Study
  • Conclusion & Perspectives

Agenda

slide-9
SLIDE 9

Eric MADELAINE ---- OASIS 9

Challenges :

Guarantee safety and security for software systems Develop and master future infrastructures of networks and services

ASP Calculus Properties Model- Checking ProActive Components

The OASIS team:

Propose fundamental principles, techniques and tools for design, development, analysis, and verification of reliable distributed systems

P L A T F O R M S

slide-10
SLIDE 10

Eric MADELAINE ---- OASIS 10

ANR (french ministry of research) International “Blanc” project.

  • Partners:

University of Tsinghua, Beijing (Pr. Yongwei Wu) INRIA Oasis team (Pr. Denis Caromel, Dr. Eric Madelaine)

  • Research Tasks:

Programming Model for Multi-Core Infrastructure with ChinaGrid and CGSP Application and User Case in Bioinformatics

MCorePhP:

A collaborative project building the basis for safe programming of heterogeneous applications

slide-11
SLIDE 11

Eric MADELAINE ---- OASIS 11

Task 1: Programming Model for Multi-Core

1.1 New Basic Programming Model for Multi-Core Extensions of the Active Object programming model:

  • Sharing memory (efficiently) between activities
  • Multi-active (multi-threaded) activities

1.2 Legacy Support and Integration 1.3 Safe Code Generation:

From model-level specification and analysis of properties, to “correct by construction” executable code. This presentation

1.4 Monitoring

MCorePhP:

slide-12
SLIDE 12

Eric MADELAINE ---- OASIS 12

Asynchronous and Deterministic Objects

[Denis Caromel – Ludovic Henrio] ASP (Asynchronous Sequential Processes) =

  • Distributed Active Objects
  • Asynchronous method calls
  • Futures and Wait-by-necessity

 Determinism/Confluence properties  Programming abstractions  Formal Basis for Verification

slide-13
SLIDE 13

Eric MADELAINE ---- OASIS 13

Active Objects (very short…)

  • Runnable (mono-threaded) objects
  • Communicating by remote method call
  • Asynchronous computation
  • Request queues (user-definable policy)
  • No shared memory
  • Futures

Client obj. A Server obj. B

slide-14
SLIDE 14

Eric MADELAINE ---- OASIS 14

ProActive Parallel Suite

Public domain library Object Web Consortium Spin-off company 2007 :

slide-15
SLIDE 15

Eric MADELAINE ---- OASIS 15

Fractal hierarchical model :

Attribute Controller Binding Controller Lifecycle Controller Content Controller

Content Controller / membrane

composites encapsulate primitives, which encapsulates code

  • Provided/Required

Interfaces

  • Hierarchy
  • Separation of

concern: functional / non-functional

  • ADL
  • Extensible
slide-16
SLIDE 16

Eric MADELAINE ---- OASIS 16

Grid Component Model (GCM):

Grid aware extension to Fractal Targetting GRIDS requires to handle:

  • Scalability => hierarchy, parallelism
  • Volatility, heterogeneity => adaptation, dynamicity, autonomicity…

Collective interfaces

  • Multicast, gathercast, gather-multicast, MxN parallel communications

Opportunity to use GCM for parallel computing

Component non-functional concerns (membrane) as a Fractal system

  • Controllers as objects or Fractal/GCM components
  • Fractal extension for properly exposing the non-functional part, including non-

functional client interfaces

  • Non-functional ADL and associated APIs

Opportunity to use GCM for autonomic computing

slide-17
SLIDE 17

Eric MADELAINE ---- OASIS

GCM

Scopes and Objectives: Grid Codes that Compose and Deploy No programming, No Scripting, … No Pain

MultiCast MultiCast GatherCast GatherCast

slide-18
SLIDE 18

Eric MADELAINE ---- OASIS 18

Opportunity to use GCM for autonomic computing

Dynamic to Autonomic component based system reconfiguration

  • Architecture of GCM

membranes

  • How to plug autonomous

strategies to drive all non- functional concerns

EU BIONETS IP project, P. Naoumenko BDO PACA ½ funded PhD:

  • A GCM-based framework for autonomic and evolvable service compositions along bio-inspired

strategies

  • Distributed and reconfigurable service compositions
slide-19
SLIDE 19

Eric MADELAINE ---- OASIS 19

  • Motivation
  • Building safe applications
  • Heterogeneous Infrastructures: Multi-Cores, P2P, Grids, Clouds
  • Context
  • MCorePhP collaborative project
  • Active Objects, Distributed Components
  • Safe Distributed Components
  • Behavioural Semantics
  • Model generation
  • Checking Properties
  • Specification and Verification Tools, Case Study
  • VerCors platform
  • Case-Study
  • Conclusion & Perspectives

Agenda

slide-20
SLIDE 20

Eric MADELAINE ---- OASIS 20

Applications :

  • Check behavioural compatibility between sub-components
  • Check correctness of component deployment
  • Check correctness of the transformation inside a running application.

Behaviour specification and Safe composition

Aim : Build reliable components from the composition of smaller pieces, using their formal specification. Component paradigm : only observe activity at interfaces. Behavioural properties: Deadlock freeness, progress/termination, safety and liveness.

slide-21
SLIDE 21

Eric MADELAINE ---- OASIS 21

pNets : Hierarchical and Parameterized LTSs

[Arnold, Nivat 92] Synchronization networks [Lin 92] symbolic graphs with assignments [Lakas 96] semantics of Lotos open expressions

  • Value-passing, Dynamic architectures, etc.
  • But close to code structure
  • Instantiation to finite structures (through abstract interpretation)

[Forte’04: T. Barros, R. Boulifa, E. Madelaine] [Annals of Telecomunications’08: T. Barros, A. Cansado, L. Henrio, E. Madelaine]

slide-22
SLIDE 22

Eric MADELAINE ---- OASIS 22 Eric MADELAINE 22

pNets : generalized parallel operator

PhiloNET : < Philo[k], Fork[k] > k ∈ [1:n]

Ag = { Think(k), TakeL(k), … } with synchronisation vectors : <Think(k), Think Philo[k] > <TakeL(k), TakeL Philo[k] , Take Fork[k-1] >

Philo[k] Fork[k]

Take Drop TakeL TakeR DropR DropL Think Eat

PhiloNET

slide-23
SLIDE 23

Eric MADELAINE ---- OASIS 23

Building Behavioural Models : Principles

For a given language/framework, define an operational semantics that builds pNets from the program structure.

For Fractal or GCM components:

  • Reason separately at each composition level

Primitive components : functional behaviour is known

  • Given by the user (specification language)
  • Obtained by static analysis (primitive components, e.g. ProActive active objects)

Composites :

  • Computed from lower level
  • Structure and NF behaviour automatically added from the component’s ADL
slide-24
SLIDE 24

Eric MADELAINE ---- OASIS 24

  • !" efine an abstract representation from a finite partition of the

value domains, on a per-formula basis ⇒ Preservation of safety and liveness properties [Cleaveland & Riely 93] " # $ %&"'# ($ " ) &*+ ' ) *#+ (,

Abstractions and Correctness

Interesting research subject here…

slide-25
SLIDE 25

Eric MADELAINE ---- OASIS 25

  • Various temporal logics (CTL, ACTL, …)
  • Regular -calculus (Mateescu’2004) : the assembly language of temporal logics
  • Specification patterns (Dwyer’199x)

Verification of Properties

  • Deadlock Freeness
  • No Error during deployment :

[ (not √ ) * . OE ] false

slide-26
SLIDE 26

Eric MADELAINE ---- OASIS 26

Functional properties under reconfiguration (respecting the topology)

  • Future update (asynchronous result messages) independent of

life-cycle or binding reconfigurations

  • Build a model including life-cycle controllers,

with the reconfiguration actions visible:

  • Then we can prove:

Verification of Properties

[ true*.Req_Get() ] µX. (< true > true ∧ [¬Resp_Get() ] X )

  • Adapt model generation to the properties you want to prove
slide-27
SLIDE 27

Eric MADELAINE ---- OASIS 27

  • Motivation
  • Building safe applications
  • Heterogeneous Infrastructures: Multi-Cores, P2P, Grids, Clouds
  • Context
  • MCorePhP collaborative project
  • Active Objects, Distributed Components
  • Safe Distributed Components
  • Behavioural Semantics
  • Model generation
  • Checking Properties
  • Specification and Verification Tools, Case Study
  • VerCors platform
  • Case-Study: dating agreement protocol with group communication
  • Conclusion & Perspectives

Agenda

slide-28
SLIDE 28

Eric MADELAINE ---- OASIS 28

The Vercors Specification and Verification Platform (middle term)

JDC Specification Graphical Editor (Eclipse Plugin) Vercors JDC Formula G C M / ProActi ve Code Generator

ADL/IDL

(final)

Java Skeletons Business code

Runtime pNets/ Fiacre Model Generator

Finite model

Formula Compiler Prover

slide-29
SLIDE 29

Eric MADELAINE ---- OASIS 29

Graphical Specifications : VCE tool

slide-30
SLIDE 30

Eric MADELAINE ---- OASIS 30

Verification Tools

CADP toolset (INRIA Rhône-Alpes, VASY team)

  • Generic Front-end

(Lotos, BCG, Sync-vectors)

  • Model generation: distributed on a cluster

Large RAM space: Up to billions of states On-the-fly, Tau-reduction, Constrained…

  • Verification Evaluator tool:

Deadlock search / Regular -calculus

  • Bisimulation ckecking, minimizing

Less optimized than “classical” US model-checkers (Spin, etc) But scales better

slide-31
SLIDE 31

Eric MADELAINE ---- OASIS 31

Case-study:

dating agreement protocol with group communication

ProActive-based application, with:

  • Active objects (single thread, no shared memory)
  • Asynchronous (bounded) request queues
  • Group communication
  • No component structure

Initiator Participant Participant A B

slide-32
SLIDE 32

Eric MADELAINE ---- OASIS 32 Eric MADELAINE ---- OASIS 32

Generated Model: the full picture

R_suggest(i,val) CO:

... Group(G)

?Suggest N:=0

Proxy_suggest[c]

!getNth(i,Results[i]) ?R_suggest(i,val) [N=G] !get(Results) get_Suggest(c,Results)

Proxy_validate[c1]

!waitN_Validate(c1,G) waitN_Suggest(c,n) getNth_Suggest(c,i,val)

Initiator

!Participant[i].Q_suggest(c,date) ?Proxy_suggest[c].Suggest !Suggest(c,date) !Validate(c1) !Cancel() waitN_Validate(c1,n) Results[i]:=val N++ !waitN(n) !CollateResults(c1) ?T_CollateResults(Ok) !Suggest(c,date)

Ok

!Validate(c1) !Cancel() ?CollateResults(c) Ok:=true i:=0

CollateResults

!T_CollateResults(Ok) ?getNth_Suggest(c,i,x) i:=i+1 [i=G]

Body

Call_suggest(c,date) Call_validate(c1)

Suggest

Queue

push(req,queue) ?Q_Suggest(c,date) ?Q_Validate(c1) ?Q_Cancel

... ...

Validate Cancel

Participant[i]

T_validate() !Call(req) ?Terminate(req) req ?Serve(req)

Body

Serve(req) !Serve(req) req:=pop(queue) ?Q_* !OutOfBounds(err_mess) Call_cancel() T_cancel() T_suggest() !T_suggest() !R_suggest(c,val) ?Call_suggest(c,date) CO: R_Validate(i) BC: Q_Validate(c1) Q_Cancel() BC: BC: Q_Suggest(c,date) !R_Validate(c1)

Results:=[⊥, . . . , ⊥] [Results[i] = ⊥]

i ∈ [1..G]

[n ≤ N] [Ok] [¬ Ok] Ok:=Ok ∪x

i ∈ [1..G] i ∈ [1..G]

This is a small system: 10 pLTS 6 int. parameters 1 array parameter 11 pNets 19 synch vectors, including 3 broadcast and 2 collectors.

slide-33
SLIDE 33

Eric MADELAINE ---- OASIS 33 Eric MADELAINE 33

State generation 1: BRUTE FORCE

  • With no hierarchical minimization: the generation of a stand-alone

group of 3 participants would be impossible

  • It is essential to build sub-systems in the correct context (=

behavioural contract) => e.g. Projector tool of CADP.

  • > 10^11 states

Group of 3 participants 406 " 458 / 1 284 170 K / 1 646 K Full system, queue

  • f length 2

11 " 54 / 1 489 3 163 / 152 081 Initiator 8 " 90 / 376 1 801 / 5 338 Single participant Total time Minimized Brute force

3 participants Data ∈ ∈ ∈ ∈ { d1,d2 } Res ∈ ∈ ∈ ∈Bool 15 visible labels

Machine: Fedora 10, 4Go RAM 2 dual-core proc@2,4Ghz

slide-34
SLIDE 34

Eric MADELAINE ---- OASIS 34 Eric MADELAINE 34

State generation 2: distributed

Principles:

  • State space partitioned on a cluster by a static hash function. No shared

memory.

  • The state space is merged before other tools (minimization, model-

checking) can be applied. Distributed MC is planned in future versions

  • f CADP.

On the fly partial order reduction available:

  • Tau-compression (collapsing tau-chains)
  • Tau-confluence (selecting only representatives of confluent-sets)
slide-35
SLIDE 35

Eric MADELAINE ---- OASIS 35 Eric MADELAINE 35

State generation 2: distributed

  • Distributed state generation has a (fixed) overhead, but allow for very

large RAM configurations. The bottleneck is the merge phase.

  • On-the-fly partial-order reduction techniques may help to save

memory space, at a high price. It may also fail…

(estimated 125 G states) Out of memory during local computation Brute force Tau-confluence Group of 3 participants Brute force Tau-confluence Brute force Tau-compression Tau-confluence

generation

11’32 19h 10’55 13 M / 48 M 392 K / 1 354 K Group of 2 participants (15x8 cores) 6’45 11’48 30’ 170 K / 1 646 K 170 K / 607 K 5 K / 14 K Full system with 3 participants (8x4 cores)

Total time Brute force

slide-36
SLIDE 36

Eric MADELAINE ---- OASIS 36 Eric MADELAINE 36

State generation 3:

hierarchical / compositional

Classical compositional state generation:

Split the application into smaller pieces, minimize each with (branching) bisimulation before combining them.

Distributed verification architecture:

Define the verification activities as a workflow, and use a generic scheduler on the cloud infrastructure. Some of the workflow nodes are multinode (distributed) tasks.

slide-37
SLIDE 37

Eric MADELAINE ---- OASIS 37

Task 1 Book nodes; Prepare nodes; Build GCF Task 2.1 Compile client; Generate state space Task 2.2 Compile server; Generate state space Task 2.3 Merge sources Task 3 Rename Participants Task 4 Build product; Minimization

Config1.gcf Config2.gcf InitiatorOptim.fcr Participant.fcr InitiatorOptim.bcg Participant.bcg Participant$K[i].svl Participant$K[i].bcg SystemMin.bcg Flac + Distributor Flac + Distributor SVL BCG tools System.exp

slide-38
SLIDE 38

Eric MADELAINE ---- OASIS 38 Eric MADELAINE 38

State generation 3: hierarchical

Classical compositional state generation:

Split the application into smaller pieces, minimize each with (branching) bisimulation before combining them.

⇒ The biggest intermediate structure has ~ 3000 states before reduction. ⇒ A group of 3 (reduced) participants would be 90^3 = 800 000 states. Distributed verification architecture:

Define the verification activities as a workflow, and use a generic scheduler on the cloud infrastructure. Some of the workflow nodes are multinode (distributed) tasks.

=> Open questions: build formalism and tool support to specify

  • the structural splitting
  • the mapping to verification tasks.
slide-39
SLIDE 39

Eric MADELAINE ---- OASIS 39

  • Motivation
  • Building safe applications
  • Heterogeneous Infrastructures: Multi-Cores, P2P, Grids, Clouds
  • Context
  • MCorePhP collaborative project
  • Active Objects, Distributed Components
  • Safe Distributed Components
  • Behavioural Semantics
  • Model generation
  • Checking Properties
  • Specification and Verification Tools, Case Study
  • VerCors platform
  • Case-Study
  • Conclusion & Perspectives

Agenda

slide-40
SLIDE 40

Eric MADELAINE ---- OASIS 40

Conclusions

Starting Point: the pNETs model for behavioural semantics

  • Semantic model for hierarchical, parameterized asynchronous systems

Flexible, expressive and compact.

  • We have define the semantics of: active objects, fractal components, asynchronous

components, group communication, component reconfiguration…

  • Tool support for (graphical) architecture specification, bridges to model-checking tool

sets. This presentation summarizes our recent work on extensions for group communication Ongoing work on component reconfiguration…

Papers, Use-cases and Tools, Position Offers at :

http://www-sop.inria.fr/oasis/Vercors

slide-41
SLIDE 41

Eric MADELAINE ---- OASIS 41

Perspectives

Extensions :

  • Parameterized components in the specification language
  • Tool support for abstraction specification
  • Platform, Engines, Languages for Distributed Model-Checking
  • Specialized model-checking engines for decidable classes of problems:

– unbound fifo channels – Counters + presburger arithmetics

Code Generation :

  • From Architecture and Behaviour Diagrams

… to ADL descriptions and GCM/ProActive code skeletons

slide-42
SLIDE 42

Eric MADELAINE ---- OASIS 42

“Safe by construction” Code Generation

Generate code from high-level specification

  • Generate behaviour model
  • Generate GCM/ProActive code:
  • Isolate the code that the developer

should not modify

  • Guaranteed properties (typically

deadlock freedom, reachability)

More precise than static analysis !

Perspectives ► User specification environment: Eclipse + Graphical Editor (Component architecture, FSM-based behaviour)

slide-43
SLIDE 43

Eric MADELAINE ---- OASIS 43

Thank you

谢谢 谢谢 谢谢 谢谢

Papers, Use-cases and Tools, Position Offers at :

http://www-sop.inria.fr/oasis/Vercors

slide-44
SLIDE 44

Eric MADELAINE ---- OASIS

GCM :

Grid Component Model

Objective

Extension of Fractal for programming Grids

Innovations:

Abstract Deployment Multicast and GatherCast Controller (NF) Components

Standardization

By the ETSI TC-GRID

slide-45
SLIDE 45

Eric MADELAINE ---- OASIS 45

Opportunity to use GCM+ProActive for agile enterprise grid computing

Autonomic to agile enterprise and business services

SOA @ IT Level Infrastructure

  • Exploit Open Architecture of GCM membranes + GCM inherent grid

awareness deployment

SOA @ Business Level

slide-46
SLIDE 46

Eric MADELAINE ---- OASIS 46

CPER: IC2D on Blades+P2P (see demo)

slide-47
SLIDE 47

Eric MADELAINE ---- OASIS 47

My Definition : Software modules, composable, reconfigurable, with well-defined interfaces, and well-defined black box behaviour Our interests :

  • 1. Encapsulation

Black boxes, offered and required services

  • 2. Composition

Design of complex systems, hierarchical organization into sub-systems

  • 3. Separation of concerns

Architecture Description Language (ADL), management components

  • 4. Distribution (e.g. Computational Grid)

Interaction at interfaces through asynchronous method calls

Software Components

slide-48
SLIDE 48

Eric MADELAINE ---- OASIS 48

The pNET model

Parameterized Networks of Synchronised Automata

Specification language Source code pNets system

Abstraction Instantiation Verification tools

Constraint: domains in pNets are “simple types”. The data domains in the source language have to be abstracted beforehand.

slide-49
SLIDE 49

Eric MADELAINE ---- OASIS 49

Parameterized LTSs : definition

Given :

A set of parameters V (with domains in first order “simple types”) An many-sorted term algebra ∑V, with a distinguished Action sort

A parameterized LTS is <V, S, s0, L> in which:

  • Each state s ∈ S has free variables fv(s) ⊆ V
  • Labels l ∈ L have the form <eB, α, xj := ej>
  • eB ∈ ∑V,Bool a guard
  • α ∈ ∑V,Action a parameterized action
  • xj := ej an assignment of the state variables

i,x i,y

y=x-1