Revealing Complexity through Domain-Specific Modelling and Analysis - - PowerPoint PPT Presentation

Revealing Complexity through Domain-Specific Modelling and Analysis

Richard Paige, Phil Brooke, Xiaocheng Ge, Chris Power, Frank Burton, Simon Poulding University of York & Teesside University

[paige, xchge, cpower, frank, smp]@cs.york.ac.uk pjb@scm.tees.ac.uk

2

Context

 Model-Driven Engineering (MDE) exploits models

throughout engineering processes.

– Abstract descriptions of phenomena of interest. – Constructed and manipulated by tools (automation

comes first, not afterwards).

 Models expressed in a variety of (general

purpose, domain-specific) languages

 Models can be analysed, transformed, compared,

merged, validated, …

– Many powerful tools.

Slide 2

3

Motivation

 MDE is predominantly used for engineering

complex systems.

– Typically from system models to code. – Less frequently used for

 understanding systems  explaining systems to different stakeholders  exploring systems to reveal inherent causes of complexity.

 We argue that domain-specific approaches to

MDE can help improve understanding of complex systems.

– Particular for explaining emergent behaviour.

Slide 3

4

Contributions and Structure

 A modelling approach designed to help reveal and

explore complex systems.

– Based on defining and implementing domain-specific

languages

– i.e., designing modelling languages for specific

problems.

 Task-specific analysis techniques for exploring

domain-specific models.

– Implemented with general-purpose MDE tooling.

 Three (work-in-progress) illustrations of using the

approach.

Slide 4

5

Modelling Approach

 General idea: construct a language that

specifically (and only) captures domain-specific concepts/behaviours that are of interest.

 Theory: reduces the semantic gap between the

logic of a domain and its formal descriptions.

 Build task-specific analysis tools using general

purpose tools.

– Particularly transformations, validations (constraints)

and model-to-text generation.

 Use these to perform task-specific analyses, e.g.,

property checking, validation, simulation.

Slide 5

6

Summary of Modelling Approach

• 1. Identify domain concepts and relationships of

interest.

• 2. Encode concepts and relationships in a DSL,

including the abstract and concrete syntax.

–

Tool support is not optional – we use EMF/GMF!

• 3. Encode analyses of interest by transforming

domain-specific models into models amenable to analysis (this is the hard/fun bit)

• 4. After analysis, present results to engineers in a

domain-specific format.

Slide 6

7

Illustrations (Work-in-Progress)

Slide 7

8

Failures in Healthcare Processes

Slide 8

9

Failures in healthcare processes

 Healthcare processes are complex sociotechnical

systems.

 We are interested in consequences of failures

that arise when executing healthcare processes.

– How do they affect completion? – E.g., is information delayed, diagnosis incorrect? – E.g., are there bottlenecks in the process?

 Goal is to provide guidance on refining processes

(or assessing the impact of changes on processes).

Slide 9

10

Modelling

 Created a small DSL for modelling healthcare

processes, tailored for capturing failure modes.

– Inspired by BPDM.

 The real challenge is identifying the failure

modes associated with a process.

– Assertion is that failures in a process are a result of

failures in one or more of the tasks that make up the process.

Slide 10

11

Snapshot of a healthcare process

 Task 15: Patients with suspected stroke should

have specialist assessment within 24 hours of

• nset of symptoms; transfer to acute stroke unit

Slide 11

12

Possible task failure modes

 Completeness: did it run to completion?  Validity: did the outcome meet requirements?  Consistency: are the results consistent across

executions?

 Timeliness: generated on time?  Adequacy: is the outcome fit for purpose?

Slide 12

13

Example: Failure Modes for Task 15

 Incomplete: human resources assigned to the

task are insufficient

– e.g., specialist + nurse required, but no nurse

available.

 Late: specialists carry out task late (e.g., after

24 hours).

 Inadequate: task not performed by stroke

specialist.

Slide 13

14

Modelling and Analysis

 The DSL includes entities for modelling failure

modes of tasks.

 Automated model transformations are used to

produce a simulation model.

– Effectively an interval timed coloured Petri net.

 The simulation model can be used to calculate

the whole-system failure behaviour.

– Similar to FPTC.

Slide 14

15

Example: analysis results

 Consider one (inadequacy) failure: incorrect

judgment after task related to reviewing the investigation results (A19).

– Vulnerable tasks: A5, A9, A10/11, A17. – Failures in these tasks propagated through to failures

• f A19.

– Failure in A19 was not sensitive to failures elsewhere. – These tasks are heavily dependent on skills of the

personnel carrying them out.

– Can tasks for these personnel be streamlined? Siloed?

Slide 15

16

Identification Scenario

Slide 16

17

Identification scenario

• 1. Alice enters a shop.
• 2. He tries to purchase alcohol from Matilda.
• 3. Alice shows an ID card to Matilda to confirm that

he is older than 18.

Slide 17

18

Analysis

 We might want to check properties related to

this scenario.

– if Alice is younger than 18, is his attempt to buy

alcohol refused?

– if Alice is at least 18, does he buy alcohol?

 We might also want to assess the impact of new

ID mechanisms (better cards, biometrics).

 There are interesting issues associated with how

to model such scenarios and their critical events.

19

Identification is interesting

 Matilda may form a belief that Alice is under 18,

thus disrupting the success scenario.

 Matilda may demand to see an ID card.

– The ID card may be damaged and thus may not

represent Alice accurately.

– Matilda may form a belief about the ID card that

supports her initial belief that Alice is under 18.

– Or, she may form a belief about the ID card that is

inconsistent with her initial belief.

 Matilda may know Alice but has reason to not sell

him alcohol.

Slide 19

20

In a nutshell

 Scenarios like this appear conceptually simple:

– Yes/no, boolean outcome. – Events happen or don’t happen. – They appear to follow a tree of boolean decisions. Easy

to explore and simulate, e.g., using a model checker.

 The complexity is hidden.  The outcome may be boolean (Alice buys alcohol

• r doesn’t buy it), but the outcome is derived

from a number of probabilistic decisions.

Slide 20

21

Probabilistic Decisions

 When Alice enters the shop, Matilda forms a

belief about his age.

– Abstractly, the belief is boolean: he is over 18, or not. – In a different model, Matilda’s belief about Alice’s age

is a probability distribution around 18.

– This represents Matilda’s ability to discern Alice’s age

(and, more generally, Matilda’s ability to determine people’s ages).

– This is actually hard, c.f. “Identifying Age, Cohort and

Period Effects in Scientific Research Productivity”, NBER Working Paper No. 11739, 2006.

Slide 21

22

Probabilistic Decisions

 When Alice presents his ID card, Matilda forms a

belief about the representation.

– Abstractly, the belief is boolean: the card accurately

represents Alice, or it does not.

– In a different model, the belief is a probability

distribution, taking into account accuracy of representation as well as effects such as recognition, tampering or damage.

 This belief is dependent on Matilda’s initial belief

about Alice’s age.

Slide 22

23

Ways to model this

 We could model this using probabilistic logic

(e.g., pCSP, prob. refinement calculi).

 “Matilda thinks Alice is over eighteen 70% of the

time, and no more than eighteen 30% of the time”

– How do you come up with the probabilities? – An (under?) approximation; ignores shading. – Ignores dependencies (though conditional probs could

be modelled).

 Model using probability distributions?

Slide 23

24

Modelling

 Constructed a DSL for modelling scenarios.  Key concepts:

– Subject: individual that initiates or triggers a scenario – Agent: a representative (e.g., police officer) of an organisation

that interacts with the subject.

– Organisation: owns artefacts of value – Cheat: a subject who is attempting to convince an agent that they

hold a property that in fact they do not.

– Actor: A generic term covering all the roles above, as well as

• thers not explicitly noted.

– Event: an atomic occurrence; probabilities and probability

distributions can be attached to events.

Slide 24

25

Prototype Tool Support

 We have built a mechanisation via a state

exploration tool.

– Takes models expressed in DSL and explores them. – Models of probability or probability distributions

associated with events

 Handles immediate events (e.g., reaction),

delayed events (e.g., movement), deferred events (internal, invisible events).

 Operates in interactive, exhaustive, or deferred

modes (probabilities not calculated).

Slide 25

26

Example Results

 Returning to our example:

– if Alice is younger than 18, what is the probability that

her attempt to buy alcohol is refused?

– if Alice is at least 18, what is the probability she buys

alcohol?

 Failure of these properties can be viewed as false

negative and false positive outcomes, respectively.

 We want to know the likelihood of false

positives/negatives, as they can be used to assess the impact of mechanisms.

Slide 26

27

Analysing the scenario

 We explored the main identification scenario

with three variants:

– One with no identification mechanisms – One with PASScards – One with PASScards and ID cards (loosely based on UK

ID card scheme)

 Modelled Alice’s age using a trapezoid.  Modelled Matilda’s perception using a normal

distribution around Alice’s actual age.

– Just for proof of concept. Needs validating.

Slide 27

28

Results

 PASScards make a very small improvement

(reducing false pos/neg).

 ID cards make a small additional improvement.  Swamped by the noise.

Slide 28

29

Capability and Acquisition

Slide 29

30

Capability-Based Management

 Some governments are moving away from

– Management of projects in terms of equipment to – Management of projects in terms of capabilities.

 It means moving from defining problems in

terms of concrete solutions to defining problems in terms of abstract needs.

 Why is this useful?

31

Example

 Previously, MoD procurers

might have defined a problem in terms of a need for artillery pieces.

 Defined as a requirement

for a capability of firing at range, we can consider a set of possible solutions.

– E.g., bombers, destroyers.

32

Example

 Each of the solutions (e.g., bomber) proposed

satisfies the same need.

 However, they differ in terms of their own

individual requirements, their cost, and their

• riginal purposes.

 In other words, solutions come with problems.

Modelling can help us understanding problems, solutions, interdependencies, and contexts.

33

It’s easier than you think to make things worse.

34

Why?

 Let’s say I buy a set of long-range missiles to

solve a military problem.

 Purchasing the missiles has a number of side-

effects:

– I have to store them, maintain them, train people to

use them (for when they arrive), purchase support equipment, update doctrines, …

– And I may scare someone else, who then buys their

• wn long-range missiles, …

– … and then I need further capability.

35

Capability is an Acquisition Problem

 Acquisition problems involve identifying,

procuring and managing resources that allow

• rganisations to achieve goals.

– Multiple objectives (save lives, minimise cost) – Heterogeneous tradeoffs (between training and

equipment)

– Different solutions may be optimal or near optimal at

different times.

– Rarely is there a single optimal solution. – Understanding what the problem is may be

challenging.

Slide 35

36

Our approach

 Developed a domain-specific modelling approach

to acquisition problems.

 The approach consists of a family of DSLs for

modelling acquisition problems.

– As well as a set of model transformations for

calculating solutions that are optimal in some way.

 Using DSLs/MDE gives us flexibility and

generality.

– The approach is independent of the acquisition

problem and methods of acquisition.

Slide 36

37

DSLs

 Modelling approach inspired by scenario-based

design and goal modelling (i.e., KAOS).

– DSL for modelling scenarios as a set of high level goals

that can be decomposed.

– DSL for modelling components, which are artefacts

that may satisfy goals.

– DSL for the user that sets configuration parameters. – DSL for correspondences between scenarios and

components.

Slide 37

38

Scenario DSL

Slide 38

39

Component DSL

Slide 39

40

Overall Black Magic

Slide 40

41

Example – Next Release Problem

 This is a standard problem in acquisition used by

many researchers.

– There are a number of customers, each with a number

• f requirements for the next release of a software

system.

– Numerical weightings are given. – Each requirement has an associated cost. – Find the Pareto Front of Customer Satisfaction Vs Cost.

Slide 41

42

Modelling the Problem

Slide 42

43

Component Model

Slide 43

44

Example Solution

Slide 44

45

Advantages of Approach

 Can handle continuous variables in goals.  Generic modelling approach.  Visualisation of solutions.  Dependencies between goals.

Slide 45

46

Conclusions

 Domain-specific modelling and domain-specific

analysis:

– Focus on the essence of the problem. – More easily tailor analysis to specific requirements. – Reveal complexity:

 Relationships between behavioural units in processes.  Relationships between actors in scenarios  Relationships between solutions/problems in acquisition.

– But do the reveal in a controlled way. – Allows us to process complex problems with large

models in efficient ways.

Slide 46

47

Questions and Feedback

? ? ? ?