Dependability Evaluation Robin Bloomfield, Bev Littlewood Centre - - PowerPoint PPT Presentation

▶

Jun 06, 2023 47 likes •122 views

no: 1 Dependability Evaluation Robin Bloomfield, Bev Littlewood Centre for Software Reliability, City University, London Adelard, London December 2001 CSR no: 2 Background ! Dependability problems no longer simply concern computer systems

SLIDE 1

no: 1

CSR Dependability Evaluation

Robin Bloomfield, Bev Littlewood

Centre for Software Reliability, City University, London Adelard, London December 2001

SLIDE 2

no: 2

CSR

Background

! Dependability problems no longer simply concern computer

systems (did they ever?).

Important dependability issues in the future will concern systems that comprise computers, people and organisations.

! Systems may be rapidly evolving, in quite unexpected ways, and

face a range of sophisticated yet unknown threats.

! Addressing the dependability issues will involve not only computer

scientists and the like, but also statisticians, psychologists, sociologists, economists, etc, in truly inter-disciplinary activity.

SLIDE 3

no: 3

CSR

Problems of dependability requirements, evaluation and confidence

We need methods and tools for

the development of dependability requirements, to describe and trade off different attributes (trust, reliability, security, safety).

! quantitative analysis of efficacy of processes and dependability of

products to aid in decision-making at all levels (building systems, procuring systems, using systems).

! to justify high confidence in a system before it has been

extensively used. This is a difficult area because of the complex nature of the systems, their disparate make-up, and the paucity of evidence

SLIDE 4

no: 4

CSR

Some technical issues to be addressed

! Quantitative ‘dependability cases’: beyond computers.

The problems that arise from human and organisational involvement (and the possible advantages that arise from diversity of roles). Understanding the interactions: software, human, organisation. Understanding the trade

ffs between security and other attributes.

! Design for assessment.

How do we design these complex systems so that their dependability can be assessed? Dependability trade-offs. How to balance re-engineering in systems vs organisations. Development of modelling techniques for large, heterogeneous networks.

! Structuring of dependability arguments.

The issue of confidence, ‘strength’ of arguments, diversity in arguments (e.g. ‘independent argument legs’), the use of disparate evidence sources. Marriage of logical and probabilistic approaches.

SLIDE 5

no: 5

CSR

More technical issues to be addressed

The role of (human intellectual) diversity.

In system design and during system operation. Reasoning about dependability. Understanding the psychology.

Relationship between process and product attributes.

What does knowledge of design processes allow us to claim? Process efficacy in delivering dependability; interactions between processes. Special examples and problems – e.g. Open Source, COTS.

Generation of evidence:

Large scale testing and static analysis (e.g. by using the GRID); analysis of large (operating) systems; legacy systems.

Validation of dependability assessment practices and procedures. If

people can’t trust dependability evaluations, they will not do them. Issues of competency and impact of registration or certification of people. Certification.

SLIDE 6

no: 6

CSR

Application areas

COTS/SOUP, Open Source

Ubiquitous, pervasive

systems where computers dominate mediation between the user and their environment; home assistive technologies.

Operational risk in the

finance and other commercial sectors

Risks of a surveillance

society – how do individuals know what is happening? Standards – unglamorous but important impact on market. Generally not validated by experience and unjustified.

There are many ways to consider this: by technology component, application, sector etc. This is a selection:

SLIDE 7

no: 7

CSR

Means

! Inter-disciplinarity.

! Genuinely inter-disciplinary research

programme, for better understanding

f roles of humans and organisations

in these complex systems.

! Empirical studies.

Case studies on realistic systems, and experiments. Cooperation from industry and commerce.

! Evaluation.

! Need to address quality of evaluation

and resources needed to develop proposals.Made worse by inter- disciplinarily and complex nature of dependability

! Focus.

! Research has many

meanings. Is the aim in