Developing System Performance Requirements John M. Green Naval - - PowerPoint PPT Presentation

▶
developing system performance
SMART_READER_LITE
LIVE PREVIEW

Developing System Performance Requirements John M. Green Naval - - PowerPoint PPT Presentation

Aug 03, 2023 469 likes •742 views

A Modeling Approach for Developing System Performance Requirements John M. Green Naval Postgraduate School jmgreen@nps.edu Issues to be Addressed The concept of system performance and how to measure effectiveness has been the topic of

slide-1
SLIDE 1

A Modeling Approach for Developing System Performance Requirements

John M. Green Naval Postgraduate School jmgreen@nps.edu

slide-2
SLIDE 2

Issues to be Addressed

  • The concept of system performance and how to measure effectiveness

has been the topic of numerous papers of over the years.

  • Typically the focus is on one system characteristic such as reliability (R) or
  • perational availability (A0) though the Air Force Weapons System

Effectiveness Industry Advisory Committee (WSEIAC) recommended that both are required along with system capability (C).

  • This is in recognition that performance measures are extremely useful to

the system engineer in five key areas:

1. Establishing requirements; 2. Assessing successful mission completion; 3. Isolating problems to gross areas; 4. Ranking problems relative to their potential to impact the mission; and 5. Providing a rational basis for evaluating and selecting between proposed problem solutions and their resulting configurations.

2

slide-3
SLIDE 3

Goal

  • This presentation will present a top-down modeling

approach based on functional flow block diagrams that shows how the system engineer can develop an overall system performance measure that is inclusive of R, A0, and C.

  • It starts with the system concept and allows the system

engineer to allocate performance at each layer of analysis, from system to components, ultimately providing detailed performance requirements which will provide a basis for evaluating candidate solutions.

3

slide-4
SLIDE 4

Prediction

  • Effectiveness calculations are about prediction
  • Objective of prediction is twofold:
  • 1. System effectiveness predictions form a basis for

judging the adequacy of system capabilities

  • 2. Cost-effectiveness predictions form a rational

basis for management decisions.

4

slide-5
SLIDE 5

Outline

  • Three key studies
  • Overview of the approach
  • An example
  • Summary
  • References

5

slide-6
SLIDE 6

Three Key Studies

– WSEIAC (Weapons System Effectiveness Industry Advisory Committee) Study (1964) – MORS C2 Measures of Effectiveness Study (1986) – Paper by John Marshall (1991) – Other support work listed in references

6

slide-7
SLIDE 7

# 1: WSEIAC Study

  • Developed for the Air Force in 1964 and follows

AFSC-375 series

  • Looked at two approaches:
  • 1. Immediately commit resources to an intuitively

plausible (re)design and surmount the problems as they arise, or

  • 2. Explore in the “minds eye” the consequences of the

(proposed) system characteristics in relation to mission objectives before irrevocably committing resources to any specific approach

  • It is a framework for evaluating effectiveness

7

slide-8
SLIDE 8

System Effectiveness

  • Concluded that system effectiveness can be

defined as a measure of the extent to which a system may be expected to achieve a set of specific mission requirements.

  • System effectiveness is a function of three

primary components: availability (A), dependability (D), and capability (C).

  • Definition allows one to determine the

effectiveness of any system type in the hierarchy

  • f systems

8

slide-9
SLIDE 9

Definitions

  • Availability (A) – a measure of the condition of

the system at the start of a mission, when the mission is called for at some random point in time.

  • Dependability (D) – a measure of the system

condition during the performance of the mission given its condition (availability) at the start of the mission.

  • Capability (C) – a measure of the results of the

mission given the condition of the system during the mission (dependability)

9

slide-10
SLIDE 10

Mathematical Formulation

11 12 1 1 2 21 22 2

(0) (0) d d c E a a d d c

1 2

A a a

System state (up/down)

1 2

(0) (0) c C c

Capability at t0 11 12 21 22

d d D d d

d11 = probability of operational at end given operational at start d12 = probability of fail at end given operational at start d21 = probability of operational at end given fail at start d22 = probability of fail at end given fail at start

10

slide-11
SLIDE 11

#2: MORS C2 Study

DECISION MAKER IMPLEMENT RESULTS ANALYSIS RESULTS VALUES OF MEASURES MEASURES FOR FUNCTIONS SYNTHESIS OF STATICS AND DYNAMICS FUNCTIONS SYSTEM ELEMENTS PROBLEM STATEMENT

MODULAR COMMAND AND CONTROL EVALUATION STRUCT URE (MCES)

AGGREGATION OF MEASURES DATA GENERATION EX, EXP, SIM, SUBJECTIVE SPECIFICATION OF MEASURES (CRITERIA) MOP, MOE, MOFE INTEGRATION OF SYSTEM ELEMENTS AND FUNCTIONS C2 PROCESS DEFINITION C2 SYSTEM BOUNDING PROBLEM FORMULATION Dimensional Parameters

System

Subsystem

Environment Force MOPs MOFEs MOEs

1 2 3

, , ,.......,

n

MOP f p p p p

11

slide-12
SLIDE 12

#3: John Marshall Paper

  • Marshall developed a

mathematical relationship between D, A, C, and S based on the work of Ball and Habayeb

  • Related concept to an
  • perational characteristics

curve

– Initial Curve is based solely on the physics involved – Subsequent shape of the curve is defined by variance in system design, operational usage, and environmental conditions

Threat Activity Probability Pta Threat Detect/Control/ Engage Pdce=Pcd*Pcl Threat Attack Pca Ship Susceptability Ph=Pta*Pdce*Pca Ship Vulnerability Pk/h Ship Killability Pk=Ph*Pk/h Ship Survivability Ps=1-Pk

Range Derived Performance Curve R P P Cumulative Probability t1 Available Performance Curve 1.0

12

slide-13
SLIDE 13

A FFBD Example

http://en.wikipedia.org/wiki/Functional_flow_block_diagram

13

slide-14
SLIDE 14

14

Aggregating Processes

Process A Process B Process C Start Outcome

t A B C

P P P P

Process A Process B Start Outcome

t A B A B

P P P P P

Process A Process B Process C Start Outcome

( )

t c A B A B

P P P P P P

slide-15
SLIDE 15

Overview of the Approach

  • 1. Establish the intended purpose of the system
  • 2. Establish those system characteristics which

contribute to the designed ability of the system to accomplishment of the system purpose.

  • 3. Measure/compute the numerical value that

describes the degree to which each of these characteristics affects the accomplishment of the system purpose

  • 4. Combine all computed/measured values into a

form suitable to obtain a system operational value.

15

slide-16
SLIDE 16

A SIMPLIFIED EXAMPLE

16

slide-17
SLIDE 17

From the Ship’s Perspective

Prime Directive: Actively Defend Ship Functions: Detect Control Engage Track TEWA

Search Radar Optical Search Track Radar

Allocated to: Search

Auto Track Auto TEWA Weapons Manual Assign Manual Track OR OR OR

Launch

OR

Functions Process Defend Ship against Cruise Missile Threats

17

slide-18
SLIDE 18

Sensor Operational Objectives

  • Required functions to be performed:

– Detection, Tracking, Classification, ID, Ranging

  • Target characteristics and separation
  • Coverage volume or area and background
  • Atmospheric and weather conditions

18

slide-19
SLIDE 19

19

Pd R Pd R Pd R Rmax Rmax IR Radar IFF MIN MAX MIN

+ + Topen fire

T 1.0 Tmax T 1.0 TFC T 1.0 Tmax Tmin Fire Control C2 Max wait for IFF response

Baseline Network

IFF Raleigh Radar Swerling IR Exponential C2 Uniform FC Fixed Time Delay IFF Wait Fixed Time Delay

slide-20
SLIDE 20

20

System Reaction Time Distribution

Reaction Time Distribution

0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 5 10 15 20 25 Range (NM) Cumulative Probability

Shown without effects of D, A, or S

slide-21
SLIDE 21

Sensor Operational Objectives into Effectiveness

Performance Parameters (C)

  • Detection range
  • Tracking range
  • Classification range
  • ID range
  • Pd
  • SNR minimum
  • Spatial resolution
  • Sensitivity
  • Total FOV (look angle)
  • False alarm time
  • Frame time
  • Physical characteristics

Reliability Parameters

  • Dependability
  • Availability
  • Survivability

21

slide-22
SLIDE 22

Parameters Drive IR Sensor Design

  • Optics
  • Detectors
  • Signal processing
  • Display and recording

22

slide-23
SLIDE 23

In Summary

  • Presented a top-down modeling approach based on functional flow block

diagrams that shows how the system engineer can develop an overall system performance measure that is inclusive of R, A0, and C.

  • It started with the system concept which allows the system engineer to

allocate performance at each layer of analysis, from system to components, ultimately providing detailed performance requirements which will provide a basis for evaluating candidate solutions.

  • Approach can be useful to the system engineer in five key areas:

1. Establishing requirements; 2. Assessing successful mission completion; 3. Isolating problems to gross areas; 4. Ranking problems relative to their potential to impact the mission; and 5. Providing a rational basis for evaluating and selecting between proposed problem solutions and their resulting configurations

23

slide-24
SLIDE 24

24

Functional and Non-functional Performance

slide-25
SLIDE 25

25

slide-26
SLIDE 26

References

  • Ball, Robert. The Fundamentals of Aircraft Combat Survivability

Analysis and Design, AIAA Press, 1985

  • Bard, Jonathon. An Analytical Model of the Reaction Time of a

Naval Platform. IEEE Vol. SMC-11, No. 10 Oct. 1981, pp. 723-726

  • DARCOM-P 706-101 CH. 24
  • Habayeb, A. R. Systems Effectiveness, Pergamon Press, 1987
  • Hitchens, Derek.
  • Friddell, Harold G. and Herbert G. Jacks. System Operational

Effectiveness (Reliability, Performance, Maintainability), 5th National Symposium on Reliability and Quality Control, January 1959

  • Marshall, John. Effectiveness, Suitability & Performance, 59th MORS,

12 June 1991

26