Software Reliability and System Reliability Recap 1 Historical - - PowerPoint PPT Presentation

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Software Reliability Modeling

Steven J Zeil

Old Dominion Univ.

Spring 2012

1 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Software Reliability and System Reliability

1

Recap

2

Historical Perspective and Implementation

3

Exponential Failure Time Models Jelinski-Moranda De-eutrophication Model Goel and Okumoto NHPP Model Schneidewind’s Model Musa’s Basic Execution Time Model Hyperexponential Model

4

Other Finite Failure Models

5

Infinite Failure Models Duane’s Model Geometric Model Musa-Okumoto logarithmic Poisson

6

Bayesian Models Littlewood-Verrall Model

7

Pre-Implementation Models

2 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Reliability Function

Denote the probability that the time to failure T is in some interval (t, t + ∆t) as P(t ≤ T ≤ t + ∆t) F(t) = P(0 ≤ T ≤ t) = t f (x)dx The reliability function is the probability of success at time t (i.e., the prob. that the time to failure exceeds t) R(t) = P(T > t) = 1 − F(t) = ∞

t

f (x)dx

3 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Failure Rate

The failure rate is the probability that a failure per unit time

• ccurs in the interval [t, t + ∆t], given that a failure has not
• ccurred before t.

Failure rate ≡ P(t ≤ T < t + ∆t|T > t) ∆t = P(t ≤ T < t + ∆t) (∆t)P(T > t) = F(t + ∆t) − F(t) (∆t)R(t) Failure rate measurable easier to understand than the prob. density function

4

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Hazard Rate

The hazard rate is defined as the limit of the failure rate as the interval ∆t approaches zero. z(t) = lim

∆t→0

F(t + ∆t) − F(t) (∆t)R(t) = f (t) Rt The hazard rate is an instantaneous rate of failure at time t, given that the system survives up to t. z(t)dt represents the probability that a system of age t will fail in the small interval t to t + dt.

5 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Failure Intensity Function

R(t) = exp

• −

t z(x)dx

• r, differentiating

f (t) = z(t) exp

• −

t z(x)dx

• 6

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Relating Reliability to Failure Rate (2)

Alternatively, Let te be time required to execute one test case. t = kte Assume that there is a finite limit for p/te as te becomes vanishingly small λ = lim

te→0

p te This is the failure intensity function.

7 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Reliability and the Failure Intensity

R(t) = lim

te→0(1 − p(te))t/te = exp(−λt)

which is the exponential distribution

8

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Data

Models typically employ failures per time period, or time between failures Conversion between the two is possible, with some caveats.

9 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Faults and Failures

In this chapter, no distinction is made between faults and failures. They are assumed to be 1-to-1. Implies that we fix all faults before they have a chance to cause a 2nd failure. Let M(t) be a random variable denoting the number of failures (faults) experienced by time t. Let µ(t) be the mean value function of M(t).

10 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Model Classification Scheme

Musa & Okumoto classify models according to

1 Time domain: wall clock versus execution time 2 Category: total number of failures over infinite time

What is limt→∞ µ(t)?

finite infinite

3 Type: distribution of the number of failures experienced by

time t

Poisson binomial

4 Class (finite failure models only): Functional form of the

failure intensity as a function of time

5 Family (infinite failure models only): Functional form of the

failure intensity as a function of expected number of failures experienced

11 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Poisson Processes

The Poisson distribution describes the probability of a given number of events occurring in a fixed interval given that events

• ccur at a fixed rate and are independent of the time since the last

event. f (k; λ) = λke−λ k! Divide time range 0 . . . t into a sequence of observation points t0, t1, . . . tn with t0 = 0 and tn = t. Let fi denote the # of failures occurring in ti−1 . . . ti. If the fi are independent Poisson variables, we have a Poisson process. E [fi] = µ(ti) − µ(ti−1)

12

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Reliability of a Poisson Process

Let ∆t be any non-negative value such that ti−1 + ∆t < ti. The the prob that the software will run reliably for another ∆t given that it has not yet failed at ti − 1 is R(∆t|t) = P(fi = 0|t) = exp

• −

t+∆t

t

λ(x)dx

• 13

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Poisson Process and Single Faults

z(∆t|ti−1) = λ(ti−1 + ∆t) µ(t) = αFa(t) where α is the number of faults (as t → ∞) and Fa(t) is the cumulative distribution of the time to failure of a single fault a. λ(t) = αfa(t)

14 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Binomial Processes

Assume N is the fixed, total number of faults in the system Faults are removed immediately upon detection and new faults are never added If Ta is a reandom variable denoting the time to failure of fault a, the Ta’s are i.i.d. The Fa(t), fa(t), and za(t) are identical for all faults in this class. λ(t) = Nfa(t) µ(t) = NFa(t) N in binomial process is similar to α in the Poisson with a single type of fault.

15 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Exponential Failure Time Models

Models in which the failure intensity function λ(t) is exponential.

1

Recap

2

Historical Perspective and Implementation

3

Exponential Failure Time Models Jelinski-Moranda De-eutrophication Model Goel and Okumoto NHPP Model Schneidewind’s Model Musa’s Basic Execution Time Model Hyperexponential Model

4

Other Finite Failure Models

5

Infinite Failure Models Duane’s Model Geometric Model Musa-Okumoto logarithmic Poisson

6

Bayesian Models Littlewood-Verrall Model

7

Pre-Implementation Models

16

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Jelinski-Moranda De-eutrophication Model

Binomial model. Time between failures is exponentially distributed with a parameter proportional to the number of remaining faults. If we start with N faults and have removed i − 1 of them, the mean time between failures is 1/φ(N − (i − 1)).

17 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Assumptions

1 Rate of fault detection is proportional to the # of faults in

the software

2 Rate of fault detection is constant in the interval between

failures

3 Faults are corrected instantly without introducing new faults 4 Testing is representative of operation 5 Every fault has same chance of being encountered within a

severity class

6 Failures are independent.

Assumptions 4-6 are standard assumptions for reliability modeling.

18 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Data Requirements

Elapsed times between failures xi or times of failures ti such that xi = ti − ti−1

19 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Model Form

Xi are independent random variables with exponential distribution with mean 1/φ(N − (i − 1)) = 1/z(Xi|Ti−1) f ((Xi|Ti−1) = z(Xi|Ti−1) exp(−z(Xi|Ti−1)Xi) = φ(N − (N − 1)) exp(−φ(N − (i − 1))Xi) µ(t) = N(1 − exp(−φt)) λ(t) = Nφ exp(−φt)

20

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Estimation and Prediction

ˆ φ and ˆ N are obtained numerically from the Xi. After removing n faults: ˆ MTTF = 1/ˆ φ(ˆ N − n)

21 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Non-Homogeneous Poisson Process Model

Somewhat similar to Jelinski-Moranda, but focus is on number of failures observed in a time interval, rather than times between failures.

22 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Assumptions

In addition to the standard assumptions

1 M(t), the cumulative failures by time t, follows a Poisson

process with mean µ(t) Expected number of failures over ∆t is proportional to the expected number of undetected faults.

2 Finite failure model 3 The number of faults fi in each time interval are independent. 23 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Data Requirements

fault counts fi in each time interval completion time ti of each time interval

24

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Model Form

µ(t) = N(1 − e−bt) λ(t) = Nbe−bt = Nfa(t) Each fi is an indep. Poisson random variable with mean µ(ti) − µ(ti−1. Therefore the joint density of the fi’s is

n

• i=1

(µ(ti) − µ(ti−1))fi exp(µ(ti) − µ(ti−1)) fi!

25 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Estimation and Prediction

ˆ b and ˆ N are obtained numerically from the fi and ti Expected # of faults in (n + 1)st interval is ˆ N

• e−ˆ

btn − e−ˆ btn+1

• ˆ

MTTF = 1/ˆ φ(ˆ N − n) Possible to predict the additional testing time required to achieve a reliability R over an observation time O.

26 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Schneidewind’s Model

Really a “meta-model” that addresses data aging: Model 1: Use all fault counts from n periods Model 2: Use fault counts only from the last s periods Model 3: Add fault counts from the last s periods, preceded by a single count n−s

i=1 fi over a time period tn−s − t0.

27 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Musa’s Basic Execution Time Model

One of the more heavily used Uses execution time de-couples the failure intensity function from properties of individual faults

28

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Assumptions

In addition to the standard assumptions

1 M(t), the cumulative failures by time t, follows a Poisson

process with mean µ(t) = β0 [1 − exp(−β1t)], β0, β1 > 0

2 Finite failure model

lim

t→∞ (β0 [1 − exp(−β1t)]) = β0

3 The execution times between failures are exponentially

distributed (with a constant hazard rate)

4 Resources available for testing are constant over the total

time of observation

29 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Data Requirements

Elapsed times between failures xi or times of failures ti such that xi = ti − ti−1 Other factors support a conversion to/from calendar time

30 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Model Form

µ(t) = β0 [1 − exp(−β1t)] , β0, β1 > 0 λ(t) = β0β1 exp(−β1t) β0 is the limiting number of faults Large β1 give a rapid decrease in λ

31 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Hyperexponential Model

Musa & Goel Different portions of the software fail at different rates (still exponentially distributed) Leads to a sum of exponential growth curves

32

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Assumptions

Assume you have K subsystems. Within each subsystem

1 The rate of fault detection is proportional to the current fault

content

2 The fault detection rate remains constant between failures 3 Faults are corrected perfectly and instantly

And, within the entire program:

1 M(t), the cumulative failures by time t, follows a Poisson

process with mean µ(t) = N K

i=1 pi [1 − exp(−βit)], where

0 < βi < 1, 0 < pi < 1, , K

i=1 pi = 1

33 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Data Requirements

Fault counts fi in each of the testing intervals Completion time ti of each test period

34 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Model Form

µ(t) = N

K

• i=1

pi [1 − exp(−βit)] λ(t) = N

K

• i=1

piβi exp(−βit) β0 is the limiting number of faults Large β1 give a rapid decrease in λ

35 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Model Form

Let N∗

i = Npi. Then obtain parameter estimates in each class.

36

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Other Finite Failure Models

Weibull distribution is used in hardware reliability because it can accommodate increasing as well as decreasing failure rates S-Shaped model uses a gamma distribution to suggest that growth in µ(t) eventually slows rather than continually increasing exponentially

37 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Infinite Failure Models

1

Recap

2

Historical Perspective and Implementation

3

Exponential Failure Time Models Jelinski-Moranda De-eutrophication Model Goel and Okumoto NHPP Model Schneidewind’s Model Musa’s Basic Execution Time Model Hyperexponential Model

4

Other Finite Failure Models

5

Infinite Failure Models Duane’s Model Geometric Model Musa-Okumoto logarithmic Poisson

6

Bayesian Models Littlewood-Verrall Model

7

Pre-Implementation Models

38 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Duane’s Model

Originally a hardware model Cumulative failure rate versus cumulative testing time, plotted

• n log-log scales, appears linear

39 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Assumptions

Standard assumptions plus M(t) follows a Poisson process with µ(t) = αtβ

40

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Data Requirements

Elapsed times between failures xi or times of failures ti such that xi = ti − ti−1

41 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Model Form

µ(t) = αtβ Let T be cumulative observation time: µ(T) T = αT β T = expected number of failures by time T total testing time Hence Y = ln µ(T) T

• = ln(α) + (β − 1) ln(T)

(linear on a log-log scale)

42 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Estimation and Prediction

ˆ β = n n−1

i=1 ln(tn/ti)

ˆ α = n t

ˆ β n

ˆ MTTF = tn/(n ˆ β)

43 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Geometric Model

Moranda Variation of Jelinski-Moranda where the mean function decreases in a geometric rather than arithmetic sequence Early fixes faults have larger impact than later fixes

44

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Assumptions

Standard assumptions plus fault detection rate forms a geometric progression but is constant between detections z(t) = Dφi−1, 0 < φ < 1, ti−i ≤ t < ti There is an infinite number of total faults. Time between failures is exponentially distributed

45 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Data Requirements

Elapsed times between failures xi or times of failures ti such that xi = ti − ti−1

46 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Model Form

E [Xi] = 1 z(ti−1) = 1 Dφi−1

47 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Musa-Okumoto logarithmic Poisson

Expected number of failures over time is a logarithmic function Particularly useful when the operation use will be highly non-uniform

Early fixes have much more impact than later ones

48

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Assumptions

Standard assumptions plus Failure intensity decreases exponentially with expected number of failures λ(t) = λ0 exp(−θµ(t)) θ > 0 is a decay rate parameter. M(t) follows a Poisson process

49 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Data Requirements

Elapsed times between failures xi or times of failures ti such that xi = ti − ti−1

50 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Model Form

µ(t) = ln(λ0θt + 1)/θ lambda(t) = λ0 λ0θt + 1

51 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Bayesian Models

1

Recap

2

Historical Perspective and Implementation

3

Exponential Failure Time Models Jelinski-Moranda De-eutrophication Model Goel and Okumoto NHPP Model Schneidewind’s Model Musa’s Basic Execution Time Model Hyperexponential Model

4

Other Finite Failure Models

5

Infinite Failure Models Duane’s Model Geometric Model Musa-Okumoto logarithmic Poisson

6

Bayesian Models Littlewood-Verrall Model

7

Pre-Implementation Models

52

Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models Bayesian Models Pre-Implementation Models

Littlewood-Verrall Model

Assumptions Xi are independent exponentially distributed with parameters ξi The ξi are indep. random variables with a gamma distribution parameterized by α and φ(i). φ(i) is an increasing function of i. Software is tested in a manner representative of usage The assumption of decreasing failure rates in ample data is explicit here, but implicit in the other models.

53 Recap Historical Perspective and Implementation Exponential Failure Time Models Other Finite Failure Models Infinite Failure Models

Pre-Implementation Models

Rome Laboratory metrics

54