Statistical Distributions in Simulation R.B. Lenin - - PowerPoint PPT Presentation

Statistical Distributions in Simulation

R.B. Lenin (rblenin@daiict.ac.in) Autumn 2007

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 1 / 68

Outline

1

Random Variables Introduction Discrete random variables Continuous random variable Mean of a random variable Properties of µ Variance of a random variable Properties of σ2 Covariance of random variables Correlation of random variables Properties of σ2 Correlation between random variables Chebyshev’s inequality Strong law of large numbers

2

Discrete Distributions Bernoulli random variable Binomial random variable Geometric random variable Poisson random variable Poisson approximation Poisson process

3

Continuous Distributions Uniform random variable Exponential random variable Erlang random variable Hyper-exponential random variable Gamma random variable Weibull random variable Beta random variable Triangular random variable Normal (Gaussian) random variable Central limit theorem (CLT) Log-normal random variable R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 2 / 68

Random Variables Introduction

Random variables

Discrete

lottery coin tossing number of customers in a supermarket

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 3 / 68

Random Variables Introduction

Random variables

Discrete

lottery coin tossing number of customers in a supermarket

Continuous

interarrival times of vehicles at a traffic light water level in a dam time taken for a chemical reaction

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 3 / 68

Random Variables Discrete random variables

Discrete random variables

A random variable X is said to be discrete if it can take on at most a countable number of values, say, x1, x2, . . . , . The probability that X is equal to xi is given by p(xi) = Pr{X = xi}, i = 1, 2, . . . and

∞

• i=1

p(xi) = 1.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 4 / 68

Random Variables Discrete random variables

Discrete random variables · · ·

p(x) is known as the probability mass function (pmf). ❘

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 5 / 68

Random Variables Discrete random variables

Discrete random variables · · ·

p(x) is known as the probability mass function (pmf). The cumulative distribution function (cdf) F(x) is given by FX(x) = Pr{X ≤ x} =

• xi≤x

p(xi), ∀x ∈ ❘.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 5 / 68

Random Variables Continuous random variable

Continuous random variable

A random variable X is said to be continuous if there exists a nonnegative function f (x), the probability density function, such that for any set of real numbers B, Pr{X ∈ B} =

• B

f (x)dx and ∞

−∞

f (x)dx = 1.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 6 / 68

Random Variables Continuous random variable

Continuous random variable · · ·

If x is a number and ∆x > 0, then Pr {X ∈ [x, x + ∆x]} = x+∆x

x

f (y)dy and Pr{X = x} = 0.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 7 / 68

Random Variables Continuous random variable

Continuous random variable · · ·

The cumulative distribution function FX(x) for a continuous random variable X is given by FX(x) = Pr{X ∈ (−∞, x]} = x

−∞

f (y)dy, ∀x ∈ ❘.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 8 / 68

Random Variables Mean of a random variable

Mean of a random variable

The mean or expected value of a random variable X, denoted by µ

• r E[X], is given by

E[X] =             

∞

• i=1

xip(xi), if X is discrete ∞

−∞

xf (x)dx, if X is continuous

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 9 / 68

Random Variables Properties of µ

Properties of µ

1 E[cX] = cE[X], for any constant c. R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 10 / 68

Random Variables Properties of µ

Properties of µ

1 E[cX] = cE[X], for any constant c. 2 E[X + Y ] = E[X] + E[Y ] regardless of whether X and Y are

independent.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 10 / 68

Random Variables Properties of µ

Properties of µ

1 E[cX] = cE[X], for any constant c. 2 E[X + Y ] = E[X] + E[Y ] regardless of whether X and Y are

independent.

3 E[XY ] = E[X]E[Y ], provided X and Y are independent. R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 10 / 68

Random Variables Properties of µ

Properties of µ

1 E[cX] = cE[X], for any constant c. 2 E[X + Y ] = E[X] + E[Y ] regardless of whether X and Y are

independent.

3 E[XY ] = E[X]E[Y ], provided X and Y are independent.

The mean is one measure of the central tendency of a random variable.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 10 / 68

Random Variables Variance of a random variable

Variance of a random variable

The variance of the random variable X, denoted by σ2 or Var[X], is given by σ2 = E[(X − µ)2] = E[X 2] − µ2

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 11 / 68

Random Variables Variance of a random variable

Variance of a random variable

The variance of the random variable X, denoted by σ2 or Var[X], is given by σ2 = E[(X − µ)2] = E[X 2] − µ2 The variance is a measure of the dispersion of a random variable about its mean.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 11 / 68

Random Variables Properties of σ2

Properties of σ2

1 Var[cX] = c2Var[X] R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 12 / 68

Random Variables Properties of σ2

Properties of σ2

1 Var[cX] = c2Var[X] 2 Var[X + Y ] = Var[X] + Var[Y ], if X and Y are independent. R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 12 / 68

Random Variables Properties of σ2

Properties of σ2

1 Var[cX] = c2Var[X] 2 Var[X + Y ] = Var[X] + Var[Y ], if X and Y are independent.

The square root of the variance is called the standard deviation and is denoted by σ.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 12 / 68

Random Variables Properties of σ2

Properties of σ2

1 Var[cX] = c2Var[X] 2 Var[X + Y ] = Var[X] + Var[Y ], if X and Y are independent.

The square root of the variance is called the standard deviation and is denoted by σ. It can be given the most definitive interpretation when X has a normal distribution.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 12 / 68

Random Variables Covariance of random variables

Covariance of random variables

The covariance between the random variables X and Y , denoted by Cov[X, Y ], is defined by Cov[X, Y ] = E{[X − E[X]][Y − E[Y ]]} = E[XY ] − E[X][Y ]

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 13 / 68

Random Variables Covariance of random variables

Covariance of random variables

The covariance between the random variables X and Y , denoted by Cov[X, Y ], is defined by Cov[X, Y ] = E{[X − E[X]][Y − E[Y ]]} = E[XY ] − E[X][Y ] The covariance is a measure of the dependence between X and Y .

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 13 / 68

Random Variables Covariance of random variables

Covariance of random variables

The covariance between the random variables X and Y , denoted by Cov[X, Y ], is defined by Cov[X, Y ] = E{[X − E[X]][Y − E[Y ]]} = E[XY ] − E[X][Y ] The covariance is a measure of the dependence between X and Y . Note that Cov[X, X] = Var[X].

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 13 / 68

Random Variables Correlation of random variables

Correlation of random variables

Cov[X, Y ] X and Y = 0 are uncorrelated > 0 are positively correlated < 0 are negatively correlated Independent random variables are uncorrelated

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 14 / 68

Random Variables Properties of σ2

Properties of σ2

Var[X − Y ] = Var[X] + Var[Y ] − 2Cov[X, Y ]

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 15 / 68

Random Variables Properties of σ2

Properties of σ2

Var[X − Y ] = Var[X] + Var[Y ] − 2Cov[X, Y ] If X and Y are independent, then Var[X − Y ] = Var[X] + Var[Y ]

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 15 / 68

Random Variables Correlation between random variables

Correlation between random variables

The correlation between the random variables X and Y , denoted by Cor[X, Y ], is defined by Cor[X, Y ] = Cov[X, Y ]

• Var[X]Var[Y ]

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 16 / 68

Random Variables Correlation between random variables

Correlation between random variables

The correlation between the random variables X and Y , denoted by Cor[X, Y ], is defined by Cor[X, Y ] = Cov[X, Y ]

• Var[X]Var[Y ]

It can be shown that −1 ≤ Cor[X, Y ] ≤ 1

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 16 / 68

Random Variables Chebyshev’s inequality

Chebyshev’s inequality

If X is a random variable having mean µ and variance σ2, then for any value k > 0, Pr{|X − µ| ≥ kσ} ≤ 1 k2

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 17 / 68

Random Variables Strong law of large numbers

Strong law of large numbers

Let X1, X2, . . . be a sequence of independent and identically distributed random variables having mean µ. Then, with probability 1, lim

n→∞

X1 + X2 + · · · + Xn n = µ

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 18 / 68

Random Variables Strong law of large numbers

Strong law of large numbers

Let X1, X2, . . . be a sequence of independent and identically distributed random variables having mean µ. Then, with probability 1, lim

n→∞

X1 + X2 + · · · + Xn n = µ That is, with certainty, the long-run average of a sequence of independent and identically distributed random variables will converge to its mean.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 18 / 68

Discrete Distributions

Discrete Distributions

Bernoulli

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 19 / 68

Discrete Distributions

Discrete Distributions

Bernoulli Binomial

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 19 / 68

Discrete Distributions

Discrete Distributions

Bernoulli Binomial Geometric

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 19 / 68

Discrete Distributions

Discrete Distributions

Bernoulli Binomial Geometric Poisson

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 19 / 68

Discrete Distributions

Discrete Distributions

Bernoulli Binomial Geometric Poisson

Poisson Approximation

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 19 / 68

Discrete Distributions

Discrete Distributions

Bernoulli Binomial Geometric Poisson

Poisson Approximation Poisson Process

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 19 / 68

Discrete Distributions Bernoulli random variable

Bernoulli random variable

A random variable X is said to be Bernoulli random variable if its pdf is given by Pr{X = x} = q = 1 − p, if x = 0 p, if x = 1

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 20 / 68

Discrete Distributions Bernoulli random variable

Bernoulli random variable

A random variable X is said to be Bernoulli random variable if its pdf is given by Pr{X = x} = q = 1 − p, if x = 0 p, if x = 1 Parameter(s): p

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 20 / 68

Discrete Distributions Bernoulli random variable

Bernoulli random variable

A random variable X is said to be Bernoulli random variable if its pdf is given by Pr{X = x} = q = 1 − p, if x = 0 p, if x = 1 Parameter(s): p E[X] = p, Var[X] = pq.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 20 / 68

Discrete Distributions Bernoulli random variable

Bernoulli random variable

A random variable X is said to be Bernoulli random variable if its pdf is given by Pr{X = x} = q = 1 − p, if x = 0 p, if x = 1 Parameter(s): p E[X] = p, Var[X] = pq. Example:

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 20 / 68

Discrete Distributions Bernoulli random variable

Bernoulli random variable

A random variable X is said to be Bernoulli random variable if its pdf is given by Pr{X = x} = q = 1 − p, if x = 0 p, if x = 1 Parameter(s): p E[X] = p, Var[X] = pq. Example:

The distribution of heads and tails in coin tossing is a Bernoulli distribution with p = q = 1/2.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 20 / 68

Discrete Distributions Binomial random variable

Binomial random variable

A random variable X is said to be a binomial random variable if its pmf is given by Pr{X = i} = N i

• piqN−i, i = 0, 1, . . . , N.

N i

• =

N! i!(N − i)i

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 21 / 68

Discrete Distributions Binomial random variable

Binomial random variable

A random variable X is said to be a binomial random variable if its pmf is given by Pr{X = i} = N i

• piqN−i, i = 0, 1, . . . , N.

N i

• =

N! i!(N − i)i Parameters: N, p.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 21 / 68

Discrete Distributions Binomial random variable

Binomial random variable

A random variable X is said to be a binomial random variable if its pmf is given by Pr{X = i} = N i

• piqN−i, i = 0, 1, . . . , N.

N i

• =

N! i!(N − i)i Parameters: N, p. Notation: B(N, p).

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 21 / 68

Discrete Distributions Binomial random variable

Binomial random variable · · ·

A binomial random variable with parameters N and p is the sum of N independent Bernoulli random variables with parameter p.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 22 / 68

Discrete Distributions Binomial random variable

Binomial random variable · · ·

A binomial random variable with parameters N and p is the sum of N independent Bernoulli random variables with parameter p. Therefore, the mean and variance of a binomial random variable is given by E[X] = np and Var[X] = np(1 − p)

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 22 / 68

Discrete Distributions Binomial random variable

Example of a binomial random variable

Consider a delivery of 10000 electric bulbs. The delivery is known to contain 1000 faulty bulbs randomly placed among the 9000 good ones. We draw randomly N bulbs (assume N is small in comparison to 10000). We want to know the probability of having X (= 0, 1, . . . , N) faulty bulbs within the random sample of size N. Denote by p the proportion of faulty bulbs (which is 0.1 in our case). The Pr{X = i} (probability of i faulty bulbs out of N) is given by the binomial distribution with parameters N and p = 0.1.

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 23 / 68

Discrete Distributions Binomial random variable

Histogram of a binomial random variable

−1 1 2 3 4 5 6 7 8 9 0.05 0.1 0.15 0.2 0.25 0.3 0.35 x, Values of a Binomial Random Variable Pr{X=x} for p = 0.5 N = 8

Figure: Binomial probabilities for N = 8, p = 0.5

R.B. Lenin (rblenin@daiict.ac.in) () Statistical Distributions in Simulation Autumn 2007 24 / 68