Laboratorio de Ciberseguridad Probability, Random Processes and - - PowerPoint PPT Presentation

INSTITUTO POLITÉCNICO NACIONAL CENTRO DE INVESTIGACION EN COMPUTACION

Probability, Random Processes and Inference

• Dr. Ponciano Jorge Escamilla Ambrosio

pescamilla@cic.ipn.mx http://www.cic.ipn.mx/~pescamilla/

Laboratorio de Ciberseguridad

CIC

❑ What should I do if I can’t calculate a probability or

expectation exactly?

➢ Simulations using Monte Carlo: “Monte Carlo” just means

that the simulations use random numbers (the term

• riginated from the Monte Carlo Casino in Monaco).

➢ Bounds using inequalities: A bound on a probability gives a

provable guarantee that the probability is in a certain range.

➢ Approximations using limit theorems: Limit theorems let us

make approximations which are likely to work well when we have a large number of data points.

2

Inequalities Inequalities and Limit and Limit Theorems Theorems

CIC

❑ If a random variable X can only take nonnegative

values (X ≥ 0), then:

𝑄 𝑌 ≥ 𝑏 ≤ 𝐹 𝑌 𝑏

for all constant a > 0

❑ The Markov inequality asserts that if a nonnegative

random variable has a small mean, then the probability that it takes a large value must also be small.

3

Markov Markov Inequality Inequality

CIC

4

Markov Markov Inequality Inequality

CIC

❑ If X is a random variable with mean μ and variance

 2. Then for any a > 0:

❑ The Chebyshev inequality asserts that if a random

variable has small variance, then the probability that it takes a value far from its mean is also small.

5

Chebyshev Chebyshev Inequality Inequality

CIC

❑ Proof. By Markov’s inequality, ❑ Substituting c for a, for c > 0, we have the following

equivalent form of Chebyshev’s inequality:

➢ This gives us an upper bound on the probability of an r.v. being more than c

standard deviations away from its mean, e.g., there can’t be more than a 25% chance of being 2 or more standard deviations from the mean.

6

Chebyshev Chebyshev Inequality Inequality

CIC

❑ We started the course by saying that, in the long

term, about half of the flips of a fair coin yield tail.

❑ This is our intuitive understanding of probability. ❑ The law of large number explains that our model of

uncertain events conforms to that property.

❑ The central limit theorem tells us how fast this

convergence happens.

7

Law of Large Numbers Law of Large Numbers

CIC

❑ Let X be a random variable for which the mean,

E[X] =  is unknown. Let X1,…, Xn denote n independent, repeated measurements of X; that is, the Xj’s are independent, identically distributed (i.i.d.) random variables with the same PDF as X. The sample mean of the sequence is used to estimate E[X]:

8

Weak Law Weak Law of Large Numbers

• f Large Numbers

CIC

❑ We have: ❑ Using independence we have:

9

Weak Law Weak Law of Large Numbers

• f Large Numbers

CIC

❑ Applying Chebyshev inequality: ❑ For any fixed 𝜗 > 0, the right-hand side of this

inequality goes to zero as n increases.

10

Weak Law Weak Law of Large Numbers

• f Large Numbers

CIC

11

Weak Law Weak Law of Large Numbers

• f Large Numbers

CIC

The weak law of large numbers asserts that the sample mean of a large number of i.i.d. random variables is very close to the true mean, with high probability.

12

Weak Law Weak Law of Large Numbers

• f Large Numbers

CIC

13

Weak Law Weak Law of Large Numbers

• f Large Numbers

CIC

14

The The Centr Central al Limit Limit Theorem Theorem

CIC

15

The The Centr Central al Limit Limit Theorem Theorem

CIC

16

The The Centr Central al Limit Limit Theorem Theorem

CIC

17

The The Centr Central al Limit Limit Theorem Theorem

❑The PDF of the standardized sum of a large

number of continuous IID random variables will converge to a Gaussian PDF.

❑In many practical situations one can model

a random variable as having arisen from the contributions of many small and similar physical effects.

CIC

18

The The Centr Central al Limit Limit Theorem Theorem

❑Example - PDF for sum of lID U(-1/2 , 1/2)

random variables. Consider the sum:

CIC

19

The The Centr Central al Limit Limit Theorem Theorem

CIC

20

The The Centr Central al Limit Limit Theorem Theorem

CIC

21

The The Centr Central al Limit Limit Theorem Theorem

PDF of sum of N i.i.d. U(0, 1) random variables

CIC

22

The The Centr Central al Limit Limit Theorem Theorem

PDF of sum of N i.i.d. U(0, 1) random variables

CIC

23

The The Str Strong Law of

• ng Law of Large

Large Numbers umbers

CIC

24

The The Str Strong Law of

• ng Law of Large

Large Numbers umbers

CIC

25

The The Str Strong Law of

• ng Law of Large

Large Numbers umbers