Probability Recap David Dalpiaz STAT 430, Fall 2017 1 - - PowerPoint PPT Presentation

Probability Recap

David Dalpiaz STAT 430, Fall 2017

1

Administration

Questions? Comments? Concerns?

2

Probability Rules

Complement Rule: P[Ac] = 1 − P[A] Conditional Probability: P[A | B] = P[A ∩ B] P[B]

3

Probability Rules

Multiplication Rule: P[A ∩ B] = P[B] · P[A | B]· For a number of events E1, E2, . . . En, the multiplication rule can be expanded into the chain rule: P [n

i=1 Ei] = P[E1]·P[E2 | E1]·P[E3 | E1 ∩E2] · · · P

• En | n−1

i=1 Ei

• 4

Bayes’ Theorem

Define a partition of a sample space Ω to be A1, A2, . . . , An such that Ai ∩ Aj = ∅ for all i = j, and

n

• i=1

Ai = Ω. Let A1, A2, . . . , An form a partition of some sample space. Then for an event B we have Bayes’ Theorem: P[Ai|B] = P[Ai]P[B|Ai] P[B] = P[Ai]P[B|Ai]

n

i=1 P[Ai]P[B|Ai] 5

Indepedence

Two events A and B are said to be independent if they satisfy P[A ∩ B] = P[A] · P[B] A collection of events E1, E2, . . . En is said to be independent if P

 

i∈S

Ei

  =

• i∈S

P[Ei] for every subset S of {1, 2, . . . n}. If this is the case, then the chain rule is greatly simplified to: P [n

i=1 Ei] = n

• i=1

P[Ei]

6

Distributions

We often talk about the distribution of a random variable, which can be thought of as: distribution = list of possible values + associated probabilities

7

Discrete Random Variables

The distribution of a discrete random variable X is most often specified by a list of possible values and a probability mass function, p(x). The mass function directly gives probabilities, that is, p(x) = pX(x) = P[X = x].

8

Binomial Distribution

p(x|n, p) =

• n

x

• px(1−p)n−x,

x = 0, 1, . . . , n, n ∈ N, 0 < p < 1.

• The function p(x|n, p) is the mass function.
• It is a function of x, the possible values of the random variable

X.

• It is conditional on the parameters n and p.
• x = 0, 1, . . . , n indicates the sample space.
• n ∈ N and 0 < p < 1 specify the parameter space.

X ∼ bin(n, p).

9

Continuous Random Variables

The distribution of a continuous random variable X is most often specified by a set of possible values and a probability density function, f (x). The probability of the event a < X < b is calculated as P[a < X < b] =

b

a

f (x)dx. Note that densities are not probabilities.

10

Normal Distribution

f (x|µ, σ2) = 1 σ √ 2π · exp

• −1

2

x − µ

σ

2

, ∞ < x < ∞, −∞ < µ < ∞, σ > 0.

• The function f (x|µ, σ2) is the density function.
• It is a function of x, the possible values of the random variable

X.

• It is conditional on the paramters µ and σ2.
• −∞ < x < ∞ indicates the sample space.
• −∞ < µ < ∞ and σ > 0 specify the parameter space.

X ∼ N(µ, σ2)

11

Expectations

For discrete random variables, we define the expectation of the function of a random variable X as follows. E[g(X)]

• x

g(x)p(x) For continuous random variables we have a similar definition. E[g(X)]

• g(x)f (x)dx

For specific functions g, expectations are given names.

12

Mean

The mean of a random variable X is given by µX = mean[X] E[X]. So for a discrete random variable, we would have mean[X] =

• x

x · p(x) For a continuous random variable we would simply replace the sum by an integral.

13

Variance

The variance of a random variable X is given by σ2

X = var[X] E[(X − E[X])2] = E[X 2] − (E[X])2.

The **standard deviation of a random variable X is given by σX = sd[X]

• σ2

X =

• var[X].

The covariance or random variables X and Y is given by cov[X, Y ] E[(X − E[X])(Y − E[Y ])] = E[XY ] − E[X] · E[Y ].

14

Likelihood

Consider n iid random variables X1, X2, . . . Xn. We can then write their likelihood as L(θ | x1, x2, . . . xn) =

n

• i=i

f (xi; θ) where f (xi; θ) is the density (or mass) function of random variable Xi evaluated at xi with parameter θ. Whereas a probability is a function of a possible observed value given a particular parameter value, a likelihood is the opposite. It is a function of a possible parameter value given observed data.

15