Gov 2000: 3. Multiple Random Variables Matthew Blackwell Fall 2016 - PowerPoint PPT Presentation

Gov 2000: 3. Multiple Random Variables Matthew Blackwell Fall 2016 1 / 57

1. Distributions of Multiple Random Variables 2. Properties of Joint Distributions 3. Conditional Distributions 4. Wrap-up 2 / 57

Where are we? Where are we going? uncertainty about one variable. relationships between variables. 3 / 57 • Distributions of one variable: how to describe and summarize • Today: distributions of multiple variables to describe • Later: use data to learn about probability distributions.

Why multiple random variables? 1. How do we summarize the relationship between two variables, 𝑌 1 , 𝑌 2 , … , 𝑌 𝑜 ? 2. What if we have many observations of the same variable, 𝑌 and 𝑍 ? 4 / 57 Log GDP/pop growth 10 9 8 7 6 1 2 3 4 5 6 7 8 Log Settler Mortality

1/ Distributions of Multiple Random Variables 5 / 57

Joint distributions pairs of observations, (𝑦, 𝑧) are more likely than others. between 𝑌 and 𝑍 country. 6 / 57 5 2 2 0 1 1 y y y -5 0 0 -10 -1 -1 -15 -2 -2 -2 -1 0 1 2 -2 -1 0 1 2 -2 -1 0 1 2 x x x • The joint distribution of two r.v.s, 𝑌 and 𝑍 , describes what ▶ Settler mortality ( 𝑌 ) and GDP per capita ( 𝑍 ) for the same • Shape of the joint distribution now includes the relationship

Discrete r.v.s Joint probability mass function The joint p.m.f. of a pair of discrete r.v.s, (𝑌, 𝑍) describes the probability of any pair of values: 𝑔 𝑌,𝑍 (𝑦, 𝑧) = ℙ(𝑌 = 𝑦, 𝑍 = 𝑧) 7 / 57 • Properties of a joint p.m.f.: ▶ 𝑔 𝑌,𝑍 (𝑦, 𝑧) ≥ 0 (probabilities can’t be negative) ▶ ∑ 𝑦 ∑ 𝑧 𝑔 𝑌,𝑍 (𝑦, 𝑧) = 1 (something must happen) ▶ ∑ 𝑦 is shorthand for sum over all possible values of 𝑌

Example: Gay marriage and gender Favor Gay 𝑔 𝑌,𝑍 (1, 1) = ℙ(𝑌 = 1, 𝑍 = 1) = 0.3 marriage? 0.27 0.22 Male 𝑌 = 0 0.21 0.3 Female 𝑌 = 1 𝑍 = 0 𝑍 = 1 Marriage Marriage Oppose Gay 8 / 57 • Joint p.m.f. can be summarized in a cross-tab: ▶ Each cell is the probability of that combination, 𝑔 𝑌,𝑍 (𝑦, 𝑧) • Probability that we randomly select a woman who favors gay

Marginal distributions r.v.s 𝑔 𝑍 (𝑧) = ℙ(𝑍 = 𝑧) = ∑ 𝑦 𝑔 𝑌,𝑍 (𝑦, 𝑧) values of 𝑦 partition the space of 𝑌 9 / 57 • Often need to fjgure out the distribution of just one of the ▶ Called the marginal distribution in this context. • Computing marginals from the joint p.m.f.: • Intuition: sum over the probability that 𝑍 = 𝑧 for all possible ▶ Works because these are mutually exclusive events that

Example: marginals for gay 0.51 𝑔 𝑍 (1) = 𝑔 𝑌,𝑍 (1, 1) + 𝑔 𝑌,𝑍 (0, 1) = 0.3 + 0.22 = 0.52 that a woman favors gay marriage. 0.48 0.52 Marginal 0.49 0.27 0.22 marriage Male 𝑌 = 0 0.21 0.3 Female 𝑌 = 1 𝑍 = 0 𝑍 = 1 Marginal Marriage Marriage Oppose Gay Favor Gay 10 / 57 • What’s the 𝑔 𝑍 (1) = ℙ(𝑍 = 1) ? ▶ Probability that a man favors gay marriage plus the probability • Works for all marginals.

Continuous r.v.s 𝑌 𝑍 𝐵 subset of the 2-dimensional plane. 11 / 57 • We will focus on getting the probability of being in some

Continuous joint p.d.f. Continuous joint distribution Two continuous r.v.s 𝑌 and 𝑍 have a continuous joint distribution if there is a nonnegative function 𝑔 𝑌,𝑍 (𝑦, 𝑧) such that for any subset 𝐵 of the 𝑦𝑧 -plane, ℙ((𝑌, 𝑍) ∈ 𝐵) = ∬ 1. 𝑔 𝑌,𝑍 (𝑦, 𝑧) ≥ 0 for all values of (𝑦, 𝑧) , (nonnegative) 2. ∫ ∞ 12 / 57 (𝑦,𝑧)∈𝐵 𝑔 𝑌,𝑍 (𝑦, 𝑧)𝑒𝑦𝑒𝑧. • 𝑔 𝑌,𝑍 (𝑦, 𝑧) is the joint probability density function. • {(𝑦, 𝑧) ∶ 𝑔 𝑌,𝑍 (𝑦, 𝑧) > 0} is called the support of the distribution. • Joint p.d.f. must meet the following conditions: −∞ ∫ ∞ −∞ 𝑔 𝑌,𝑍 (𝑦, 𝑧)𝑒𝑦𝑒𝑧 = 1 , (probabilities “sum” to 1) • ℙ(𝑌 = 𝑦, 𝑍 = 𝑧) = 0 for similar reasons as with single r.v.s.

Joint densities are 3D 𝑔 𝑌,𝑍 (𝑦, 𝑧) . 13 / 57 4 0.15 2 0.10 0 y 0.05 -2 -4 0.00 -4 -2 0 2 4 x • 𝑌 and 𝑍 axes are on the “fmoor,” height is the value of • Remember 𝑔 𝑌,𝑍 (𝑦, 𝑧) ≠ ℙ(𝑌 = 𝑦, 𝑍 = 𝑧) .

Probability = volume 14 / 57 4 0.15 2 0.10 0 y 0.05 -2 -4 0.00 -4 -2 0 2 4 x • ℙ((𝑌, 𝑍) ∈ 𝐵) = ∬ (𝑦,𝑧)∈𝐵 𝑔 𝑌,𝑍 (𝑦, 𝑧)𝑒𝑦𝑒𝑧 • Probability = volume above a specifjc region.

Working with joint p.d.f.s ∞ 2 = (2𝑑𝑧 + 𝑑𝑧 2 )∣ 2 = 𝑑 ∫ 𝑒𝑧 𝑦=0 𝑦=2 2 = 𝑑 ∫ 2 = ∫ 2 ∞ for 0 < 𝑦 < 2 and 0 < 𝑧 < 2 𝑔 𝑌,𝑍 (𝑦, 𝑧) = ⎧ { ⎨ { ⎩ 𝑑(𝑦 + 𝑧) 15 / 57 0 otherwise 1 = ∫ • Suppose we have the following form of a joint p.d.f.: • What does 𝑑 have to be for this to be a valid p.d.f.? −∞ ∫ −∞ 𝑔 𝑌,𝑍 (𝑦, 𝑧)𝑒𝑦𝑒𝑧 0 ∫ 0 𝑑(𝑦 + 𝑧)𝑒𝑦𝑒𝑧 0 (𝑦 2 2 + 𝑦𝑧)∣ 0 (2 + 2𝑧)𝑒𝑧 0 = 8𝑑 • Thus to be a valid p.d.f., 𝑑 = 1/8

Example continuous distribution 𝑔 𝑌,𝑍 (𝑦, 𝑧) = otherwise 0 for 0 < 𝑦 < 2 and 0 < 𝑧 < 2 (𝑦 + 𝑧)/8 ⎩ { ⎨ { ⎧ 16 / 57 2.0 0.5 0.4 1.5 z 0.3 p l u 1.0 y s 0.2 0.5 0.1 0.0 0.0 y 0.0 0.5 1.0 1.5 2.0 x x

Continuous marginal distributions integrating over the distribution of the other variable: 𝑔 𝑍 (𝑧) = ∫ ∞ 𝑔 𝑌 (𝑦) = ∫ ∞ 17 / 57 • We can recover the marginal PDF of one of the variables by −∞ 𝑔 𝑌,𝑍 (𝑦, 𝑧)𝑒𝑦 • Works for either variable: −∞ 𝑔 𝑌,𝑍 (𝑦, 𝑧)𝑒𝑧

Visualizing continuous marginals 𝑔 𝑍 (𝑧) = ∫ ∞ 18 / 57 z y x • Marginal integrates (sums, basically) over other r.v.: −∞ 𝑔 𝑌,𝑍 (𝑦, 𝑧)𝑒𝑦 • Pile up/fmatten all of the joint density onto a single dimension.

Deriving continuous marginals 2 𝑔 𝑍 (𝑧) = (𝑧 + 1)/4 4 𝑧=0 𝑧=2 16)∣ = (𝑦𝑧 8(𝑦 + 𝑧)𝑒𝑧 1 𝑔 𝑌,𝑍 (𝑦, 𝑧) = 0 𝑔 𝑌 (𝑦) = ∫ ⎩ ⎧ { ⎨ { (𝑦 + 𝑧)/8 for 0 < 𝑦 < 2 and 0 < 𝑧 < 2 0 otherwise 19 / 57 • Let’s calculate the marginals for this p.d.f.: 8 + 𝑧 2 = 𝑦 4 + 1 4 = 𝑦 + 1 • By symmetry we have the same for 𝑧 :

Joint c.d.f.s Joint cumulative distribution function For two r.v.s 𝑌 and 𝑍 , the joint cumulative distribution function or joint c.d.f. 𝐺 𝑌,𝑍 (𝑦, 𝑧) is a function such that for fjnite values 𝑦 and 𝑧 , 𝐺 𝑌,𝑍 (𝑦, 𝑧) = ℙ(𝑌 ≤ 𝑦, 𝑍 ≤ 𝑧). 𝜖𝑦𝜖𝑧 20 / 57 • Deriving p.d.f. from c.d.f.: 𝑔 𝑌,𝑍 (𝑦,𝑧) = 𝜖 2 𝐺 𝑌,𝑍 (𝑦,𝑧) • Deriving c.d.f. from p.d.f: 𝐺 𝑌,𝑍 (𝑦, 𝑧) = ∫ 𝑧 −∞ ∫ 𝑦 −∞ 𝑔 𝑌,𝑍 (𝑠, 𝑡)𝑒𝑠𝑒𝑡

2/ Properties of Joint Distributions 21 / 57

Properties of joint distributions dependence is between the variables. 22 / 57 • Single r.v.: summarized 𝑔 𝑌 (𝑦) with 𝔽[𝑌] and 𝕎[𝑌] • With 2 r.v.s, we can additionally measure how strong the • First: expectations over joint distributions and independence

Expectations over multiple r.v.s 𝔽[𝑍] = ∑ 𝑦𝑧 𝑔 𝑌,𝑍 (𝑦, 𝑧) 𝑧 ∑ 𝑦 𝔽[𝑌𝑍] = ∑ 𝑧 𝑔 𝑌,𝑍 (𝑦, 𝑧) 𝑧 ∑ 𝑦 23 / 57 𝔽[𝑕(𝑌, 𝑍)] = ∫ 𝑕(𝑦, 𝑧) 𝑔 𝑌,𝑍 (𝑦, 𝑧) 𝑧 ∑ 𝑦 𝔽[𝑕(𝑌, 𝑍)] = ∑ • 2-d LOTUS: take expectations over the joint distribution. • With discrete 𝑌 and 𝑍 : • With continuous 𝑌 and 𝑍 : 𝑦 ∫ 𝑧 𝑕(𝑦, 𝑧) 𝑔 𝑌,𝑍 (𝑦, 𝑧)𝑒𝑦𝑒𝑧 • Marginal expectations: • Example: expectation of the product:

Marginal expectations from joint 8(𝑦 + 𝑧)𝑒𝑦𝑒𝑧 6 0 2 4(𝑧 + 1)𝑒𝑧 2 = ∫ 8(𝑦 + 𝑧)𝑒𝑦𝑒𝑧 1 0 2 𝑔 𝑌,𝑍 (𝑦, 𝑧) = 2 = ∫ 24 / 57 { ⎨ 2 𝔽[𝑍] = ∫ otherwise 0 for 0 < 𝑦 < 2 and 0 < 𝑧 < 2 (𝑦 + 𝑧)/8 ⎩ ⎧ 2 { • Marginal expectation of 𝑍 : 0 ∫ 0 𝑧1 0 𝑧 ∫ 0 𝑧1 = ( 𝑧 3 12 + 𝑧 2 8 )∣ = 2 3 + 1 2 = 7 • By symmetry, 𝔽[𝑌] = 𝔽[𝑍] = 7/6

Independence Independence Two r.v.s 𝑍 and 𝑌 are independent (which we write 𝑌 ⟂ ⟂ 𝑍 ) if for all sets 𝐵 and 𝐶 : ℙ(𝑌 ∈ 𝐵, 𝑍 ∈ 𝐶) = ℙ(𝑌 ∈ 𝐵)ℙ(𝑍 ∈ 𝐶). value of 𝑍 . ⟂ 𝑕(𝑍) for any functions ℎ() and 𝑕() (functions of independent r.v.s are independent) 25 / 57 • Knowing the value of 𝑌 gives us no information about the • If 𝑌 and 𝑍 are independent, then: ▶ 𝑔 𝑌,𝑍 (𝑦, 𝑧) = 𝑔 𝑌 (𝑦)𝑔 𝑍 (𝑧) (joint is the product of marginals) ▶ 𝐺 𝑌,𝑍 (𝑦, 𝑧) = 𝐺 𝑌 (𝑦)𝐺 𝑍 (𝑧) ▶ ℎ(𝑌) ⟂

Key properties of independent r.v.s = (∑ = 𝔽[𝑌]𝔽[𝑍] ⎠ ⎟ 𝑧 𝑔 𝑍 (𝑧)⎞ 𝑧 ∑ ⎝ ⎜ 𝑦 𝑦𝑧 𝑔 𝑌 (𝑦)𝑔 𝑍 (𝑧) 𝑧 ∑ 𝑦 = ∑ 𝑦𝑧 𝑔 𝑌,𝑍 (𝑦, 𝑧) 𝑧 ∑ 𝑦 𝔽[𝑌𝑍] = ∑ 𝔽[𝑌𝑍] = 𝔽[𝑌]𝔽[𝑍]. 26 / 57 • Theorem If 𝑌 and 𝑍 are independent r.v.s, then • Proof for discrete 𝑌 and 𝑍 : 𝑦 𝑔 𝑌 (𝑦)) ⎛