joint distributions independence covariance and
play

Joint Distributions, Independence Covariance and Correlation 18.05 - PowerPoint PPT Presentation

Dec 10, 2022 •179 likes •542 views

Joint Distributions, Independence Covariance and Correlation 18.05 Spring 2014 X \ Y 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 3 1/36 1/36 1/36 1/36 1/36 1/36 4 1/36 1/36 1/36 1/36

  1. Joint Distributions, Independence Covariance and Correlation 18.05 Spring 2014 X \ Y 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 3 1/36 1/36 1/36 1/36 1/36 1/36 4 1/36 1/36 1/36 1/36 1/36 1/36 5 1/36 1/36 1/36 1/36 1/36 1/36 6 1/36 1/36 1/36 1/36 1/36 1/36 January 1, 2017 1 / 36

  2. Joint Distributions X and Y are jointly distributed random variables. Discrete: Probability mass function (pmf): p ( x i , y j ) Continuous: probability density function (pdf): f ( x , y ) Both: cumulative distribution function (cdf): F ( x , y ) = P ( X ≤ x , Y ≤ y ) January 1, 2017 2 / 36

  3. Discrete joint pmf: example 1 Roll two dice: X = # on first die, Y = # on second die X takes values in 1, 2, . . . , 6, Y takes values in 1, 2, . . . , 6 Joint probability table: X \ Y 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 3 1/36 1/36 1/36 1/36 1/36 1/36 4 1/36 1/36 1/36 1/36 1/36 1/36 5 1/36 1/36 1/36 1/36 1/36 1/36 6 1/36 1/36 1/36 1/36 1/36 1/36 pmf: p ( i , j ) = 1 / 36 for any i and j between 1 and 6. January 1, 2017 3 / 36

  4. Discrete joint pmf: example 2 Roll two dice: X = # on first die, T = total on both dice X \ T 2 3 4 5 6 7 8 9 10 11 12 1 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 0 0 2 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 0 3 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 4 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 5 0 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 6 0 0 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 January 1, 2017 4 / 36

  5. Continuous joint distributions X takes values in [ a , b ], Y takes values in [ c , d ] ( X , Y ) takes values in [ a , b ] × [ c , d ]. Joint probability density function (pdf) f ( x , y ) f ( x , y ) dx dy is the probability of being in the small square. y d Prob. = f ( x, y ) dx dy dy dx c x a b January 1, 2017 5 / 36

  6. Properties of the joint pmf and pdf Discrete case: probability mass function (pmf) 1. 0 ≤ p ( x i , y j ) ≤ 1 2. Total probability is 1. n m m m p ( x i , y j ) = 1 i =1 j =1 Continuous case: probability density function (pdf) 1. 0 ≤ f ( x , y ) 2. Total probability is 1. � d � b f ( x , y ) dx dy = 1 c a Note: f ( x , y ) can be greater than 1: it is a density not a probability. January 1, 2017 6 / 36

  7. Example: discrete events Roll two dice: X = # on first die, Y = # on second die. Consider the event: A = ‘ Y − X ≥ 2’ Describe the event A and find its probability. answer: We can describe A as a set of ( X , Y ) pairs: A = { (1 , 3) , (1 , 4) , (1 , 5) , (1 , 6) , (2 , 4) , (2 , 5) , (2 , 6) , (3 , 5) , (3 , 6) , (4 , 6) } . Or we can visualize it by shading the table: X \ Y 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 3 1/36 1/36 1/36 1/36 1/36 1/36 4 1/36 1/36 1/36 1/36 1/36 1/36 5 1/36 1/36 1/36 1/36 1/36 1/36 6 1/36 1/36 1/36 1/36 1/36 1/36 P ( A ) = sum of probabilities in shaded cells = 10/36. January 1, 2017 7 / 36

  8. Example: continuous events Suppose ( X , Y ) takes values in [0 , 1] × [0 , 1]. Uniform density f ( x , y ) = 1. Visualize the event ‘ X > Y ’ and find its probability. answer: y 1 ‘ X > Y ’ x 1 The event takes up half the square. Since the density is uniform this is half the probability. That is, P ( X > Y ) = 0 . 5 January 1, 2017 8 / 36

  9. Cumulative distribution function y x � � � � F ( x , y ) = P ( X ≤ x , Y ≤ y ) = f ( u , v ) du dv . c a ∂ 2 F f ( x , y ) = ( x , y ) . ∂ x ∂ y Properties 1. F ( x , y ) is non-decreasing. That is, as x or y increases F ( x , y ) increases or remains constant. 2. F ( x , y ) = 0 at the lower left of its range. If the lower left is ( −∞ , −∞ ) then this means lim F ( x , y ) = 0 . ( x , y ) → ( −∞ , −∞ ) 3. F ( x , y ) = 1 at the upper right of its range. January 1, 2017 9 / 36

  10. Marginal pmf and pdf Roll two dice: X = # on first die, T = total on both dice. The marginal pmf of X is found by summing the rows. The marginal pmf of T is found by summing the columns X \ T 2 3 4 5 6 7 8 9 10 11 12 p ( x i ) 1 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 0 0 1/6 2 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 0 1/6 3 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 1/6 4 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 1/6 5 0 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 1/6 6 0 0 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 1/6 p ( t j ) 1/36 2/36 3/36 4/36 5/36 6/36 5/36 4/36 3/36 2/36 1/36 1 For continuous distributions the marginal pdf f X ( x ) is found by integrating out the y . Likewise for f Y ( y ). January 1, 2017 10 / 36

  11. Board question Suppose X and Y are random variables and ( X , Y ) takes values in [0 , 1] × [0 , 1]. the pdf is 3( x 2 + y 2 ) . 2 Show f ( x , y ) is a valid pdf. 1 Visualize the event A = ‘ X > 0 . 3 and Y > 0 . 5’. Find its 2 probability. Find the cdf F ( x , y ). 3 Find the marginal pdf f X ( x ). Use this to find P ( X < 0 . 5). 4 Use the cdf F ( x , y ) to find the marginal cdf F X ( x ) and 5 P ( X < 0 . 5). See next slide 6 January 1, 2017 11 / 36

  12. Board question continued 6. (New scenario) From the following table compute F (3 . 5 , 4). X \ Y 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 3 1/36 1/36 1/36 1/36 1/36 1/36 4 1/36 1/36 1/36 1/36 1/36 1/36 5 1/36 1/36 1/36 1/36 1/36 1/36 6 1/36 1/36 1/36 1/36 1/36 1/36 answer: See next slide January 1, 2017 12 / 36

  13. Solution answer: 1. Validity: Clearly f ( x , y ) is positive. Next we must show that total probability = 1: 1 1 1 1 1 � � � � 3 � � 1 3 � � 1 3 2 2 ) dx dy = 3 2 2 dy = 1 . ( x + y x + xy dy = + y 2 2 2 2 2 0 0 0 0 0 2. Here’s the visualization y 1 A . 5 x . 3 1 The pdf is not constant so we must compute an integral 1 1 1 1 � � � � 3 � � 3 1 2 2 ) dy dx = 2 3 P ( A ) = ( x + y x y + y dx 2 2 2 . 3 . 5 . 3 . 5 (continued) January 1, 2017 13 / 36

  14. Solutions 2, 3, 4, 5 1 2 � � 3 x 7 2. (continued) = + dx = 0 . 5495 4 16 . 3 y x 3 3 � � � � � 3( u + v x y xy 2 2 ) du dv = 3. F ( x , y ) = + . 2 2 2 0 0 4. 1 1 3 y 3 3 2 3 1 � � 2 2 ) dy = 2 f X ( x ) = ( x + y x y + = x + 2 2 2 2 2 0 0 . 5 . 5 . 5 3 1 1 1 5 � � � � 2 + 3 + x P ( X < . 5) = f X ( x ) dx = x dx = x = . 2 2 2 2 16 0 0 0 5. To find the marginal cdf F X ( x ) we simply take y to be the top of the F X ( x ) = F ( x , 1) = 1( x 3 + x ) . y -range and evalute F : 2 1 1 1 5 Therefore P ( X < . 5) = F ( . 5) = ( + ) = . 2 8 2 16 6. On next slide January 1, 2017 14 / 36

  15. Solution 6 6. F (3 . 5 , 4) = P ( X ≤ 3 . 5 , Y ≤ 4). X \ Y 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 3 1/36 1/36 1/36 1/36 1/36 1/36 4 1/36 1/36 1/36 1/36 1/36 1/36 5 1/36 1/36 1/36 1/36 1/36 1/36 6 1/36 1/36 1/36 1/36 1/36 1/36 Add the probability in the shaded squares: F (3 . 5 , 4) = 12 / 36 = 1 / 3 . January 1, 2017 15 / 36

  16. Independence Events A and B are independent if P ( A ∩ B ) = P ( A ) P ( B ) . Random variables X and Y are independent if F ( x , y ) = F X ( x ) F Y ( y ) . Discrete random variables X and Y are independent if p ( x i , y j ) = p X ( x i ) p Y ( y j ) . Continuous random variables X and Y are independent if f ( x , y ) = f X ( x ) f Y ( y ) . January 1, 2017 16 / 36

  17. Concept question: independence I Roll two dice: X = value on first, Y = value on second X \ Y 1 2 3 4 5 6 p ( x i ) 1 1/36 1/36 1/36 1/36 1/36 1/36 1/6 2 1/36 1/36 1/36 1/36 1/36 1/36 1/6 3 1/36 1/36 1/36 1/36 1/36 1/36 1/6 4 1/36 1/36 1/36 1/36 1/36 1/36 1/6 5 1/36 1/36 1/36 1/36 1/36 1/36 1/6 6 1/36 1/36 1/36 1/36 1/36 1/36 1/6 p ( y j ) 1/6 1/6 1/6 1/6 1/6 1/6 1 Are X and Y independent? 1. Yes 2. No answer: 1. Yes. Every cell probability is the product of the marginal probabilities. January 1, 2017 17 / 36

  18. Concept question: independence II Roll two dice: X = value on first, T = sum X \ T 2 3 4 5 6 7 8 9 10 11 12 p ( x i ) 1 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 0 0 1/6 2 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 0 1/6 3 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 0 1/6 4 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 0 1/6 5 0 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 0 1/6 6 0 0 0 0 0 1/36 1/36 1/36 1/36 1/36 1/36 1/6 p ( y j ) 1/36 2/36 3/36 4/36 5/36 6/36 5/36 4/36 3/36 2/36 1/36 1 Are X and Y independent? 1. Yes 2. No answer: 2. No. The cells with probability zero are clearly not the product of the marginal probabilities. January 1, 2017 18 / 36

  19. Concept Question Among the following pdf’s which are independent? (Each of the � � � � � ranges is a rectangle chosen so that f ( x , y ) dx dy = 1.) (i) f ( x , y ) = 4 x 2 y 3 . (ii) f ( x , y ) = 1 2 ( x 3 y + xy 3 ). − 3 x − 2 y (iii) f ( x , y ) = 6 e Put a 1 for independent and a 0 for not-independent. (a) 111 (b) 110 (c) 101 (d) 100 (e) 011 (f) 010 (g) 001 (h) 000 answer: (c). Explanation on next slide. January 1, 2017 19 / 36

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Joint Distributions, Independence Covariance and Correlation 18.05 Spring 2014 Jeremy Orloff and

Joint Distributions, Independence Covariance and Correlation 18.05 Spring 2014 Jeremy Orloff and

Joint Distributions, Independence Covariance and Correlation 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom X \ Y 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 3 1/36 1/36 1/36 1/36 1/36

638 views • 45 slides
Joint Distributions, Independence Covariance and Correlation 18.05 Spring 2014 X \ Y 1 2 3 4 5

Joint Distributions, Independence Covariance and Correlation 18.05 Spring 2014 X \ Y 1 2 3 4 5

Joint Distributions, Independence Covariance and Correlation 18.05 Spring 2014 X \ Y 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 3 1/36 1/36 1/36 1/36 1/36 1/36 4 1/36 1/36 1/36 1/36

732 views • 28 slides
Formal Modeling in Cognitive Science 1 Distributions Lecture 20: Joint, Marginal, and Conditional

Formal Modeling in Cognitive Science 1 Distributions Lecture 20: Joint, Marginal, and Conditional

Distributions Distributions Independence Independence Formal Modeling in Cognitive Science 1 Distributions Lecture 20: Joint, Marginal, and Conditional Distributions Joint Distributions Marginal Distributions Conditional Distributions Steve

390 views • 4 slides
High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains Majid

High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains Majid

High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains Majid Janzamin and Anima Anandkumar U.C. Irvine High-Dimensional Covariance Estimation n i.i.d. samples, p variables X := [ X 1 , . . . , X p ] T . Covariance

868 views • 61 slides
High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains Majid

High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains Majid

High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains Majid Janzamin and Anima Anandkumar U.C. Irvine High-Dimensional Covariance Estimation n i.i.d. samples, p variables X := [ X 1 , . . . , X p ] T .

692 views • 53 slides
Formal Modeling in Cognitive Science Lecture 20: Joint, Marginal, and Conditional Distributions

Formal Modeling in Cognitive Science Lecture 20: Joint, Marginal, and Conditional Distributions

Distributions Independence Formal Modeling in Cognitive Science Lecture 20: Joint, Marginal, and Conditional Distributions Steve Renals (notes by Frank Keller) School of Informatics University of Edinburgh s.renals@ed.ac.uk 26 February 2007

157 views • 15 slides
Review: probability Covariance, correlation relationship to independence Law of

Review: probability Covariance, correlation relationship to independence Law of

Review: probability Covariance, correlation relationship to independence Law of iterated expectations Bayes Rule Examples: emacsitis, weighted dice Model learning 1 Review: graphical models Bayes net = DAG + CPT

539 views • 32 slides
Independence, conditional distributions So far density of X specified explicitly. Often modelling

Independence, conditional distributions So far density of X specified explicitly. Often modelling

Independence, conditional distributions So far density of X specified explicitly. Often modelling leads to a specification in terms of marginal and conditional distributions. Defn : Events A and B are independent if P ( AB ) = P ( A ) P ( B ) .

197 views • 5 slides
Why Student Distributions? A Combination . . . Why Materns Covariance Main Result Derivation

Why Student Distributions? A Combination . . . Why Materns Covariance Main Result Derivation

Scale-Invariance: A . . . Heavy-Tailed . . . What Is Usually Done Multi-D Case Why Student Distributions? A Combination . . . Why Materns Covariance Main Result Derivation of Student . . . Model? A Symmetry-Based What Next? Alternative

619 views • 40 slides
Lecture 16 : Independence, Covariance and Correlation of Discrete Random Variables 0/ 31

Lecture 16 : Independence, Covariance and Correlation of Discrete Random Variables 0/ 31

Lecture 16 : Independence, Covariance and Correlation of Discrete Random Variables 0/ 31 Definition Two discrete random variables X and Y defined on the same sample space are said to be independent if for nay two numbers x and y the two events (

698 views • 32 slides
r.r.fi use indicator others 0 IT In PrEIj Xn I In e EE In E EE Ide c In Xn Iie E ECTS

r.r.fi use indicator others 0 IT In PrEIj Xn I In e EE In E EE Ide c In Xn Iie E ECTS

variance and covariance July 23,2020 f Applications of of Expectation Linearity Variance 2 Covariance and correlation 3 Last time Distributions p if Xsl Bernoulli CP x PREM ipso p i f pin pj I Xu PHX Bincngp Linearity of

518 views • 13 slides
Bayesian networks 1 Outline Syntax Semantics Parameterized distributions 2 Bayesian

Bayesian networks 1 Outline Syntax Semantics Parameterized distributions 2 Bayesian

Bayesian networks 1 Outline Syntax Semantics Parameterized distributions 2 Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact specification of full joint distributions

426 views • 31 slides
UQ, STAT2201, 2017, Lecture 5 Unit 4 Joint Distributions and Unit 5 Descriptive

UQ, STAT2201, 2017, Lecture 5 Unit 4 Joint Distributions and Unit 5 Descriptive

UQ, STAT2201, 2017, Lecture 5 Unit 4 Joint Distributions and Unit 5 Descriptive Statistics. 1 Unit 4 - Joint Probability Distributions 2 A joint probability distribution two (or more) random variables in the experiment. In case of

646 views • 63 slides
Improved Blind Side-Channel Analysis by Exploitation of Joint Distributions of Leakages Christophe

Improved Blind Side-Channel Analysis by Exploitation of Joint Distributions of Leakages Christophe

Improved Blind Side-Channel Analysis by Exploitation of Joint Distributions of Leakages Christophe Clavier, L eo Reynaud Universit e de Limoges - XLIM Introduction Joint distributions and maximum of likelihood Extension to masked

657 views • 31 slides
Bayesian networks A simple, graphical notation for conditional independence assertions and

Bayesian networks A simple, graphical notation for conditional independence assertions and

Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact specification of full joint distributions Bayesian Networks Syntax: a set of nodes, one per variable a set of nodes one

350 views • 8 slides
Covariance and spectrum Repetition Covariance function: r w ( ) Ew ( t + ) w T ( t )

Covariance and spectrum Repetition Covariance function: r w ( ) Ew ( t + ) w T ( t )

Covariance and spectrum Repetition Covariance function: r w ( ) Ew ( t + ) w T ( t ) r w ( ) = r T w ( ) Spectral density: Discrete-time systems/processes: for &lt; r w (0) = 1 r w (

448 views • 7 slides
Key Points: NAMS 2007, 2008, 2010, 2012 Acronym Full

Key Points: NAMS 2007, 2008, 2010, 2012 Acronym Full

Controversies in Womens Health December 12, 2013 Key Points: NAMS 2007, 2008, 2010, 2012 Acronym Full name PosiDon Statement ET

345 views • 15 slides
(D39MS) UNIT 2: ACCIDENTS Lecture plan Some examples and the real definition of accidents

(D39MS) UNIT 2: ACCIDENTS Lecture plan Some examples and the real definition of accidents

SAFETY MANAGEMENT &amp; SITE ESTABLISHMENT (D39MS) UNIT 2: ACCIDENTS Lecture plan Some examples and the real definition of accidents The causes of accidents Accident statistics The recording and reporting of accidents and

1.04k views • 57 slides
HSPC and CIIC September 13, 2018 1 Agenda Why semantic interoperability? What would the

HSPC and CIIC September 13, 2018 1 Agenda Why semantic interoperability? What would the

HSPC and CIIC September 13, 2018 1 Agenda Why semantic interoperability? What would the architecture of an interoperable system look like? Background and history of HSPC and CIIC The merger of HSPC and CIIC Current activities

1.05k views • 84 slides
Healthcares New Entrants: Who will be the industrys Amazon.com? June 3, 2014

Healthcares New Entrants: Who will be the industrys Amazon.com? June 3, 2014

Healthcares New Entrants: Who will be the industrys Amazon.com? June 3, 2014 FierceHealthcare FastCast Todays presenters from PwC Vaughn Kauffman Trine Tsouderos Health Industries Global New Director, PwC Health Entrants Leader

319 views • 19 slides
Standardized survival curves and related measures using flexible parametric survival models Paul

Standardized survival curves and related measures using flexible parametric survival models Paul

Standardized survival curves and related measures using flexible parametric survival models Paul C Lambert 1,2 1 Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden 2 Department of Health Sciences,

667 views • 44 slides
FICTION AND FACTS IN BREAST CANCER SCREENING Daniel B. Kopans, M.D. Professor of Radiology

FICTION AND FACTS IN BREAST CANCER SCREENING Daniel B. Kopans, M.D. Professor of Radiology

FICTION AND FACTS IN BREAST CANCER SCREENING Daniel B. Kopans, M.D. Professor of Radiology Harvard Medical School Senior Radiologist Breast Imaging Division Massachusetts General Hospital BREAST CANCER SCREENING Mammography screening is

865 views • 60 slides
Risk Factors in African-American Women Michele L. Cote, PhD Associate Professor W Wayne State

Risk Factors in African-American Women Michele L. Cote, PhD Associate Professor W Wayne State

Risk Factors in African-American Women Michele L. Cote, PhD Associate Professor W Wayne State University St t U i it Age Adjusted Incidence and Mortality Rates for all Endometrial Cancers 2000-2010 by Race/Ethnicity for SEER 18

318 views • 17 slides
Pharmaceuticals and Personal Care Products (PPCPs) Webinar Helen Domske, NY Sea Grant Program

Pharmaceuticals and Personal Care Products (PPCPs) Webinar Helen Domske, NY Sea Grant Program

Pharmaceuticals and Personal Care Products (PPCPs) Webinar Helen Domske, NY Sea Grant Program Marti Martz, Pennsylvania Sea Grant Robin Goettel, Terri Hallesy, Laura Kammin Illinois-Indiana Sea Grant Program What is Sea Grant? The National

164 views • 12 slides

More recommend