Outline Outline 2 Probability Models of N Random Variables - - PowerPoint PPT Presentation

▶
outline outline
SMART_READER_LITE
LIVE PREVIEW

Outline Outline 2 Probability Models of N Random Variables - - PowerPoint PPT Presentation

May 01, 2023 462 likes •551 views

Outline Outline 2 Probability Models of N Random Variables Probability Models of N Random Variables (5.1, Y&G) Vector Notation (5.2, Y&G) Vector Notation (5 2 Y&G) 204312 PROBABILITY AND 204312 PROBABILITY AND

slide-1
SLIDE 1

204312 PROBABILITY AND 204312 PROBABILITY AND RANDOM PROCESSES FOR COMPUTER ENGINEERS COMPUTER ENGINEERS

Lecture 8: Chapters 5.1-5.4, 5.6 p

1st Semester, 2007 Monchai Sopitkamon, Ph.D.

Outline Outline

Probability Models of N Random Variables

2

Probability Models of N Random Variables

(5.1, Y&G)

Vector Notation (5 2 Y&G) Vector Notation (5.2, Y&G) Marginal Probability Functions (5.3, Y&G) Independence of Random Variables and

Random Vectors (5.4, Y&G) ( , )

Expected Value Vector and Correlation

Matrix (5.6, Y&G) Matrix (5.6, Y&G)

Probability Models of N Random Variables I (5.1)

3

Represent n RVs using vector notation A random vector treats a collection of n RVs as a single

entity

Perform an experiment that produces n RVs, X1, …, Xn

p p ,

1,

,

n

defined with multivariate joint CDF ) ..., , ( ) ..., , (

1 1 1

1

n n n X X

x X x X P x x F ≤ ≤ =

However, joint PMF/PDF provides a better way to

analyze prob models ) ( ) (

1 1 1 , ,

1

n n n X X

n

…

analyze prob. models

Multivariate joint PMF of the discrete RVs X1, …, Xn is:

) , , ( ) ..., , (

1 1 1 , ,

1

n n n X X

x X x X P x x P

n

= = = …

…

Probability Models of N Random Variables II

4

Multivariate joint PDF of the continuous RVs X1, …,

Xn is:

n

x x F ∂ ) (

n n X X n X X

x x x x F x x f

n n

∂ ∂ ∂ = ) , , ( ) ..., , (

1 1 , , 1 ..., ,

1 1

  • …

…

If X1, …, Xn are discrete RVs with joint PMF

) ( P P

1.

≥ 0

) ..., , ( 1

..., ,

1

n X X

x x P P

n

) ..., , ( 1

..., ,

1

n X X

x x P P

n

2.

.

1 n

∑ ∑

∈ ∈

=

n X n n X

S x X X S x

P 1

, ,

1 1 1

…

slide-2
SLIDE 2

Probability Models of N Random Variables III

5

If X1, …, Xn are continuous RVs with joint PDF

) ..., , ( 1

..., ,

1

n X X

x x f

1.

.

) ( 1

..., ,

1

n X X

f

n

) ..., , ( 1

..., ,

1

≥

n X X

x x f

n

∫ ∫

j n

x x

d d f F ) ( ) (

2.

.

∫ ∫

∞ − ∞ −

=

j n n

n n X X n X X

du du u u f x x F ) , , ( ) ..., , (

1 1 , , 1 , ,

1 1

  • …
  • …

…

∫ ∫

∞ ∞

=1 ) ( dx dx x x f

  • 3.

.

  • Prob. of an event A expressed in terms of the RVs

∫ ∫

∞ − ∞ −

=1 ) , , (

1 1 , ,

1

n n X X

dx dx x x f

n

  • …
  • …

X1, …, Xn is

Discrete:

∑

=

A n X X

n

x x P A P

) ( 1 , ,

1

) , , ( ) (

…

…

Continuous:

∈A x x

n )

, , ( 1 …

∫ ∫

=

n n X X A

dx dx dx x x f A P

n

… …

  • …

2 1 1 , ,

) , , ( ) (

1

Vector Notation I (5.2) Vector Notation I (5.2)

6

U b ldf i f l

Use boldface notation x for a column vector. Row vectors are transposed column vectors; x′.

C t f l t itt i l

Components of a column vector are written in a column. To save space, use the transpose of a row vector to

represent a column vector: y = [y1 y ]′ is a column

  • represent a column vector: y

[y1 yn] is a column vector

A random vector is a column vector X = [X1, …, Xn]′

  • 1

n

where each Xi is a RV

A RV is a random vector with n = 1. A sample value of a random vector is a column vector

x = [x1, …, xn]′ where the ith component, xi, of the vector x is a sample value of a RV Xi vector x is a sample value of a RV Xi

Vector Notation II Vector Notation II

7

Random Vector Probability Functions CDF of a random vector X is

) ,..., ( ) (

1 ,...,

1

n X X X

x x F x F

n

=

PMF of a discrete random vector X is

PDF f ti d t X i

) ,..., ( ) (

1 ,...,

1

n X X X

x x P x P

n

=

PDF of a continuous random vector X is

) ,..., ( ) (

1 n X X X

x x f x f = ) , , ( ) (

1 ,...,

1

n X X X

f f

n

Vector Notation III Vector Notation III

8

Probability Functions of a Pair of Random Vectors:

For random vectors X with n components and Y with m components

The joint CDF of X and Y is

) ,..., , ,..., ( ) , (

1 1 ,..., , ,..., ,

1 1

m n Y Y X X Y X

y y x x F y x F

m n

=

The joint PMF of discrete random vectors X and Y is

) ,..., , ,..., ( ) , (

1 1 ,..., , ,..., ,

1 1

m n Y Y X X Y X

y y x x P y x P

m n

=

The joint PDF of continuous random vectors X and Y is

1 1 ,..., , ,..., ,

1 1

m n Y Y X X Y X

m n

) ,..., , ,..., ( ) , (

1 1 ,..., , ,..., ,

1 1

m n Y Y X X Y X

y y x x f y x f

m n

=

slide-3
SLIDE 3

Vector Notation IV Vector Notation IV

9

Ex 5 4 Random vector X has PDF Ex.5.4: Random vector X has PDF

⎩ ⎨ ⎧ ≥ =

′ −

th i 6 ) ( x e x f

x a X

where a = [1 2 3]′. What is the CDF of X?

⎩

  • therwise

Since a has 3 components, X should be a 3- dimensional random vector. Expanding a′x, write the PDF as a function of the vector components,

⎨ ⎧ ≥ =

− − −

6 ) (

3 2 1

3 2 i x x x

x e x f

Find CDF of X by integrating the PDF w/ respect to

⎩ ⎨ =

  • therwise

) (

X x

f

/ the 3 variables to obtain

Vector Notation V Vector Notation V

10

∫ ∫ ∫

3 2 1

x x x

∫ ∫ ∫

− − −

=

3 2 1 3 2 1

3 2 1 3 2

6 ) (

x x x x x x X

dx dx dx e x F

  • ⎩

⎨ ⎧ ≥ − − − =

− − −

  • therwise

) 1 )( 1 )( 1 ( ) (

3 2 1

3 2 i x x x X

x e e e x F ⎩

  • therwise

Marginal Probability Functions I (5.3) Marginal Probability Functions I (5.3)

11

Study some of the RVs and ignore other ones. For a joint PMF PW X Y Z(w, x, y, z) of discrete RVs

j

W, X, Y, Z( , , y, )

W, X, Y, Z, some marginal PMFs are

∑

= z y x w P z y x P ) ( ) (

∑

∈

=

W

S w Z Y X W Z Y X

z y x w P z y x P ) , , , ( ) , , (

, , , , ,

∑ ∑

= z y x w P z w P ) ( ) (

∑ ∑

∈ ∈

=

X Y

S x S y Z Y X W Z W

z y x w P z w P ) , , , ( ) , (

, , , ,

∑ ∑ ∑

=

Z Y X W X

z y x w P x P ) ( ) (

∑ ∑ ∑

∈ ∈ ∈

W Y Z

S w S y S z Z Y X W X

z y x w P x P ) , , , ( ) (

, , ,

Marginal Probability Functions II Marginal Probability Functions II

12

For a joint PDF fW, X, Y, Z(w, x, y, z) of continuous RVs

W, X, Y, Z, some marginal PDFs are

d f f

∫

∞

) ( ) ( dw z y x w f z y x f

Z Y X W Z Y X

∫ ∞

=

  • ,

, , , ,

) , , , ( ) , , (

∫ ∫

∞ ∞

d d f f ) ( ) (

∫ ∫

∞ − ∞ −

= dy dx z y x w f z w f

Z Y X W Z W

) , , , ( ) , (

, , , ,

∫ ∫ ∫

∞ ∞ ∞

d d d f f ) ( ) (

∫ ∫ ∫

∞ − ∞ − ∞ −

= dz dy dw z y x w f x f

Z Y X W X

) , , , ( ) (

, , ,

slide-4
SLIDE 4

Marginal Probability Functions III Marginal Probability Functions III

13

Ex.5.5: RVs Y1, …, Y4 have the joint PDF

⎨ ⎧ ≤ ≤ ≤ ≤ ≤ ≤ = 1 , 1 4 ) (

4 3 2 1

y y y y y y f

Find the marginal PDFs fY1,Y4(y1, y4), fY2,Y3(y2, y3),

⎩ ⎨ =

  • therwise

) ,..., (

4 1 ,..., 4

1

y y f

Y Y

, ,

and fY3(y3).

∫ ∫

∞ ∞

=

3 2 4 1 4 1

) ,..., ( ) , ( dy dy y y f y y f

Y Y Y Y

∫ ∫

∞ − ∞ − 3 2 4 1 ,..., 4 1 ,

) ,..., ( ) , (

4 1 4 1

dy dy y y f y y f

Y Y Y Y 4 1 1 3 2 4 1 ,

) 1 ( 4 4 ) , (

4 1 4 1

y y dy dy y y f

y y Y Y

− = = ∫ ∫

1

y

∫ ∫

⎩ ⎨ ⎧ ≤ ≤ ≤ ≤ − = th i 1 , 1 ) 1 ( 4 ) , (

4 1 4 1 4 1 , 4

1

y y y y y y f

Y Y

⎩

  • therwise

, 4

1

Marginal Probability Functions IV Marginal Probability Functions IV

14

1

2

y

∫ ∫

) 1 ( 4 4 ) , (

3 2 1 4 1 3 2 ,

3 2 3 2

y y dy dy y y f

y y Y Y

− = = ∫ ∫

⎧ ≤ ≤ ≤ ≤ 1 1 ) 1 ( 4 ⎩ ⎨ ⎧ ≤ ≤ ≤ ≤ − =

  • therwise

1 , 1 ) 1 ( 4 ) , (

3 2 3 2 3 2 , 3

2

y y y y y y f

Y Y

) 1 ( 2 ) 1 ( 4 ) y , (y ) (

3 2 1 3 2 2 3 2 , Y 3

3 2 3

y dy y y dy f y f

Y Y

− = − = =

∫ ∫

∞ ∞ −

⎩ ⎨ ⎧ ≤ ≤ − =

  • therwise

1 ) 1 ( 2 ) (

3 3 3

3

y y y fY ⎩

Independence of Random Variables and Random Vectors I (5.4)

15

X1, …, Xn are independent when the joint PMF or

PDF can be factored into a product of n marginal PMFs or PDFs.

RVs X1, …, Xn are independent if for all x1, …, xn

1,

,

n

p

1,

,

n

Discrete: C ti

) ( ) ( ) ( ) ,..., (

2 1 1 ,...,

2 1 1

n X X X n X X

x P x P x P x x P

n n

  • =

) ( ) ( ) ( ) ( f f f f

Continuous:

) ( ) ( ) ( ) ,..., (

2 1 1 ,...,

2 1 1

n X X X n X X

x f x f x f x x f

n n

  • =

Independence of Random Variables and Random Vectors II

16

Ex.5.6: From Ex.5.5, RVs Y1, …, Y4 have joint PDF

⎨ ⎧ ≤ ≤ ≤ ≤ ≤ ≤ 1 , 1 4 ) (

4 3 2 1

y y y y f

A Y Y i d d t RV ?

⎩ ⎨ =

  • therwise

) ,..., (

4 1 ,..., 4

1

y y f

Y Y

Are Y1, …, Y4 independent RVs? From Ex.5.5, we found the marginal PDF fY1,Y4(y1, y4) So,

1 ) 1 ( 2 ) 1 ( 4 ) , ( ) (

1 1 1 4 4 1 1 4 4 1 , 1

4 1 1

≤ ≤ − = − = =

∫ ∫

y y dy y y dy y y f y f

Y Y Y

1 2 ) 1 ( 4 ) , ( ) (

4 4 1 1 4 1 1 1 4 1 , 4

4 1 4

≤ ≤ = − = =

∫ ∫

y y dy y y dy y y f y f

Y Y Y

slide-5
SLIDE 5

Independence of Random Variables and Random Vectors III

17

Similarly, the marginal PDF fY2,Y3(y2, y3) can be used to find fY2 (y2) as:

1 2 ) 1 ( 4 ) , ( ) (

1 2 1 3 3 2 1 3 3 2 , 2

3 2 2

≤ ≤ = − = =

∫ ∫

y y dy y y dy y y f y f

Y Y Y

So, ) ( ) ( ) ( ) (

4 3 2 1

4 3 2 1

y f y f y f y f

Y Y Y Y

] 2 )][ 1 ( 2 ][ 2 )][ 1 ( 2 [

4 3 2 1

y y y y − − = ] 2 )][ 1 ( 2 ][ 2 )][ 1 ( 2 [

4 3 2 1

y y y y

th i 1 , , , ) 1 ( ) 1 ( 16

4 3 2 1 4 3 2 1

y y y y y y y y ⎩ ⎨ ⎧ ≤ ≤ − − = ) ,..., (

  • therwise

4 1 ,..., 4

1

y y f

Y Y

≠ ⎩ ⎨

Therefore, Y1, …, Y4 are not independent RVs

Independence of Random Variables and Random Vectors IV

18

Independence of n RVs is a property of an

experiment consisting of n independent subexperiments.

Subexperiment i produces RV Xi

p p

i

If all subexperiments follow the same procedure, all

  • f the X have the same PMF or PDF and the RV X
  • f the Xi have the same PMF or PDF, and the RV Xi

are identically distributed.

Independence of Random Variables and Random Vectors V

19

RVs X1, …, Xn are independent and identically

distributed (iid) if Discrete: Continuous:

) ( ) ( ) ( ) ,..., (

2 1 1 ,...,

1

n X X X n X X

x P x P x P x x P

n

  • =

) ( ) ( ) ( ) ( x f x f x f x x f

  • =

Continuous:

Definition of independence: Random vectors X and

Y i d d t if

) ( ) ( ) ( ) ,..., (

2 1 1 ,...,

1

n X X X n X X

x f x f x f x x f

n

  • =

Y are independent if Discrete:

) ( ) ( ) , (

,

y P x P y x P

Y X Y X

=

Continuous:

) ( ) ( ) , (

,

y f x f y x f

Y X Y X

=

Independence of Random Variables and Random Vectors VI

20

Ex.5.7: From Ex.5.5, RVs Y1, …, Y4 have joint PDF

⎨ ⎧ ≤ ≤ ≤ ≤ ≤ ≤ = 1 , 1 4 ) (

4 3 2 1

y y y y y y f

L t V = [Y Y ]’ d W = [Y Y ]’ A V d W

⎩ ⎨ =

  • therwise

) ,..., (

4 1 ,..., 4

1

y y f

Y Y

Let V = [Y1 Y4] and W = [Y2 Y3]. Are V and W independent random vectors? Components of V are V1 = Y1 and V2 = Y4. And W1 = Y2 and W2 = Y3. Therefore,

⎪ ⎨ ⎧ ≤ ≤ ≤ ≤ ≤ ≤ = = , 1 ; 1 4 ) , , , ( ) , (

2 2 1 1 2 2 1 1

v w w v v w w v f w v f

Y Y W V

⎪ ⎩ ⎨

  • therwise.

, ) , , , ( ) , (

2 2 2 2 1 1 ,..., ,

4 1

f f

Y Y W V

slide-6
SLIDE 6

Independence of Random Variables and Random Vectors VII

21

Since V = [Y1 Y4]’ and W = [Y2 Y3]’,

) , ( ) ( ) , ( ) (

2 1 2 1

w w f w f v v f v f

Y Y W Y Y V

= =

From the marginal PDFs fY1,Y4(y1, y4) and fY2,Y3(y2, y ) found in Ex 5 5 we have

) , ( ) ( ) , ( ) (

2 1 , 2 1 ,

3 2 4 1

f f f f

Y Y W Y Y V

y3) found in Ex.5.5, we have

⎩ ⎨ ⎧ ≤ ≤ ≤ ≤ − = th i 1 , 1 ) 1 ( 4 ) , (

4 1 4 1 4 1

4 1

y y y y y y f

Y Y

Therefore,

⎩ ⎨

  • therwise

) (

4 1 , 4

1

y y f

Y Y

⎩ ⎨ ⎧ ≤ ≤ ≤ ≤ − =

  • therwise

1 , 1 ) 1 ( 4 ) (

2 1 2 1

v v v v v fV ⎩

Independence of Random Variables and Random Vectors VIIi

22

and, similarly,

⎩ ⎨ ⎧ ≤ ≤ ≤ ≤ − = the i e 1 , 1 ) 1 ( 4 ) , (

3 2 3 2 3 2 , 3

2

y y y y y y f

Y Y

Therefore,

⎩

  • therwise

, 3

2

⎨ ⎧ ≤ ≤ ≤ ≤ − 1 , 1 ) 1 ( 4 ) (

2 1 2 1

w w w w w f

As a result,

⎩ ⎨ =

  • therwise

) (w fW 1 ) 1 ( ) 1 ( 16 ⎧ ≤ ≤

  • therwise

1 , , , ) 1 ( ) 1 ( 16 ) ( ) (

2 2 1 1 2 1 2 1

v w w v w w v v w f v f

W V

⎩ ⎨ ⎧ ≤ ≤ − − = ) , (

,

w v f

W V

≠

Therefore, V and W are not independent.

Expected Value Vector and Correlation Matrix I (5.6)

23

The expected value of a random vector X is a

column vector E(X) = μX = [E(X1) E(X2) … E(Xn)]′ In other words expected value of a random vector In other words, expected value of a random vector is the vector of expected values. Th l ti f d t X i

The correlation of a random vector X is an n x n

matrix RX with i, jth element RX(i, j) = E(XiXj) RX = E(XX′)

Expected Value Vector and Correlation Matrix II

24

Ex.5.10: If X = [X1 X2 X3]′, the correlation matrix

  • f X is

⎥ ⎥ ⎤ ⎢ ⎢ ⎡ = ⎥ ⎥ ⎤ ⎢ ⎢ ⎡ = ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) (

2 2 3 , , 2 1 3 2 2 2 1 2 3 1 2 1 2 1

3 2 1 2 1 2 1

r X E r r r X E X X E X E X X E X X E X X E X E R

X X X X X X X X X

⎥ ⎥ ⎦ ⎢ ⎢ ⎣ ⎥ ⎥ ⎦ ⎢ ⎢ ⎣ ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) (

2 3 , , , 2 , 2 3 2 3 1 3 3 2 2 1 2

2 3 1 3 3 2 1 2

X E r r X E X X E X X E

X X X X X X X X X

The i, jth element of the correlation matrix is the expected value of the RV XiXj.

slide-7
SLIDE 7

Expected Value Vector and Correlation Matrix III

25

The covariance of a random vector X is an n x n

matrix CX with components CX(i, j) = Cov(Xi, Xj)

j

CX = E[(X − μX)(X − μX)′]

For a random vector X with correlation matrix RX,

i t i C d t t d l covariance matrix CX and vector expected value μX CX = RX − μXμ′X

Expected Value Vector and Correlation Matrix IV

26

Ex.5.12: Find the expected value E(X), the

correlation matrix RX, and the covariance matrix CX

  • f the 2-dimensional random vector X with PDF

⎨ ⎧ ≤ ≤ ≤ 1 2 ) (

2 1

x x x f

Elements of the expected value vector

⎩ ⎨ =

  • therwise

) (x f X

p E(Xi) = W E(X ) 1/3 d E(X ) 2/3 d

2. 1, 2 ) (

2 1 1 2 1

2

= = ∫ ∫

∫ ∫

∞ ∞ − ∞ ∞ −

i dx dx x dx dx x f x

x i X i

We get E(X1) = 1/3 and E(X2) = 2/3, and μX = E(X) = [1/3 2/3]′

Expected Value Vector and Correlation Matrix V

27

The elements of the correlation matrix are

6 / 1 2 ) ( ) (

2 1 2 1 1 2 1 2 1 2 1

2

= = =

∫ ∫ ∫ ∫

∞ ∞

dx dx x dx dx x f x X E

x X

) ( ) (

2 1 1 2 1 1 1

∫ ∫ ∫ ∫

∞ − ∞ −

f X 2 / 1 2 ) ( ) (

2 1 2 2 1 2 1 2 2 2 2

2

= = =

∫ ∫ ∫ ∫

∞ ∞ ∞ ∞

dx dx x dx dx x f x X E

x X

∫ ∫ ∫ ∫

∞ − ∞ −

4 / 1 2 ) ( ) (

2 1 2 1 1 2 1 2 1 2 1

2

= = =

∫ ∫ ∫ ∫

∞ ∞ − ∞ ∞ −

dx dx x x dx dx x f x x X X E

x X

⎤ ⎡ / /

Therefore, RX =

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 2 / 1 4 / 1 4 / 1 6 / 1

From CX = RX − μXμ′X

[ ]

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ − ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = 3 / 2 3 / 1 3 / 2 3 / 1 2 / 1 4 / 1 4 / 1 6 / 1 ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ − ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = 18 / 1 36 / 1 36 / 1 18 / 1 9 / 4 9 / 2 9 / 2 9 / 1 2 / 1 4 / 1 4 / 1 6 / 1

HW 5 HW 5

28

Problem 5.1.3 5 3 4 5.3.4 5.4.1 5.4.1 5.4.5 5.6.8