The Multivariate Normal Distribution Defn : Z R 1 N (0 , 1) iff 1 - PDF document

The Multivariate Normal Distribution Defn : Z ∈ R 1 ∼ N (0 , 1) iff 1 2 πe − z 2 / 2 . f Z ( z ) = √ Defn : Z ∈ R p ∼ MV N (0 , I ) if and only if Z = ( Z 1 , . . . , Z p ) t with the Z i independent and each Z i ∼ N (0 , 1). In this case according to our theorem 1 2 πe − z 2 i / 2 � √ f Z ( z 1 , . . . , z p ) = = (2 π ) − p/ 2 exp {− z t z/ 2 } ; superscript t denotes matrix transpose. Defn : X ∈ R p has a multivariate normal distri- bution if it has the same distribution as AZ + µ for some µ ∈ R p , some p × p matrix of constants A and Z ∼ MV N (0 , I ). 34

Matrix A singular: X does not have a density. A invertible: derive multivariate normal density by change of variables: X = AZ + µ ⇔ Z = A − 1 ( X − µ ) ∂X ∂Z ∂X = A − 1 . ∂Z = A So f X ( x ) = f Z ( A − 1 ( x − µ )) | det( A − 1 ) | = exp {− ( x − µ ) t ( A − 1 ) t A − 1 ( x − µ ) / 2 } . (2 π ) p/ 2 | det A | Now define Σ = AA t and notice that Σ − 1 = ( A t ) − 1 A − 1 = ( A − 1 ) t A − 1 and det Σ = det A det A t = (det A ) 2 . Thus f X is exp {− ( x − µ ) t Σ − 1 ( x − µ ) / 2 } ; (2 π ) p/ 2 (det Σ) 1 / 2 the MV N ( µ, Σ) density. Note density is the same for all A such that AA t = Σ. This justi- fies the notation MV N ( µ, Σ). 35

For which µ , Σ is this a density? Any µ but if x ∈ R p then x t Σ x = x t AA t x = ( A t x ) t ( A t x ) p y 2 � = i ≥ 0 1 where y = A t x . Inequality strict except for y = 0 which is equivalent to x = 0. Thus Σ is a positive definite symmetric matrix. Conversely, if Σ is a positive definite symmet- ric matrix then there is a square invertible ma- trix A such that AA t = Σ so that there is a MV N ( µ, Σ) distribution. ( A can be found via the Cholesky decomposition, e.g.) When A is singular X will not have a density: ∃ a such that P ( a t X = a t µ ) = 1; X is confined to a hyperplane. Still true: distribution of X depends only on if AA t = BB t then AZ + µ and Σ = AA t : BZ + µ have the same distribution. 36

Properties of the MV N distribution 1 : All margins are multivariate normal: if � � X 1 X = X 2 � � µ 1 µ = µ 2 and � � Σ 11 Σ 12 Σ = Σ 21 Σ 22 then X ∼ MV N ( µ, Σ) ⇒ X 1 ∼ MV N ( µ 1 , Σ 11 ). 2 : All conditionals are normal: the conditional distribution of X 1 given X 2 = x 2 is MV N ( µ 1 + Σ 12 Σ − 1 22 ( x 2 − µ 2 ) , Σ 11 − Σ 12 Σ − 1 22 Σ 21 ) MX + ν ∼ MV N ( Mµ + ν, M Σ M t ): 3 : affine transformation of MVN is normal. 37