[ ] F , , , = is the m -dimisional vector of F F F - - PowerPoint PPT Presentation

SLIDE 1

Factor analysis (cf. sections 9.1-9.3) Example 9.3: In a consumer preference study a random sample of customers were asked to rate several attributes of a new product using a 7-point scale Sample correlation matrix

1

The variables seem to form two “groups” of variables: {1, 3} and {2, 4, 5} Example 9.8: We register the scores in 6 subject areas for 220 students Sample correlation matrix

2

There are indications for 3 “groups” of variables: {1, 2}, {3}, and {4, 5, 6} Model formulation: An observable random vector X of dimension p has mean vector and covariance matrix Σ The (orthogonal) factor model postulates that X depends linearly on unobservable random variables (latent variables) F1, F2, ..., Fm, called common factors and p additional (unobservable) sources of variation ε1, ε2, ...., εp, called errors or specific factors

3

Model formulation:

1 1 11 1 12 2 1 1 m m

X l F l F l F µ ε − = + + + ⋯

2 2 21 1 22 2 2 2 m m

X l F l F l F µ ε − = + + + ⋯

1 1 2 2 p p p p pm m p

X l F l F l F µ ε − = + + + ⋯

The coefficient is called the loading of the i-th variable on the j-th factor

ij

l ⋮ − = + X

• LF

ε

In matrix notation the factor model takes the form: Here is the p x m matrix of factor loadings, is the m-dimisional vector of common factors, and is the p-vector of errors

{ }

ij

l = L

[ ]

1 2

, , ,

m

F F F ′ = F …

1 2

, , ,

p

ε ε ε ′   =   ε …

4

( ) Cov( ) ( ) E E = ′ = = F F FF I

p-vector of errors Assumptions:

1 2

( ) Cov( ) ( ) diag{ , , , }

p

E E ψ ψ ψ = ′ = = = ε ε εε Ψ … Cov( , ) = ε F

SLIDE 2

( )( )′ − − X X

• The model implies a structure of the covariance matrix Σ of X

To find this structure, we first note that

( )( )′ = + + LF ε LF ε ( ) ( ) ′ ′ ′ ′ = + + + LF LF ε LF LFε εε ( ) ( ) ( ) ′ ′ ′ ′ ′ ′ = + + + L FF L εF L L Fε εε cov( ) {( )( ) } E ′ = = − − Σ X X X

• Therefore

5

cov( ) {( )( ) } E ′ = = − − Σ X X X

• (

) ( ) ( ) ( ) E E E E ′ ′ ′ ′ ′ ′ = + + + L FF L εF L L Fε εε ′ ′ ′ = + + + LIL 0L L 0 Ψ ′ = + LL Ψ

We may write

Var( )

ii i

X σ =

2 1 m ij i j

l ψ

=

= +

∑

2

def communality specific variance

i i

h ψ = +

• (

) ′ − X F

Further we have

( ) ′ = + LF ε F ( ) ′ ′ = + L FF εF

so that

1

Cov( , )

m i k ij kj j

X X l l

=

=∑

Also

6

Cov( , ) {( ) } E ′ = − X F X F ( ) ( ) E E ′ ′ = + L FF εF = L Cov( , )

i j ij

X F l =

This gives The factor loadings are the covariances between the

• bservable variables and the unobservable factors

Example 9.1: Consider the covariance matrix A direct computation shows that

7

Or

′ = + Σ LL Ψ

When we have no structure for the covariance matrix Σ, it depends on p(p+1)/2 parameters, namely the p variances and the p(p –1)/2 covariances

ii

σ ,

ik i

k σ ≠

When the relation holds, the covariance matrix Σ, may be expressed by p(m+1) parameters, namely the pm factor loadings and the p specific variances

ij

l

i

ψ

′ = + Σ LL Ψ

8

the pm factor loadings and the p specific variances

ij

l

i

ψ

Unfortunately many covariance matrix Σ, may not be expressed as with m (much) smaller than p (Example 9.2 discusses one problem that may occur)

′ = + Σ LL Ψ

SLIDE 3

There is some inherent ambiguity associated with the factor model when m>1 To look closer at this, let T be a m x m orthogonal matrix

− = + X

• LF

ε

The factor model may then be reformulated as

′ = + LTT F ε

* *

= + L F ε

where

* *

and ′ = = L LT F T F

9

Now

*

( ) ( ) E E ′ = = F T F

*

Cov( ) Cov( ) ′ ′ = = = F T F T T T I

Therefore it is impossible on the basis of observations to distinguish the loadings L from the loadings L* Thus the factor loadings L are determined only up to an orthogonal matrix T There are different methods for estimation of the factor model We will consider:

• estimation using principal components
• maximum likelihood estimation

10

• maximum likelihood estimation

For both methods the solution may be rotated by multiplication by an orthogonal matrix to simplify the interpretation of factors (to be described later)