1 Principal Components Analysis 1. PC analysis is one of several - - PDF document

• Announcement: Papers due on Wednesday.
• Next week set aside for presentations.

You decide amongst yourselves how to split up the presentation duties (everyone should say something though).

• We will do the presentations on Zoom. Too many students in quarantine.

1 Principal Components Analysis

• 1. PC analysis is one of several statistical factor analysis techniques.
• 2. Factor analysis: a method to explain the cross-sectional correlation of

a large number of variables in terms of a smaller number of observed

• r unobserved variables. Fama-French 3 factor model, CAPM, these are
• bserved factors. The econometrician has to take a stand on what the

factors are. The statistical approach says the factors are unobserved and estimates them.

• 3. A two factor model for stock returns, rt,i, i = 1, ...n, t = 1, .., T, has the

two-factor representation if rt,i = αi + δ1,if1,t + δ2,if2,t

• common part

+ ǫt,i

• idiosyncratic

The slope coefficients δ1,i, δ2,i are called factor loadings. The factor load- ings can (and will) differ across firms, according to their exposure to the (risk) factors. The market compensates investors for bearing systematic risk, which is represented in the common part. Idiosyncratic risk can be diversified away by forming portfolios. No compensation for bearing idiosyncratic

• risk. And we start by assuming that f1,t and f2,t are unobserved.
• 4. The method of Principal Components: We make a couple of identifying

assumptions to estimate the factors from returns rt,i. Spoiler alert: the factors are going to be certain clever linear combinations of the returns data. (a) There’s no need to restrict ourselves to returns in computing the PCs. Example: each month, about 30 economic announcements. Can use the announcements as data and estimate factors (principal components) from these. 1

• 5. Write out the factor representation for each firm (ignore the constant)

rt,1 = δ1,1ft,1 + δ2,1f2,t + ǫt,1 rt,2 = δ1,2ft,1 + δ2,2f2,t + ǫt,2 rt,3 = δ1,3ft,1 + δ2,3f2,t + ǫt,3 . . . rt,n = δ1,nft,1 + δ2,nf2,t + ǫt,n We estimate the factors (the principal components) sequentially. Estimate ft,1 first, then ft,2. Let’s assume a one-factor structure and write the equation system above in matrix form. rt,1 = δ1,1ft,1 + ǫt,1 rt,2 = δ1,2ft,1 + ǫt,2 rt,3 = δ1,3ft,1 + ǫt,3 . . . rt,n = δ1,nft,1 + ǫt,n      r1,1 r1,2 · · · r1,n r2,1 r2,2 r2,n . . . rT,1 rT,2 rT,n     

• r

=      δ1,1f1,1 δ1,2f1,1 · · · δ1,nf1,1 δ1,1f2,1 δ1,2f2,1 δ1,nf2,1 . . . δ1,1fT,1 δ1,2fT,1 δ1,nfT,1     

• break apart

r =      f1,1 f2,1 . . . fT,1     

• f1

δ1,1 δ1,2 · · · δ1,n

• δ′

1

r = f1δ′

1

(ignore the idiosyncratic part). The PC is not unique. Let c be some constant r = f1δ′

1 = (f1c)

δ′

1

c

• So we normalize the factors such that var(f1) = 1.
• 6. We want to find f1 and δ1 that explains as much variation in r as possible.

The sum of squares of (r − f1δ′

1) (the unexplained part) is

Tr (r − f1δ′

1)′ (r − f1δ′ 1)

2

where Tr is the trace of the matrix, which is the sum of the diagonal

• elements. We choose f1 and δ1 to minimize this thing. Let me show you.

Let ˜ rt,i = rt,i − ft,1δ1,i. and let T = 3, n = 2. (a) Tr (r − f1δ′

1)′ (r − f1δ′ 1) = Tr

˜ r1,1 ˜ r2,1 ˜ r3,1 ˜ r1,2 ˜ r2,2 ˜ r3,2   ˜ r1,1 ˜ r1,2 ˜ r2,1 ˜ r2,2 ˜ r3,1 ˜ r3,2   = Tr

• ˜

r2

1,1 + ˜

r2

2,1 + ˜

r2

3,1

˜ r1,1˜ r1,2 + ˜ r2,1˜ r2,2 + ˜ r3,1˜ r3,2 ˜ r1,1˜ r1,2 + ˜ r2,1˜ r2,2 + ˜ r3,1˜ r3,2 ˜ r2

1,2 + ˜

r2

2,2 + ˜

r2

3,2

• = ˜

r2

1,1 + ˜

r2

1,2 + ˜

r2

2,1 + ˜

r2

2,2 + ˜

r2

3,1 + ˜

r2

3,2

= (˜ r1,1 − f1,1δ1,1)2 + (˜ r1,2 − f1,1δ1,2)2 + (˜ r2,1 − f2,1δ1,1)2 + (˜ r2,2 − f2,1δ1,2)2 + (˜ r3,1 − f3,1δ1,1)2 + (˜ r3,2 − f3,2δ1,2)2 It’s the sum of squared deviation of every observation from fδ.Choose f and δ to minimize this thing, and we get the first PC ft,1 and the factor loadings δ1

• 7. To get the second PC and second set of factor loadings: Very simple.

Repeat the above procedure, but using returns after controlling for the first factor. That is, replace rt,i with rt,i − ft,1δ1i, and define ˜ rt,i = [(rt,i − ft,1δ1,i) − ft,2δ2,i], choose ft,2 and δ2,i to minimize the Trace of the analogous matrix. The result is rt,i = ft,1δ1,i + ft,2δ2,i + ǫt,i And because the factors are normalized, var (rt,i) = δ2

1,ivar (ft,1) + δ2 2,ivar (ft,2) + var (ǫ t,i)

= δ2

1,i + δ2 2,i + var (ǫt,i)

The fraction of total variation in returns is the sum of squares of the factor

• loadings. Oh, I forgot to mention, the factors are chosen to be mutually

uncorrelated.

• 8. In our dataset with n firms returns, we can compute n principal compo-
• nents. The procedure is useful to determine how many factors explain the
• data. After that, we might want to identify these unobserved factors with
• bservable ones.

3