PS 406 Week 1 Section: Review of OLS and Matrix Algebra D.J. Flynn - - PowerPoint PPT Presentation

PS 406 – Week 1 Section: Review of OLS and Matrix Algebra

D.J. Flynn April 4, 2014

D.J. Flynn PS406 – Week 1 Section Spring 2014 1 / 24

Today’s plan

1

Logistics

2

Matrix algebra review

3

OLS review

4

Lab 1

D.J. Flynn PS406 – Week 1 Section Spring 2014 2 / 24

Logistics

Logistics

Section: Wednesday, 5:00-6:00, Scott 212 Office Hours: Wednesday, 3:30-5:00, Scott 230 Grades: Labs (50%), Final Project (50%) Website: djflynn.org/teaching Rescheduling

Reschedule section next week, 4/9 (I’m out of town) Meet somewhere else on 4/23 (no room available in Scott) We’ll do the L

A

T EX session afer the next section, 6:00-7:00

D.J. Flynn PS406 – Week 1 Section Spring 2014 3 / 24

Matrix algebra review

Matrices

A matrix is a rectangular array of numbers, symbols, or expressions, arranged in rows and columns.1 The individual items in a matrix are called its elements or entries.2 We describe matrices by how many rows and columns they have: rows by columns. This is a 4 x 2 matrix:               

1 5 2 6 3 7 4 8

              

1Thanks Wikipedia. 2Elements/entries are usually numbers, but they can also be letters, images, etc. D.J. Flynn PS406 – Week 1 Section Spring 2014 4 / 24

Matrix algebra review

Vectors and scalars

One-dimensional matrices are called vectors. They come in two varieties: row vectors and column vectors:

• 12

14 16 18 20

• 

                   

2 4 6 8 10

                     A matrix with one row and one column is a scalar.

D.J. Flynn PS406 – Week 1 Section Spring 2014 5 / 24

Matrix algebra review

Matrix operations

We’ll talk briefly about:

1

addition and subtraction

2

multiplication

3

transpose

4

inverse Note: the homeworks won’t ask you to manipulate matrices. This is just so you can better understand the proofs in the slides.

D.J. Flynn PS406 – Week 1 Section Spring 2014 6 / 24

Matrix algebra review

Matrix addition/subtraction

Two matrices must have same number of rows and columns Intuitive: given two matrices A and B, their sum, R, is simply:

R = A + B

For row i and column j, this is:

rij = aij + bij

Example:          

1 2 3 4 5 6

          +          

7 8 9 10 11 12

          =          

8 10 12 14 16 18

         

D.J. Flynn PS406 – Week 1 Section Spring 2014 7 / 24

Matrix algebra review

Matrix multiplication: scalar*matrix

Intuitive: given scalar a and matrix B, the result, R is simply:

R = aB

For row i and column j, this is:

rij = abij

Example: 8

• 2

6 3 7

• =
• 16

48 24 56

• D.J. Flynn

PS406 – Week 1 Section Spring 2014 8 / 24

Matrix algebra review

Matrix multiplication: row vector*column vector

The row vector must have as many columns as the column vector has rows. The product of a row vector and column vector will always be a scalar. Given row vector a and column vector b, their product, r, is:

r =

n

• i=1

aibi

Example:

• 2

6 3

• 

        

8 1 4

          = (2*8) + (6*1) + (3*4) = 34

D.J. Flynn PS406 – Week 1 Section Spring 2014 9 / 24

Matrix algebra review

Matrix multiplication: matrix*matrix

The first matrix must have as many columns as the second matrix has

• rows. (If matrices violate this rule, then multiplying them is illegal).

Multipling these matrices is legal:           −3

4 7

         

• 8

2

• Multiplying these matrices is illegal:

         

8 5 6 1 9 4

                   

2 8 5

          The resulting matrix will have as many rows as the first matrix and as many columns as the second matrix.

D.J. Flynn PS406 – Week 1 Section Spring 2014 10 / 24

Matrix algebra review

Matrix multiplication: matrix*matrix

Given two matrices A and B, the i, j entry of AB is the inner product of the ith row of A and the jth column of B. Example:          

a11 a12 a13 a14 a21 a22 a23 a24 a31 a32 a33 a34

                        

b11 b12 b21 b22 b31 b32 b41 b42

               =           ∗ ∗ ∗ ∗ ∗ (AB)32           (AB)32 = a31b12 + a32b22 + a33b32 + a34b42 Let’s do an example with actual numbers on the board.

D.J. Flynn PS406 – Week 1 Section Spring 2014 11 / 24

Matrix algebra review

Matrix transpose

The transpose of matrix A is denoted AT or A′. The first row of a matrix becomes the first column of the transpose matrix, the second row of the matrix becomes the second column of the transpose, etc. The transpose of a row vector will be a column vector, and the transpose of a column vector will be a row vector. Example: A=          

1 2 3 4 5 6

         

AT=

• 1

3 5 2 4 6

• D.J. Flynn

PS406 – Week 1 Section Spring 2014 12 / 24

Matrix algebra review

Matrix inverse

Algebra review: the inverse of a number is that number which, when multiplied by the original number, gives a product of 1. The inverse of a matrix is that matrix which, when multiplied by the

• riginal matrix, gives the identity matrix:

I =          

1 1 1

          More formally, the matrix A is invertible if there exists a matrix A−1 such that

A−1A = I

and

AA−1 = I

Only square matrices (same number of rows and columns) have inverses – but not all square matrices have inverses.

D.J. Flynn PS406 – Week 1 Section Spring 2014 13 / 24

Matrix algebra review

Matrix inverse3

There are several ways to find the inverse of a matrix. For large matrices, the process is time-consuming. That’s why we use R. For example, this is a shortcut for finding the inverse of a 2x2 matrix:

D.J. Flynn PS406 – Week 1 Section Spring 2014 14 / 24

OLS review

OLS Review

y = Xβ + ǫ

y is an N ∗ 1 vector of responses, X is an N ∗ p matrix of covariates, β is a p ∗ 1 vector of coefficients, ǫ is an N ∗ 1 vector of errors.

D.J. Flynn PS406 – Week 1 Section Spring 2014 15 / 24

OLS review

The OLS estimator in matrix form

We know from lecture that ˆ β = (XTX)−1XTy We know from last quarter that this equals

• (Xi − ¯

X)(Yi − ¯ Y)

• (Xi − ¯

X)2

D.J. Flynn PS406 – Week 1 Section Spring 2014 16 / 24

OLS review

Suppose we have a dataset, N = 10, p = 5

D.J. Flynn PS406 – Week 1 Section Spring 2014 17 / 24

OLS review

The response vector: n x 1

D.J. Flynn PS406 – Week 1 Section Spring 2014 18 / 24

OLS review

The response vector: n x 1                                                 

Yi1 Yi2 Yi3 Yi4 Yi5 Yi6 Yi7 Yi8 Yi9 Yi10

                                                

D.J. Flynn PS406 – Week 1 Section Spring 2014 18 / 24

OLS review

The covariate matrix: n x p

D.J. Flynn PS406 – Week 1 Section Spring 2014 19 / 24

OLS review

The covariate matrix: n x p                                                 

1 X1,i1 X2,i1 X3,i1 X4,i1 X5,i1 1 X1,i2 X2,i2 X3,i2 X4,i2 X5,i2 1 X1,i3 X2,i3 X3,i3 X4,i3 X5,i3 1 X1,i4 X2,i4 X3,i4 X4,i4 X5,i4 1 X1,i5 X2,i5 X3,i5 X4,i5 X5,i5 1 X1,i6 X2,i6 X3,i6 X4,i6 X5,i6 1 X1,i7 X2,i7 X3,i7 X4,i7 X5,i7 1 X1,i8 X2,i8 X3,i8 X4,i8 X5,i8 1 X1,i9 X2,i9 X3,i9 X4,i9 X5,i9 1 X1,i10 X2,i10 X3,i10 X4,i10 X5,i10

                                                

D.J. Flynn PS406 – Week 1 Section Spring 2014 19 / 24

OLS review

The parameter vector: k x 1

D.J. Flynn PS406 – Week 1 Section Spring 2014 20 / 24

OLS review

The parameter vector: k x 1                      β1 β2 β3 β4 β5                     

D.J. Flynn PS406 – Week 1 Section Spring 2014 20 / 24

OLS review

The error vector: n x 1

D.J. Flynn PS406 – Week 1 Section Spring 2014 21 / 24

OLS review

The error vector: n x 1                                                  ǫ1 ǫ2 ǫ3 ǫ4 ǫ5 ǫ6 ǫ7 ǫ8 ǫ9 ǫ10                                                 

D.J. Flynn PS406 – Week 1 Section Spring 2014 21 / 24

OLS review

Keep in mind: a lot of stuff goes in ǫ

• mitted variables

measurement error correlation among regressors anomalous cases causal heterogeneity

Using our knowledge from 405, how might we account for this? We’ll learn better ways in 406.

D.J. Flynn PS406 – Week 1 Section Spring 2014 22 / 24

OLS review

OLS Assumptions4

1

The y variables are drawn from Xβ + ǫ

2

ǫ is i.i.d. with mean 0 and variance σ2

3

X ǫ

4

ǫ ∼ N(0, σ2)

4Standard disclaimer: every regression text writes these differently. D.J. Flynn PS406 – Week 1 Section Spring 2014 23 / 24

Lab 1

Lab 15

Lab 1: OLS, interactions, diagnostics Any questions?

5Note: PS405 Final Exams will be returned with comments soon. Sorry for the delay. D.J. Flynn PS406 – Week 1 Section Spring 2014 24 / 24