Lecture 8 ARMA Models Colin Rundel 02/13/2017 1 AR(p) 2 AR(p) - - PowerPoint PPT Presentation

Lecture 8

ARMA Models

Colin Rundel 02/13/2017

1

AR(p)

2

AR(p)

From last time, AR(p) : yt = δ + ϕ1 yt−1 + ϕ2 yt−2 + · · · + ϕp yt−p + wt

= δ + wt +

p

∑

i=1

ϕi yt−i

What are the properities of AR(p),

• 1. Expected value?
• 2. Covariance / correlation?
• 3. Stationarity?

3

Lag operator

The lag operator is convenience notation for writing out AR (and other) time series models. We define the lag operator L as follows, L yt = yt−1 this can be generalized where, L2yt L L yt L yt

1

yt

2

therefore, Lk yt yt

k 4

Lag operator

The lag operator is convenience notation for writing out AR (and other) time series models. We define the lag operator L as follows, L yt = yt−1 this can be generalized where, L2yt = L L yt

= L yt−1 = yt−2

therefore, Lk yt = yt−k

4

Lag polynomial

An AR(p) model can be rewitten as yt = δ + ϕ1 yt−1 + ϕ2 yt−2 + · · · + ϕp yt−p + wt yt = δ + ϕ1 L yt + ϕ2 L2 yt + · · · + ϕp Lp yt + wt yt − ϕ1 L yt − ϕ2 L2 yt − · · · − ϕp Lp yt = δ + wt

(1 − ϕ1 L − ϕ2 L2 − · · · − ϕp Lp) yt = δ + wt

This polynomial of the lags

p L

1

1 L 2 L2 p Lp

is called the lag or characteristic polynomial of the AR process.

5

Lag polynomial

An AR(p) model can be rewitten as yt = δ + ϕ1 yt−1 + ϕ2 yt−2 + · · · + ϕp yt−p + wt yt = δ + ϕ1 L yt + ϕ2 L2 yt + · · · + ϕp Lp yt + wt yt − ϕ1 L yt − ϕ2 L2 yt − · · · − ϕp Lp yt = δ + wt

(1 − ϕ1 L − ϕ2 L2 − · · · − ϕp Lp) yt = δ + wt

This polynomial of the lags

ϕp(L) = (1 − ϕ1 L − ϕ2 L2 − · · · − ϕp Lp)

is called the lag or characteristic polynomial of the AR process.

5

Stationarity of AR(p) processes

An AR(p) process is stationary if the roots of the characteristic polynomial lay outside the complex unit circle Example AR(1):

6

Stationarity of AR(p) processes

An AR(p) process is stationary if the roots of the characteristic polynomial lay outside the complex unit circle Example AR(1):

6

Example AR(2)

7

AR(2) Stationarity Conditions

From http://www.sfu.ca/~baa7/Teaching/econ818/StationarityAR2.pdf

8

Proof

We can rewrite the AR(p) model into an AR(1) form using matrix notation yt = δ + ϕ1 yt−1 + ϕ2 yt−2 + · · · + ϕp yt−p + wt

ξt = δ + F ξt−1 + wt

where

       

yt yt−1 yt−2 . . . yt−p+1

       

=

       

δ

. . .

       

+

       

ϕ1 ϕ2 ϕ3 · · · ϕp−1 ϕp

1

· · ·

1

· · ·

. . . . . . . . .

· · ·

. . . . . . 1

· · ·

1

               

yt−1 yt−2 yt−3 . . . yt−p

       

+

       

wt . . .

       

=

       

δ + wt + ∑p

i=1 ϕi yt−i

yt−1 yt−2 . . . yt−p+1

       

9

Proof sketch (cont.)

So just like the original AR(1) we can expand out the autoregressive equation

ξt = δ + wt + F ξt−1 = δ + wt + F (δ + wt−1) + F2 (δ + wt−2) + · · · + Ft−1 (δ + w1) + Ft (δ + w0) = δ

t

∑

i=0

Fi +

t

∑

i=0

Fi wt−i and therefore we need lim

t→∞Ft → 0. 10

Proof sketch (cont.)

We can find the eigen decomposition such that F = QΛQ−1 where the columns of Q are the eigenvectors of F and Λ is a diagonal matrix of the corresponding eigenvalues. A useful property of the eigen decomposition is that Fi = QΛiQ−1 Using this property we can rewrite our equation from the previous slide as

t t i

Fi

t i

Fi wt

i t i

Q

iQ 1 t i

Q

iQ 1 wt i 11

Proof sketch (cont.)

We can find the eigen decomposition such that F = QΛQ−1 where the columns of Q are the eigenvectors of F and Λ is a diagonal matrix of the corresponding eigenvalues. A useful property of the eigen decomposition is that Fi = QΛiQ−1 Using this property we can rewrite our equation from the previous slide as

ξt = δ

t

∑

i=0

Fi +

t

∑

i=0

Fi wt−i

= δ

t

∑

i=0

QΛiQ−1 +

t

∑

i=0

QΛiQ−1 wt−i

11

Proof sketch (cont.)

Λi =

     

λi

1

· · · λi

2

· · ·

. . . . . . ... . . .

· · · λi

p

     

Therefore, lim

t→∞Ft → 0

when lim

t→∞Λt → 0

which requires that

|λi| < 1

for all i

12

Proof sketch (cont.)

Eigenvalues are defined such that for λ, det(F − λ I) = 0 based on our definition of F our eigenvalues will therefore be the roots of

λp − ϕ1 λp−1 − ϕ2 λp−2 − · · · − ϕp1 λ1 − ϕp = 0

which if we multiply by 1

p where L

1 gives 1

1 L 2 L2 p1 Lp 1 p Lp 13

Proof sketch (cont.)

Eigenvalues are defined such that for λ, det(F − λ I) = 0 based on our definition of F our eigenvalues will therefore be the roots of

λp − ϕ1 λp−1 − ϕ2 λp−2 − · · · − ϕp1 λ1 − ϕp = 0

which if we multiply by 1/λp where L = 1/λ gives 1 − ϕ1 L − ϕ2 L2 − · · · − ϕp1 Lp−1 − ϕp Lp = 0

13

Properties of AR(p)

For a stationary AR(p) process where wt has E(wt) = 0 and Var(wt) = σ2

w

E(Yt) =

δ

1 − ϕ1 − ϕ2 − · · · − ϕp Var(Yt) = γ0 = ϕ1γ1 + ϕ2γ2 + · · · + ϕpγp + σ2

w

Cov(Yt, Yt−j) = γj = ϕ1γj−1 + ϕ2γj−2 + · · · + ϕpγj−p Corr(Yt, Yt−j) = ρj = ϕ1ρj − 1 + ϕ2ρj − 2 + · · · + ϕpρj−p

14

Moving Average (MA) Processes

15

MA(1)

A moving average process is similar to an AR process, except that the autoregression is on the error term. MA(1) : yt = δ + wt + θ wt−1 Properties:

16

Time series

17

ACF

2 4 6 8 10 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF

θ=−0.1

2 4 6 8 10 −0.5 0.0 0.5 1.0 Lag ACF

θ=−0.8

2 4 6 8 10 −0.2 0.2 0.4 0.6 0.8 1.0 Lag ACF

θ=−2.0

2 4 6 8 10 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF

θ=0.1

2 4 6 8 10 −0.2 0.2 0.4 0.6 0.8 1.0 Lag ACF

θ=0.8

2 4 6 8 10 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF

θ=2.0

18

MA(q)

MA(q) : yt = δ + wt + θ1 wt−1 + θ2 wt−2 + · · · + θq wt−q Properties:

19

Time series

20

ACF

2 4 6 8 10 −0.5 0.0 0.5 1.0 Lag ACF

θ={−1.5}

2 4 6 8 10 −0.2 0.2 0.6 1.0 Lag ACF

θ={−1.5, −1}

2 4 6 8 10 −0.5 0.0 0.5 1.0 Lag ACF

θ={−1.5, −1, 2}

2 4 6 8 10 −0.5 0.0 0.5 1.0 Lag ACF

θ={−1.5, −1, 2, 3}

21

ARMA Model

22

ARMA Model

An ARMA model is a composite of AR and MA processes, ARMA(p, q) : yt = δ + ϕ1 yt−1 + · · · ϕp yt−p + wt + θ1wt−1 + · · · + θqwtq

ϕp(L)yt = δ + θq(L)wt

Since all MA processes are stationary, we only need to examine the AR aspect to determine stationarity (roots of ϕp(L) lie outside the complex unit circle).

23

Time series

24

ACF

2 4 6 8 10 −0.2 0.2 0.6 1.0 Lag ACF

φ={0.9}, θ={−}

2 4 6 8 10 −0.5 0.0 0.5 1.0 Lag ACF

φ={−0.9}, θ={−}

2 4 6 8 10 −0.2 0.2 0.6 1.0 Lag ACF

φ={−}, θ={0.9}

2 4 6 8 10 −0.5 0.0 0.5 1.0 Lag ACF

φ={−}, θ={−0.9}

2 4 6 8 10 −0.2 0.2 0.6 1.0 Lag ACF

φ={0.9}, θ={0.9}

2 4 6 8 10 −0.2 0.2 0.6 1.0 Lag ACF

φ={−0.9}, θ={0.9}

2 4 6 8 10 −0.2 0.2 0.6 1.0 Lag ACF

φ={0.9}, θ={−0.9}

2 4 6 8 10 −0.2 0.2 0.6 1.0 Lag ACF

φ={−0.9}, θ={−0.9}

25