Lecture 4, Non-linear Time Series Erik Lindstrm Its not a bug, its - - PowerPoint PPT Presentation

Lecture 4, Non-linear Time Series

Erik Lindström

“It’s not a bug, it’s a feature!”

◮ Why are we using linear models?

◮ Properties ◮ Limitations

Properties of non-linear systems.

Limit cycles Jumps Non-symmetric distributions Bifurcations Chaos Non-linear dependence

“It’s not a bug, it’s a feature!”

◮ Why are we using linear models?

◮ Properties ◮ Limitations

◮ Properties of non-linear systems.

◮ Limit cycles ◮ Jumps ◮ Non-symmetric distributions ◮ Bifurcations ◮ Chaos ◮ Non-linear dependence

General properties

◮ Assume causal system

f(Yn, Yn−1, . . . , Y1) = εn

◮ Invertable system

Yn = f⋆(εn, . . . , ε1)

◮ Volterra series.

Suppose that f is sufficiently well-behaved, then there exists a sequence of bounded functions

k k k 0 l kl k 0 l 0 m klm

General properties

◮ Assume causal system

f(Yn, Yn−1, . . . , Y1) = εn

◮ Invertable system

Yn = f⋆(εn, . . . , ε1)

◮ Volterra series. Suppose that f∗ is sufficiently

well-behaved, then there exists a sequence of bounded functions

∞

∑

k=0

|ψk| < ∞,

∞

∑

k=0 ∞

∑

l=0

|ψkl| < ∞,

∞

∑

k=0 ∞

∑

l=0 ∞

∑

m=0

|ψklm| < ∞, ...

Volterra series

where µ = f∗(0), ψk = ( ∂f∗ ∂ϵt−k ), ψkl = ( ∂2f∗ ∂ϵt−k∂ϵt−l ), ... (1) Approximate the general model by Yt

k k t k k 0 l kl t k t l k 0 l 0 m klm t k t l t m

(2) This results in generalized transfer functions. NOTE that superposition is lost! These transfer functions does not care if is deterministic of stochastic!

Volterra series

where µ = f∗(0), ψk = ( ∂f∗ ∂ϵt−k ), ψkl = ( ∂2f∗ ∂ϵt−k∂ϵt−l ), ... (1) Approximate the general model by Yt = µ +

∞

∑

k=0

ψkϵt−k +

∞

∑

k=0 ∞

∑

l=0

ψklϵt−kϵt−l +

∞

∑

k=0 ∞

∑

l=0 ∞

∑

m=0

ψklmϵt−kϵt−lϵt−m + . . . (2) This results in generalized transfer functions. NOTE that superposition is lost! These transfer functions does not care if is deterministic of stochastic!

Volterra series

where µ = f∗(0), ψk = ( ∂f∗ ∂ϵt−k ), ψkl = ( ∂2f∗ ∂ϵt−k∂ϵt−l ), ... (1) Approximate the general model by Yt = µ +

∞

∑

k=0

ψkϵt−k +

∞

∑

k=0 ∞

∑

l=0

ψklϵt−kϵt−l +

∞

∑

k=0 ∞

∑

l=0 ∞

∑

m=0

ψklmϵt−kϵt−lϵt−m + . . . (2) This results in generalized transfer functions. NOTE that superposition is lost! These transfer functions does not care if {ϵ} is deterministic of stochastic!

Frequency doubling

Now assume that we introduce a spectral representation of the noise.

◮ Let’s start with a single frequency, ϵk = A exp(iω ∗ k)

This results in frequency doubling Proof by inserting the signal in Eq (2). Question: What happens with a non-linear system if the noise

k is white noise?

Conclusion: Black box non-linear system identification is far more complicated that linear system identification.

Frequency doubling

Now assume that we introduce a spectral representation of the noise.

◮ Let’s start with a single frequency, ϵk = A exp(iω ∗ k) ◮ This results in frequency doubling ◮ Proof by inserting the signal in Eq (2).

Question: What happens with a non-linear system if the noise

k is white noise?

Conclusion: Black box non-linear system identification is far more complicated that linear system identification.

Frequency doubling

Now assume that we introduce a spectral representation of the noise.

◮ Let’s start with a single frequency, ϵk = A exp(iω ∗ k) ◮ This results in frequency doubling ◮ Proof by inserting the signal in Eq (2). ◮ Question: What happens with a non-linear system if

the noise ϵk is white noise? Conclusion: Black box non-linear system identification is far more complicated that linear system identification.

Frequency doubling

Now assume that we introduce a spectral representation of the noise.

◮ Let’s start with a single frequency, ϵk = A exp(iω ∗ k) ◮ This results in frequency doubling ◮ Proof by inserting the signal in Eq (2). ◮ Question: What happens with a non-linear system if

the noise ϵk is white noise?

◮ Conclusion: Black box non-linear system identification

is far more complicated that linear system identification.

Regime models

The model is generated from a set of simple models

◮ SETAR ◮ STAR ◮ HMM

SETAR - Self-Exciting Threshold AR

The SETAR(l; d; k1, k2, . . . , kl) model is given by : Yt = a(Jt) +

kJt

∑

i=1

a(Jt)

i

Yt−i + ϵ(Jt)

t

(3) where the index (Jt) is described by Jt =          1 for Yt−d ∈ R1 2 for Yt−d ∈ R2 . . . . . . l for Yt−d ∈ Rl. (4) NOTE that it is difficult to estimate the boundaries for the regimes

SETAR - Self-Exciting Threshold AR

The SETAR(l; d; k1, k2, . . . , kl) model is given by : Yt = a(Jt) +

kJt

∑

i=1

a(Jt)

i

Yt−i + ϵ(Jt)

t

(3) where the index (Jt) is described by Jt =          1 for Yt−d ∈ R1 2 for Yt−d ∈ R2 . . . . . . l for Yt−d ∈ Rl. (4) NOTE that it is difficult to estimate the boundaries for the regimes

SETARMA

◮ Similar ideas can be included in ARMA models,

leading to SETARMA models.

◮ Often easy to add ’asymmetric’ terms in the AR or

MA polynomials, e.g. yn + a1yn−1 = en + ( c1 + c′

11{en−1≤0}

) en−1

STAR - Smooth Threshold AR

The STAR(k) model: Yt = a0+

k

∑

j=1

ajYt−j+  b0 +

k

∑

j=1

bjYt−j   G(Yt−d)+ϵt (5) where G(Yt−d) now is the transition function lying between zero and one, as for instance the standard Gaussian distribution. In the literature two specifications for G(·) are commonly considered, namely the logistic and exponential functions: G(y) = (1 + exp(−γL(y − cL)))−1; γL > 0 (6) G(y) = 1 − exp(−γE(y − cE)2); γE > 0 (7) where γL and γE are transition parameters, cL and cE are threshold parameters (location parameters).

PJM electricity market

Prices at the PJM market

Simple model of the power market

◮ Demand

D(Q) = a + bQ + c cos(2πt/50) + ε (8)

◮ Supply

S(Q) = α0+β0Q+G(Q, Qbreak)(α1+β1(Q−Qbreak)+) (9) where G is a transition function.

◮ Solve numerically for t = 1, . . . to get the quantity Q

and price P.

Supply and Demand

50 60 70 80 90 100 110 120 100 200 300 400 500 600 700 800 900 1000 Supply MaxDemand MinDemand

Figure: Supply and demand curves (varies across the season) for

• ur artificial market

Prices

50 100 150 200 250 250 300 350 400 450 500 550 600 650 700 750

Figure: Note the seasonality as well as the non-Gaussian distribution.

Distribution of prices

300 350 400 450 500 550 600 650 700

Data

0.001 0.003 0.01 0.02 0.05 0.10 0.25 0.50 0.75 0.90 0.95 0.98 0.99 0.997 0.999

Probability Normal Probability Plot

Figure: Same property

HMM - Hidden Markov Models

Another alternative is to let the regime shift stochastically, as in the Hidden Markov Model. Let Yt = a(Jt) +

kJt

∑

i=1

a(Jt)

i

Yt−i + ϵ(Jt)

t

(10) where the state variable Jt follows a latent Markov chain. NOTE that parameter estimation is slightly more complicated than before.

HMM - Hidden Markov Models

Another alternative is to let the regime shift stochastically, as in the Hidden Markov Model. Let Yt = a(Jt) +

kJt

∑

i=1

a(Jt)

i

Yt−i + ϵ(Jt)

t

(10) where the state variable Jt follows a latent Markov chain. NOTE that parameter estimation is slightly more complicated than before.

Case: Electricity spot price, (Regland & Lindström, 2012)

The electricity spot price is very non-Gaussian

Feb05 Feb07 Feb09 50 100 150 200 250 EEX spot Feb05 Feb07 Feb09 −4 −3 −2 −1 1 EEX log(spot)−log(forward)

Figure: The electricity spot price (left) and spread, defined as the difference between the logarithm of the spot and the logarithm

• f the forward (right). Data from the German EEX market.

◮ The spread accounts for virtually all seasonality, but

there are still bursts of volatility.

◮ The logarithm of the spot, yt, was modeled using a

HMM regime switching model with three states, a normal state with mean-reverting dynamics, a spike (upward jumps) state and a drop (downward jumps) state.

This is mathematically given by : ∆y(B)

t+1 = α

( µt − y(B)

t

) + σϵt y(S)

t+1 = ZS,t + µt,

ZS ∼ F (µS, σS) y(D)

t+1 = −ZD,t + µt,

ZD ∼ F (µD, σD) where µt is approximately the logarithm of the month ahead forward price. The regimes are switching according to a Markov chain Rt = {B, S, D} governed by the transition matrix Π =   1 − πBU − πBD πBS πBD πSB 1 − πSB πDB 1 − πDB   .

Feb05 Feb06 Feb07 Feb08 Feb09 Feb10 −4 −3 −2 −1 1 Spread Feb05 Feb06 Feb07 Feb08 Feb09 Feb10 −1 1 Regime prob

Figure: Fit of the independent spike model applied to EEX data

Extension used for stability evaluation of the power system in (Lindström, Norén & Madsen, 2015) by making the transition matrix time-inhomogeneous.

Case: What happens with large scale introduction of electric cars/battery?

Jan02 Jan04 Jan06 Jan08 Jan10 Jan12 0.4 0.5 0.6 0.7 0.8 0.9 1 Modified Normalized Consumption 0 % 10 %

Battery capacity (%) 5 10 15 Base prob. 0.8794 0.8827 0.9066 0.9461 Spike prob. 0.0304 0.0292 0.0196 0.0081 Drop prob. 0.0902 0.0881 0.0738 0.0458

Table: Unconditional regime probabilities when having a perfect battery with 0, 5, 10, and 15 % system capacity.

HMMs for portfolio optimization

Recall the stylized facts for stoch indices. We can also use HMMs for portfolio optimization. Model given by Xt

St St

(11) with

1 2 1 2,

• stat. prob. for the first state

and

11 22

1 is the second largest eigenvalue to the transition matrix, . First, consider the autocorrelation: r k

1 1 1 1 2 2 2 k

(12) See Nystrup et al (2016) for details.

HMMs for portfolio optimization

Recall the stylized facts for stoch indices. We can also use HMMs for portfolio optimization.

◮ Model given by

Xt = µSt + εSt (11) with µ1, µ2, σ1, σ2, π stat. prob. for the first state and λ = γ11 + γ22 − 1 is the second largest eigenvalue to the transition matrix, Γ. First, consider the autocorrelation: r(k) = π1(1 − π1)(µ1 − µ2)2 σ2 λk (12) See Nystrup et al (2016) for details.

Simulation example

Here the model is given by Γ = [ 0.98 0.02 0.1 0.9 ] . with µ = [0.01 − 0.02] and σ = [0.04 0.20]. Interpretation of parameters: Staying on average 1/(1 − 0.98) = 50 days in the good state vs 10 days in the bad state.

Realizations

100 200 300 400 500 600 700 800 900 1000 2 4 6 100 200 300 400 500 600 700 800 900 1000 1 1.5 2 100 200 300 400 500 600 700 800 900 1000

• 1

1

Figure: Cumulative returns (top), Markov states (middle) and returns (bottom).

Autocorrelation

2 4 6 8 10 12 14 16 18 20 Lag

• 0.2

0.2 0.4 0.6 0.8 1 Sample Autocorrelation Sample Autocorrelation Function 2 4 6 8 10 12 14 16 18 20 Lag

• 0.2

0.2 0.4 0.6 0.8 1 Sample Autocorrelation Sample Autocorrelation Function

Figure: Autocorrelation for returns (left) and abs returns (right)

Trading strategy

Figure: Trading strategy in US stocks and bonds

General State space models

A Grey box approach is to include as much prior knowledge as possible. Consider the General State Space model: xn

1

f n xn un g n xn un en

1

yn

1

h n 1 xn

1 un 1

wn

1

where xn n

0 is a latent process and

yn n

0 is the

sequence of observations. Interpretations? Practical considerations

General State space models

A Grey box approach is to include as much prior knowledge as possible. Consider the General State Space model: xn+1 = f(n, xn, un) + g(n, xn, un)en+1 yn+1 = h(n + 1, xn+1, un+1) + wn+1 where {xn}n≥0 is a latent process and {yn}n≥0 is the sequence of observations.

◮ Interpretations? ◮ Practical considerations

Example: The Black & Scholes model

The Black & Scholes (1973) model is often used for option valuation. x :dS = µStdt + σStdWt, y : [ SMarket

n

cMarket

K

(Sn, ·) ] = [ SModel

n

cModel

K

(Sn, ·) ] + wn.

• Ask-Bix spread

This structure allows us to separate actual price variation from market micro structure.

Some references

◮ Lindström, E., & Regland, F. (2012). Modeling

extreme dependence between European electricity

• markets. Energy economics, 34(4), 899-904. DOI:

10.1016/j.eneco.2012.04.006

◮ Lindström, E., Norén, V., & Madsen, H. (2015).

Consumption management in the Nord Pool region: A stability analysis. Applied Energy, 146, 239-246. DOI: 10.1016/j.apenergy.2015.01.113

◮ Nystrup, P., Hansen, B. W., Madsen, H., &

Lindström, E. (2015). Regime-based versus static asset allocation: Letting the data speak. The Journal

• f Portfolio Management, 42(1), 103-109. DOI:

10.3905/jpm.2015.42.1.103

◮ Nystrup, P., Madsen, H., & Lindström, E. (2016).

Long memory of financial time series and hidden Markov models with time-varying parameters. Journal

• f Forecasting DOI: 10.1002/for.2447

Cont.

◮ Lindström, E., Ströjby, J., Brodén, M., Wiktorsson,

M., & Holst, J. (2008). Sequential calibration of

• ptions. Computational Statistics & Data Analysis,

52(6), 2877-2891.

◮ Lindström, E., & Gou, J. (2013). Simultaneous

calibration and quadratic hedging of options. Quantitative and Qualitative Analysis in Social Sciences

◮ Lindström, E., & Åkerlindh, C. (2018). Optimal

Adaptive Sequential Calibration of Option Models. In Handbook of Recent Advances in Commodity and Financial Modeling (pp. 165-181). Springer,