Exponential smoothing and non-negative data Muhammad Akram Rob J - - PowerPoint PPT Presentation

Exponential smoothing and non-negative data 1

Exponential smoothing and non-negative data

Muhammad Akram Rob J Hyndman J Keith Ord Business & Economic Forecasting Unit

Exponential smoothing and non-negative data 2

Outline

1

Exponential smoothing models

2

Problems with some of the models

3

A new model for positive data

4

Conclusions

Exponential smoothing and non-negative data Exponential smoothing models 3

Outline

1

Exponential smoothing models

2

Problems with some of the models

3

A new model for positive data

4

Conclusions

Exponential smoothing and non-negative data Exponential smoothing models 4

Problem

Most forecasting methods in business are based on exponential smoothing.

Exponential smoothing and non-negative data Exponential smoothing models 4

Problem

Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative.

Exponential smoothing and non-negative data Exponential smoothing models 4

Problem

Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative. But exponential smoothing models for non-negative data don’t always work!

Exponential smoothing and non-negative data Exponential smoothing models 4

Problem

Forecasts from ETS(A,A,N)

Time 5 10 15 20 25 30 35 10 20 30

1

They can produce negative forecasts

Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative. But exponential smoothing models for non-negative data don’t always work!

Exponential smoothing and non-negative data Exponential smoothing models 4

Problem

Forecasts from ETS(A,M,N)

Time 5 10 15 20 25 30 −5 5 10

1

They can produce negative forecasts

2

They can produce infinite forecast variance

Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative. But exponential smoothing models for non-negative data don’t always work!

Exponential smoothing and non-negative data Exponential smoothing models 4

Problem

Simulation from ETS(M,N,N)

y 500 1000 1500 2000 10 20 30 40

1

They can produce negative forecasts

2

They can produce infinite forecast variance

3

They can converge almost surely to zero.

Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative. But exponential smoothing models for non-negative data don’t always work!

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

General notation ETS(Error,Trend,Seasonal)

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(A,N,N): Simple exponential smoothing with additive errors

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(A,A,N): Holt’s linear method with additive errors

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(A,A,A): Additive Holt-Winters’ method with additive errors

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(M,A,M): Multiplicative Holt-Winters’ method with multiplicative errors

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(A,Ad,N): Damped trend method with addi- tive errors

Exponential smoothing and non-negative data Exponential smoothing models 5

Taxonomy of models

Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M Ad (Additive damped) Ad,N Ad,A Ad,M M (Multiplicative) M,N M,A M,M Md (Multiplicative damped) Md,N Md,A Md,M

General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing There are 30 separate models in the ETS framework

Exponential smoothing and non-negative data Exponential smoothing models 6

Innovations state space model

No trend or seasonality and multiplicative errors Example: ETS(M,N,N) yt = ℓt−1(1 + εt)

ℓt = αyt + (1 − α)ℓt−1

0 ≤ α ≤ 1

εt is white noise with mean zero.

Exponential smoothing and non-negative data Exponential smoothing models 6

Innovations state space model

No trend or seasonality and multiplicative errors Example: ETS(M,N,N) yt = ℓt−1(1 + εt)

ℓt = ℓt−1(1 + αεt)

0 ≤ α ≤ 1

εt is white noise with mean zero.

Exponential smoothing and non-negative data Exponential smoothing models 6

Innovations state space model

No trend or seasonality and multiplicative errors All exponential smoothing models can be written using analogous state space equations. Example: ETS(M,N,N) yt = ℓt−1(1 + εt)

ℓt = ℓt−1(1 + αεt)

0 ≤ α ≤ 1

εt is white noise with mean zero.

Exponential smoothing and non-negative data Exponential smoothing models 7

New book!

State space modeling framework Prediction intervals Model selection Maximum likelihood estimation All the important research results in one place with consistent notation Many new results 375 pages but only US$39.95.

1฀3

Springer Series in Statistics

Rob J.Hyndman · Anne B.Koehler J.Keith Ord · Ralph D.Snyder

Forecasting with Exponential Smoothing

The State Space Approach 1

Exponential smoothing and non-negative data Exponential smoothing models 7

New book!

State space modeling framework Prediction intervals Model selection Maximum likelihood estimation All the important research results in one place with consistent notation Many new results 375 pages but only US$39.95.

1฀3

Springer Series in Statistics

Rob J.Hyndman · Anne B.Koehler J.Keith Ord · Ralph D.Snyder

Forecasting with Exponential Smoothing

The State Space Approach 1

www.exponentialsmoothing.net

Exponential smoothing and non-negative data Problems with some of the models 8

Outline

1

Exponential smoothing models

2

Problems with some of the models

3

A new model for positive data

4

Conclusions

Exponential smoothing and non-negative data Problems with some of the models 9

Negative forecasts

Could solve by taking logs or some other Box-Cox

• transformation. However, this limits models to be

additive in the transformed space.

Forecasts from ETS(A,A,N)

Time 5 10 15 20 25 30 35 10 20 30

Exponential smoothing and non-negative data Problems with some of the models 9

Negative forecasts

Could solve by taking logs or some other Box-Cox

• transformation. However, this limits models to be

additive in the transformed space. Could solve by only using multiplicative models. But these can have other problems.

Forecasts from ETS(A,A,N)

Time 5 10 15 20 25 30 35 10 20 30

Exponential smoothing and non-negative data Problems with some of the models 10

Infinite forecast variances

ETS(A,M,N) model yt = ℓt−1bt−1 + εt

ℓt = ℓt−1bt−1 + αεt

bt = bt−1 + βεt/ℓt−1.

ℓ0 = 0.1

b0 = 1

α = 0.1 β = 0.05 σ = 1

Forecasts from ETS(A,M,N)

5 10 15 20 25 30 −5 5 10

Exponential smoothing and non-negative data Problems with some of the models 10

Infinite forecast variances

ETS(A,M,N) model yt = ℓt−1bt−1 + εt

ℓt = ℓt−1bt−1 + αεt

bt = bt−1 + βεt/ℓt−1.

ℓ0 = 0.1

b0 = 1

α = 0.1 β = 0.05 σ = 1

Time y 2 4 6 8 −400 200 400 Time ell 2 4 6 8 −1.0 −0.5 0.0 0.5 1.0 Time b 2 4 6 8 −15 −10 −5

Exponential smoothing and non-negative data Problems with some of the models 11

Infinite forecast variances

Suppose εt has positive density at 0 For ETS models (A,M,N), (A,M,A), (A,Md,N), (A,Md,A), (A,M,M), (A,Md,M), (M,M,A) and (M,Md,A): V(yn+h | xn) = ∞ for h ≥ 3; E(yn+h | xn) is undefined for h ≥ 3.

Exponential smoothing and non-negative data Problems with some of the models 11

Infinite forecast variances

Suppose εt has positive density at 0 For ETS models (A,M,N), (A,M,A), (A,Md,N), (A,Md,A), (A,M,M), (A,Md,M), (M,M,A) and (M,Md,A): V(yn+h | xn) = ∞ for h ≥ 3; E(yn+h | xn) is undefined for h ≥ 3. For ETS models (A,N,M), (A,A,M) and (A,Ad,M): V(yn+h | xn) = ∞ for h ≥ m + 2; E(yn+h | xn) is undefined for h ≥ m + 2.

Exponential smoothing and non-negative data Problems with some of the models 11

Infinite forecast variances

Suppose εt has positive density at 0 For ETS models (A,M,N), (A,M,A), (A,Md,N), (A,Md,A), (A,M,M), (A,Md,M), (M,M,A) and (M,Md,A): V(yn+h | xn) = ∞ for h ≥ 3; E(yn+h | xn) is undefined for h ≥ 3. For ETS models (A,N,M), (A,A,M) and (A,Ad,M): V(yn+h | xn) = ∞ for h ≥ m + 2; E(yn+h | xn) is undefined for h ≥ m + 2.

➥ These problems occur regardless of

the sample space of {yt}.

Exponential smoothing and non-negative data Problems with some of the models 12

Convergence to zero

ETS(M,N,N) model yt = ℓt−1(1 + εt)

ℓt = ℓt−1(1 + αεt) ℓ0 = 10 α = 0.3 σ = 0.3 with

truncated Gaussian errors

Time y 500 1000 1500 2000 10 20 30 40 Time y 500 1000 1500 2000 5 10 15 Time y 500 1000 1500 2000 5 10 15 20

Exponential smoothing and non-negative data Problems with some of the models 13

Convergence to zero

ETS(M,N,N) model yt = ℓt−1(1 + εt)

ℓt = ℓt−1(1 + αεt),

0 < α ≤ 1

Exponential smoothing and non-negative data Problems with some of the models 13

Convergence to zero

ETS(M,N,N) model yt = ℓt−1(1 + εt)

ℓt = ℓt−1(1 + αεt),

0 < α ≤ 1

δt = 1 + εt has mean 1 and variance σ2.

Exponential smoothing and non-negative data Problems with some of the models 13

Convergence to zero

ETS(M,N,N) model yt = ℓt−1(1 + εt)

ℓt = ℓt−1(1 + αεt),

0 < α ≤ 1

δt = 1 + εt has mean 1 and variance σ2. δt are iid with positive distribution such as

truncated normal, lognormal, gamma, etc.

Exponential smoothing and non-negative data Problems with some of the models 13

Convergence to zero

ETS(M,N,N) model yt = ℓt−1(1 + εt)

ℓt = ℓt−1(1 + αεt),

0 < α ≤ 1

δt = 1 + εt has mean 1 and variance σ2. δt are iid with positive distribution such as

truncated normal, lognormal, gamma, etc.

Exponential smoothing and non-negative data Problems with some of the models 13

Convergence to zero

ETS(M,N,N) model yt = ℓt−1(1 + εt)

ℓt = ℓt−1(1 + αεt),

0 < α ≤ 1

δt = 1 + εt has mean 1 and variance σ2. δt are iid with positive distribution such as

truncated normal, lognormal, gamma, etc.

ℓt = ℓ0(1 + αε1)(1 + αε2) · · · (1 + αεt) = ℓ0Ut,

where Ut = Ut−1(1 + αεt) and U0 = 1. Thus Ut is a non-negative product martingale.

Exponential smoothing and non-negative data Problems with some of the models 14

Convergence to zero

Kakutani’s Theorem says that ℓt will converge to 0 almost surely if εt has mean zero and is non-degenerate.

Exponential smoothing and non-negative data Problems with some of the models 14

Convergence to zero

Kakutani’s Theorem says that ℓt will converge to 0 almost surely if εt has mean zero and is non-degenerate. Consequently, all sample paths for yt converge to 0 almost surely.

Exponential smoothing and non-negative data Problems with some of the models 14

Convergence to zero

Kakutani’s Theorem says that ℓt will converge to 0 almost surely if εt has mean zero and is non-degenerate. Consequently, all sample paths for yt converge to 0 almost surely. Similar results follow for all purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,Md,N) and (M,Md,M).

Exponential smoothing and non-negative data Problems with some of the models 15

Four model classes

Class M: Purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,Md,N) and (M,Md,M). Class A: Purely additive models: (A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,Ad,N) and (A,Ad,A). Class X: Mixed models: (A,M,∗), (A,Md,∗), (A,∗,M), (M,M,A), (M,Md,A). (11 models) Class Y: Mixed models: (M,A,∗), (M,Ad,∗) or (M,N,A). (7 models)

Exponential smoothing and non-negative data Problems with some of the models 15

Four model classes

Class M: Purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,Md,N) and (M,Md,M). Class A: Purely additive models: (A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,Ad,N) and (A,Ad,A). Class X: Mixed models: (A,M,∗), (A,Md,∗), (A,∗,M), (M,M,A), (M,Md,A). (11 models) Class Y: Mixed models: (M,A,∗), (M,Ad,∗) or (M,N,A). (7 models) Only Class M can guarantee a sample space restricted to the positive half-line.

Exponential smoothing and non-negative data Problems with some of the models 15

Four model classes

Class M: Purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,Md,N) and (M,Md,M). Class A: Purely additive models: (A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,Ad,N) and (A,Ad,A). Class X: Mixed models: (A,M,∗), (A,Md,∗), (A,∗,M), (M,M,A), (M,Md,A). (11 models) Class Y: Mixed models: (M,A,∗), (M,Ad,∗) or (M,N,A). (7 models) Only Class M can guarantee a sample space restricted to the positive half-line. All Class M models converge to 0 if E(ε) = 0

Exponential smoothing and non-negative data Problems with some of the models 15

Four model classes

Class M: Purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,Md,N) and (M,Md,M). Class A: Purely additive models: (A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,Ad,N) and (A,Ad,A). Class X: Mixed models: (A,M,∗), (A,Md,∗), (A,∗,M), (M,M,A), (M,Md,A). (11 models) Class Y: Mixed models: (M,A,∗), (M,Ad,∗) or (M,N,A). (7 models) Only Class M can guarantee a sample space restricted to the positive half-line. All Class M models converge to 0 if E(ε) = 0 All Class X models have infinite forecast variance for h ≥ m + 2 where m is the seasonal period.

Exponential smoothing and non-negative data A new model for positive data 16

Outline

1

Exponential smoothing models

2

Problems with some of the models

3

A new model for positive data

4

Conclusions

Exponential smoothing and non-negative data A new model for positive data 17

New models

Let δt = (1 + εt) be a positive random variable. METS(M,N,N) model yt = ℓt−1δt

ℓt = ℓt−1δα

t

Exponential smoothing and non-negative data A new model for positive data 17

New models

Let δt = (1 + εt) be a positive random variable. METS(M,N,N) model yt = ℓt−1δt

ℓt = ℓt−1δα

t

log(yt) = log(ℓt−1) + log(δt) log(ℓt) = log(ℓt−1) + α log(δt)

Exponential smoothing and non-negative data A new model for positive data 17

New models

Let δt = (1 + εt) be a positive random variable. METS(M,N,N) model yt = ℓt−1δt

ℓt = ℓt−1δα

t

log(yt) = log(ℓt−1) + log(δt) log(ℓt) = log(ℓt−1) + α log(δt) Thus the log-transformed model is identical to Gaussian ETS(A,N,N) model if δt is logNormal with median 1.

Exponential smoothing and non-negative data A new model for positive data 18

Long term forecast behaviour

δt ∼ logN(µ, ω)

METS(M,N,N; LN) model yt = ℓt−1δt

ℓt = ℓt−1δα

t ,

Exponential smoothing and non-negative data A new model for positive data 18

Long term forecast behaviour

Range E(δα

t )

E(δα/2

t

)

E(yh) V(yh)

µ + αω < 0 < 1 < 1

Decreasing Decreasing

µ + αω = 0 < 1 < 1

Decreasing Finite

−αω < µ < −αω/2 < 1 < 1

Decreasing Increasing

µ + αω/2 = 0 = 1 < 1

Finite Increasing

−αω/2 < µ < −αω/4 > 1 < 1

Increasing Increasing

µ + αω/4 = 0 > 1 = 1

Increasing Increasing

µ + αω/4 > 0 > 1 > 1

Increasing Increasing

δt ∼ logN(µ, ω)

METS(M,N,N; LN) model yt = ℓt−1δt

ℓt = ℓt−1δα

t ,

Exponential smoothing and non-negative data A new model for positive data 19

Long term forecast behaviour

METS(M,N,N;LN)

δt ∼ logN(µ, ω):

(a) µ = αω/4 (b) µ = 0 (c) µ = −αω/4 (d) µ = −3αω/8 (e) µ = −αω/2 (f) µ = −3αω/4

t (a) 500 1000 1500 2000 2 4 6 8 10 12 14 t (b) 500 1000 1500 2000 0.5 0.7 0.9 1.1 t (c) 500 1000 1500 2000 0.2 0.4 0.6 0.8 1.0 t (d) 500 1000 1500 2000 0.0 0.2 0.4 0.6 0.8 1.0 t (e) 500 1000 1500 2000 0.0 0.2 0.4 0.6 0.8 1.0 t (f) 500 1000 1500 2000 0.0 0.2 0.4 0.6 0.8 1.0

Exponential smoothing and non-negative data Conclusions 20

Outline

1

Exponential smoothing models

2

Problems with some of the models

3

A new model for positive data

4

Conclusions

Exponential smoothing and non-negative data Conclusions 21

Conclusions

1

The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line.

Exponential smoothing and non-negative data Conclusions 21

Conclusions

1

The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line.

2

This is not a concern for most short-term forecasting.

Exponential smoothing and non-negative data Conclusions 21

Conclusions

1

The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line.

2

This is not a concern for most short-term forecasting.

3

An alternative formulation has been proposed that avoids these problems.

Exponential smoothing and non-negative data Conclusions 21

Conclusions

1

The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line.

2

This is not a concern for most short-term forecasting.

3

An alternative formulation has been proposed that avoids these problems.

4

The ergodic behaviour of the alternative formulation requires careful parameter choice.

Exponential smoothing and non-negative data Conclusions 21

Conclusions

1

The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line.

2

This is not a concern for most short-term forecasting.

3

An alternative formulation has been proposed that avoids these problems.

4

The ergodic behaviour of the alternative formulation requires careful parameter choice.

Exponential smoothing and non-negative data Conclusions 21

Conclusions

1

The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line.

2

This is not a concern for most short-term forecasting.

3

An alternative formulation has been proposed that avoids these problems.

4

The ergodic behaviour of the alternative formulation requires careful parameter choice. Paper: www.robhyndman.info

Exponential smoothing and non-negative data Conclusions 21

Conclusions

1

The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line.

2

This is not a concern for most short-term forecasting.

3

An alternative formulation has been proposed that avoids these problems.

4

The ergodic behaviour of the alternative formulation requires careful parameter choice. Paper: www.robhyndman.info Book: www.exponentialsmoothing.net