Combining Predictive Densities using a Bayesian Nonlinear Filtering - - PowerPoint PPT Presentation

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combining Predictive Densities using a Bayesian Nonlinear Filtering Approach

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Herman K. van Dijk Norges Bank Erasmus University Rotterdam Luxembourg, September 28, 2010

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Outline

Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Motivations:

• Combining densities in a multivariate setting.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Motivations:

• Combining densities in a multivariate setting.
• Random and time-varying weights.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Motivations:

• Combining densities in a multivariate setting.
• Random and time-varying weights.
• Parameters and model uncertainty are taken into account.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Motivations:

• Combining densities in a multivariate setting.
• Random and time-varying weights.
• Parameters and model uncertainty are taken into account.
• Applications to Macro and Finance.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Previous Works: BMA (General)

• Barnes (1963): the first mention of model combination.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Previous Works: BMA (General)

• Barnes (1963): the first mention of model combination.
• Roberts (1965): obtained a distribution which includes the

predictions from two experts (or models). This distribution is essentially a weighted average of the posterior distributions of two

• models. This is similar to a Bayesian Model Averaging (BMA)

procedure.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Previous Works: BMA (General)

• Barnes (1963): the first mention of model combination.
• Roberts (1965): obtained a distribution which includes the

predictions from two experts (or models). This distribution is essentially a weighted average of the posterior distributions of two

• models. This is similar to a Bayesian Model Averaging (BMA)

procedure.

• Bates and Granger (1969): seminal paper about combining

predictions from different forecasting models.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Previous Works: BMA (General)

• Barnes (1963): the first mention of model combination.
• Roberts (1965): obtained a distribution which includes the

predictions from two experts (or models). This distribution is essentially a weighted average of the posterior distributions of two

• models. This is similar to a Bayesian Model Averaging (BMA)

procedure.

• Bates and Granger (1969): seminal paper about combining

predictions from different forecasting models.

• Useful reviews: Hoeting et al. (1999) (on BMA with historical

perspective), Granger (2006) and Timmermann (2006) (forecasts combination).

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Previous Works: BMA (State-space models)

• Granger and Ramanathan (1984): combine the forecasts with

unrestricted regression coefficients as weights;

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Previous Works: BMA (State-space models)

• Granger and Ramanathan (1984): combine the forecasts with

unrestricted regression coefficients as weights;

• Terui and Van Dijk (2002): generalize the least squares model

weights by representing the dynamic forecast combination as a state space. In their work the weights are assumed to follow a random walk process;

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Previous Works: BMA (State-space models)

• Granger and Ramanathan (1984): combine the forecasts with

unrestricted regression coefficients as weights;

• Terui and Van Dijk (2002): generalize the least squares model

weights by representing the dynamic forecast combination as a state space. In their work the weights are assumed to follow a random walk process;

• Guidolin and Timmermann (2009): introduced Markov-switching

weights;

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Previous Works: BMA (State-space models)

• Granger and Ramanathan (1984): combine the forecasts with

unrestricted regression coefficients as weights;

• Terui and Van Dijk (2002): generalize the least squares model

weights by representing the dynamic forecast combination as a state space. In their work the weights are assumed to follow a random walk process;

• Guidolin and Timmermann (2009): introduced Markov-switching

weights;

• Hoogerheide et al. (2010) and Groen et al. (2009): robust

time-varying weights and accounting for both model and parameter uncertainty in model averaging.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Our contributions: non-linear combination of densities

We extend the state-space representation of Terui and Van Dijk (2002) and Hoogerheide et al. (2010):

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Our contributions: non-linear combination of densities

We extend the state-space representation of Terui and Van Dijk (2002) and Hoogerheide et al. (2010):

• We assume time-varying (and logistic-transformed) weights and

propose a distributional state-space representation of the predictive densities and of the combination scheme.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Our contributions: non-linear combination of densities

We extend the state-space representation of Terui and Van Dijk (2002) and Hoogerheide et al. (2010):

• We assume time-varying (and logistic-transformed) weights and

propose a distributional state-space representation of the predictive densities and of the combination scheme.

• This representation (see Harrison and West (1987) for a review).

It is general enough to include: linear and Gaussian models, dynamic mixtures and Markov-switching models, as special cases.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Our contributions: non-linear combination of densities

We extend the state-space representation of Terui and Van Dijk (2002) and Hoogerheide et al. (2010):

• We assume time-varying (and logistic-transformed) weights and

propose a distributional state-space representation of the predictive densities and of the combination scheme.

• This representation (see Harrison and West (1987) for a review).

It is general enough to include: linear and Gaussian models, dynamic mixtures and Markov-switching models, as special cases.

• We apply to the context of combining forecast simulation based

filtering methods (Sequential Monte Carlo (SMC)).

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Our contribution: combination of multivariate densities

Extension of the univariate model in Hoogerheide et al. (2010):

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Our contribution: combination of multivariate densities

Extension of the univariate model in Hoogerheide et al. (2010):

• We consider convex combinations of the predictive densities (the

time-varying weights associated to the different forecasts densities belong to the standard simplex);

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Our contribution: combination of multivariate densities

Extension of the univariate model in Hoogerheide et al. (2010):

• We consider convex combinations of the predictive densities (the

time-varying weights associated to the different forecasts densities belong to the standard simplex);

• Multivariate combination schemes: continuous and

Markov-switching dynamics in a convex combination setting;

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Our contribution: combination of multivariate densities

Extension of the univariate model in Hoogerheide et al. (2010):

• We consider convex combinations of the predictive densities (the

time-varying weights associated to the different forecasts densities belong to the standard simplex);

• Multivariate combination schemes: continuous and

Markov-switching dynamics in a convex combination setting;

• Learning: weights dynamics driven by the past performance of

the model and the current performances of the other models (we extend Diebold and Pauly (1987)).

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Notation

• yt ∈ Y ⊂ RL: vector of observable variables;

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Notation

• yt ∈ Y ⊂ RL: vector of observable variables;
• yt ∼ p(yt|y1:t−1): observable conditional density;

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Notation

• yt ∈ Y ⊂ RL: vector of observable variables;
• yt ∼ p(yt|y1:t−1): observable conditional density;
• ˜

yk,t ∈ Y ⊂ RL, with k = 1, . . . , K: a set of one-step-ahead predictors for yt. (The combination methodology easily extends to multi-step-ahead predictors).

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Notation

• yt ∈ Y ⊂ RL: vector of observable variables;
• yt ∼ p(yt|y1:t−1): observable conditional density;
• ˜

yk,t ∈ Y ⊂ RL, with k = 1, . . . , K: a set of one-step-ahead predictors for yt. (The combination methodology easily extends to multi-step-ahead predictors).

• ˜

yk,t ∼ p(˜ yk,t|y1:t−1), k = 1, . . . , K: prediction densities.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Notation

• yt ∈ Y ⊂ RL: vector of observable variables;
• yt ∼ p(yt|y1:t−1): observable conditional density;
• ˜

yk,t ∈ Y ⊂ RL, with k = 1, . . . , K: a set of one-step-ahead predictors for yt. (The combination methodology easily extends to multi-step-ahead predictors).

• ˜

yk,t ∼ p(˜ yk,t|y1:t−1), k = 1, . . . , K: prediction densities.

• ˜

yt = vec( ˜ Y ′

t), where ˜

Yt = (˜ y1,t, . . . , ˜ yK,t).

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (a general representation)

Combination scheme: a probabilistic relation between the density

• f the observable variable and the predictive densities:

p(yt|y1:t−1) =

• YKt p(yt|˜

y1:t, y1:t−1)p(˜ y1:t|y1:t−1)d˜ y1:t (Conditional dependence structure between yt and ˜ y1:t: not defined yet).

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (the latent space)

• 1n = (1, . . . , 1)′ ∈ Rn, 0n = (0, . . . , 0)′ ∈ Rn

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (the latent space)

• 1n = (1, . . . , 1)′ ∈ Rn, 0n = (0, . . . , 0)′ ∈ Rn
• ∆[0,1]n ⊂ Rn: the set of w ∈ Rn s.t. w′1n = 1 and wk ≥ 0,

k = 1, . . . , n. ∆[0,1]n is called the standard n-dimensional simplex and is the latent space.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (the latent space)

• 1n = (1, . . . , 1)′ ∈ Rn, 0n = (0, . . . , 0)′ ∈ Rn
• ∆[0,1]n ⊂ Rn: the set of w ∈ Rn s.t. w′1n = 1 and wk ≥ 0,

k = 1, . . . , n. ∆[0,1]n is called the standard n-dimensional simplex and is the latent space.

• Wt ∈ W ⊂ RL × RKL: time-varying weights of the combination
• scheme. Denote with wl

k,t the k-column and l-row elements of Wt,

wl

t = (wl 1,t, . . . , wl KL,t)′ s.t. wl t ∈ ∆[0,1]K

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (the latent space)

• 1n = (1, . . . , 1)′ ∈ Rn, 0n = (0, . . . , 0)′ ∈ Rn
• ∆[0,1]n ⊂ Rn: the set of w ∈ Rn s.t. w′1n = 1 and wk ≥ 0,

k = 1, . . . , n. ∆[0,1]n is called the standard n-dimensional simplex and is the latent space.

• Wt ∈ W ⊂ RL × RKL: time-varying weights of the combination
• scheme. Denote with wl

k,t the k-column and l-row elements of Wt,

wl

t = (wl 1,t, . . . , wl KL,t)′ s.t. wl t ∈ ∆[0,1]K

Latent space: the time series of [0, 1] weights

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (the latent space)

• 1n = (1, . . . , 1)′ ∈ Rn, 0n = (0, . . . , 0)′ ∈ Rn
• ∆[0,1]n ⊂ Rn: the set of w ∈ Rn s.t. w′1n = 1 and wk ≥ 0,

k = 1, . . . , n. ∆[0,1]n is called the standard n-dimensional simplex and is the latent space.

• Wt ∈ W ⊂ RL × RKL: time-varying weights of the combination
• scheme. Denote with wl

k,t the k-column and l-row elements of Wt,

wl

t = (wl 1,t, . . . , wl KL,t)′ s.t. wl t ∈ ∆[0,1]K

Latent space: the time series of [0, 1] weights Weights: interpreted as a discrete p.d.f. over the set of predictors.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (the latent space)

• 1n = (1, . . . , 1)′ ∈ Rn, 0n = (0, . . . , 0)′ ∈ Rn
• ∆[0,1]n ⊂ Rn: the set of w ∈ Rn s.t. w′1n = 1 and wk ≥ 0,

k = 1, . . . , n. ∆[0,1]n is called the standard n-dimensional simplex and is the latent space.

• Wt ∈ W ⊂ RL × RKL: time-varying weights of the combination
• scheme. Denote with wl

k,t the k-column and l-row elements of Wt,

wl

t = (wl 1,t, . . . , wl KL,t)′ s.t. wl t ∈ ∆[0,1]K

Latent space: the time series of [0, 1] weights Weights: interpreted as a discrete p.d.f. over the set of predictors. Generalizes: for example Hoogerheide et al. (2010)

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (weight dynamics)

Let Wt ∼ p(Wt|Wt−1, ˜ yt−τ:t−1) be the density of the time-varying weights, then p(yt|y1:t−1) can be written as

• YKt
• W

p(yt|Wt, ˜ yt)p(Wt|y1:t−1, ˜ y1:t−1)dWt

• p(˜

y1:t|y1:t−1)d˜ y1:t where p(Wt|y1:t−1, ˜ y1:t−1) =

• W

p(Wt|Wt−1, ˜ yt−τ:t−1)p(Wt−1|y1:t−2, ˜ y1:t−2)dWt−1

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (non-linear filtering)

The conditional density p(yt|yt−1) can be approximated as follows.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (non-linear filtering)

The conditional density p(yt|yt−1) can be approximated as follows.

• First, draw j independent values yj

1:t+1, with j = 1, . . . , M from

p(˜ ys+1|y1:s), with s = 1, . . . , t.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (non-linear filtering)

The conditional density p(yt|yt−1) can be approximated as follows.

• First, draw j independent values yj

1:t+1, with j = 1, . . . , M from

p(˜ ys+1|y1:s), with s = 1, . . . , t.

• Conditionally on ˜

yj

1:t+1 obtain the particle sets

Ξi,j

1:t+1 = {zi,j 1:t+1, ωi,j t }N i=1, with j = 1, . . . , M.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (non-linear filtering)

The conditional density p(yt|yt−1) can be approximated as follows.

• First, draw j independent values yj

1:t+1, with j = 1, . . . , M from

p(˜ ys+1|y1:s), with s = 1, . . . , t.

• Conditionally on ˜

yj

1:t+1 obtain the particle sets

Ξi,j

1:t+1 = {zi,j 1:t+1, ωi,j t }N i=1, with j = 1, . . . , M.

• Simulate yi,j

t+1 from p(yt+1|zi,j t+1, ˜

yj

t+1) and obtain

pN,M(yt+1|y1:t) = 1 M

M

• j=1

N

• i=1

ωi,j

t δyi,j

t+1(yt+1) Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (Example)

Gaussian combination, Logistic-Gaussian Weights p(yt|Wt, ˜ yt) ∝ exp

• −1

2 (yt − Wt˜ yt)′ Σ−1 (yt − Wt˜ yt)

• where the weights are logistic transforms

wl

k,t =

exp{xl

k}

KL

j=1 exp{xl j }

, with k = 1, . . . , KL with l = 1, . . . , L of the latent process xt, which has transition p(xt|xt−1) ∝ exp

• −1

2 (xt − xt−1)′ Λ−1 (xt − xt−1)

• Monica Billio

Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (alternatives)

Dirichlet Autoregressive Dynamics Set wl

t = xl t and

p(xl

t|xt−1, ˜

y1:t−1) ∝ DKL

• ηl

1,tφ, . . . , ηl KL,tφ

• with φ > 0 (precision parameter) and ηl

t = g(xl t−1) (mean).

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (alternatives)

Dirichlet Autoregressive Dynamics Set wl

t = xl t and

p(xl

t|xt−1, ˜

y1:t−1) ∝ DKL

• ηl

1,tφ, . . . , ηl KL,tφ

• with φ > 0 (precision parameter) and ηl

t = g(xl t−1) (mean).

• Advantages: naturally defined on the simplex (Grunwald et al.

(1993)); conditional mean and variance easy to find.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (alternatives)

Dirichlet Autoregressive Dynamics Set wl

t = xl t and

p(xl

t|xt−1, ˜

y1:t−1) ∝ DKL

• ηl

1,tφ, . . . , ηl KL,tφ

• with φ > 0 (precision parameter) and ηl

t = g(xl t−1) (mean).

• Advantages: naturally defined on the simplex (Grunwald et al.

(1993)); conditional mean and variance easy to find.

• Drawbacks: negative correlation; difficult modeling dependence

between the observable and the weights (future res.).

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (alternatives)

Logistic-Gaussian Dynamics with Learning p(xt|xt−1, ˜ y1:t−1)∝exp

• −1

2 (∆xt + ∆et)′ Λ−1 (∆xt + ∆et)

• where et = vec(Et), with the elements of et defined by

el,d

k,t = (1 − λ) τ

• i=1

λi−1(y l

t−i −

y l,d

k,t−i)2

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (alternatives)

Logistic-Gaussian Dynamics with Learning p(xt|xt−1, ˜ y1:t−1)∝exp

• −1

2 (∆xt + ∆et)′ Λ−1 (∆xt + ∆et)

• where et = vec(Et), with the elements of et defined by

el,d

k,t = (1 − λ) τ

• i=1

λi−1(y l

t−i −

y l,d

k,t−i)2

• We do not choose between learning and time-varying weights

(Diebold and Pauly (1987), Timmermann (2006)), but combine the two approaches.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (alternatives)

Markov-switching Logistic-Gaussian Dynamics p(xl

t|xl t, ξt, ˜

y1:t−1) ∝ exp

• −1

2

• ∆xl

t + ∆el t−1

′ Λ−1

ξt

• ∆xl

t + ∆el t−1

′ Λξt = γ(ξl,t)Λ + (1 − γ(ξl,t))IK ξl,t ∼ P(ξl,t = i|ξl,t−1 = j) = pij, ∀i, j ∈ {0, 1}

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Combination of Densities (alternatives)

Markov-switching Logistic-Gaussian Dynamics p(xl

t|xl t, ξt, ˜

y1:t−1) ∝ exp

• −1

2

• ∆xl

t + ∆el t−1

′ Λ−1

ξt

• ∆xl

t + ∆el t−1

′ Λξt = γ(ξl,t)Λ + (1 − γ(ξl,t))IK ξl,t ∼ P(ξl,t = i|ξl,t−1 = j) = pij, ∀i, j ∈ {0, 1}

• Markov-switching accounts for the discontinuity (e.g. due to

structural breaks) in the data generating process, which calls for a sudden variation of the current combination scheme.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Empirical Results (the data)

• Variables: GDP and inflation measured as PCE deflator.
• Source: Bureau of Economic Analysis.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Empirical Results (the data)

• Variables: GDP and inflation measured as PCE deflator.
• Source: Bureau of Economic Analysis.
• Sample: 1960Q1 - 2009Q4.
• Forecasting: 1-step ahead 1980Q1 - 2009Q4.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Empirical Results (the data)

• Variables: GDP and inflation measured as PCE deflator.
• Source: Bureau of Economic Analysis.
• Sample: 1960Q1 - 2009Q4.
• Forecasting: 1-step ahead 1980Q1 - 2009Q4.
• Point and density forecasting.
• Individual models: AR and VAR, (2-state) MS AR and VAR.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Empirical Results (the data)

• Variables: GDP and inflation measured as PCE deflator.
• Source: Bureau of Economic Analysis.
• Sample: 1960Q1 - 2009Q4.
• Forecasting: 1-step ahead 1980Q1 - 2009Q4.
• Point and density forecasting.
• Individual models: AR and VAR, (2-state) MS AR and VAR.
• BMA: constant weight (KLIC).
• TVW: time variation.
• TVW(λ, τ): learning with (λ = 0.95, τ = 9)

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Univariate Results (GDP)

AR VAR ARMS VARMS BMA TVW TVW(λ, τ) RMSPE 0.882 0.875 0.907 1.000 0.885 0.799 0.691 CW 1.625 1.274 1.587

• 0.103

7.185 7.984 LS

• 1.323
• 1.381
• 1.403
• 1.361
• 2.791
• 1.146
• 1.151

p 0.337 0.003 0.008 0.001 0.016 0.020 PITS 0.480 0.467 0.472 0.523 0.316 0.468 0.851

Table: TVW : time-varying weights without learning. TVW(λ, τ): time-varying weights with learning mechanism (smoothness parameter λ = 0.95 and window size τ = 9.)

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Univariate Results

λ τ 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 2 4 6 8 10 12 14 16 18 20

−0.34 −0.335 −0.33 −0.325 −0.32 −0.315

Root mean square prediction error (RMSPE), in logarithmic scale, of the TVW(λ, τ)

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Univariate Results

20 40 60 80 100 120 140 160 0.1 0.2 0.3 0.4 wit AR ARMS VAR VARMS 20 40 60 80 100 120 140 160 2 4 p(yt|y1:t−1)

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Univariate Results

50 100 150 0.5 1 wARt 50 100 150 0.5 1 wARMSt 50 100 150 0.5 1 wVARt 50 100 150 0.5 1 wVARMSt

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Multivariate Results

AR VAR ARMS VARMS BMA TVW(λ, τ) GDP RMSPE 0.882 0.875 0.907 1.000 0.885 0.718 CW 1.625 1.274 1.587

• 0.103

8.554 LS

• 1.323
• 1.381
• 1.403
• 1.361
• 2.791
• 1.012

(p-value) 0.337 0.003 0.008 0.001 0.015 PITS 0.038 0.098 0.164 0.000 0.316 0.958 PCE RMSPE 0.385 0.384 0.384 0.612 0.382 0.307 CW 1.036 1.902 1.476 1.234 6.715 LS

• 1.538
• 1.267
• 1.373
• 1.090
• 1.759
• 0.538

(p-value) 0.008 0.024 0.007 0.020 0.024 PITS 0.001 0.000 0.000 0.000 0.000 0.095

Table: Upper table: GDP. Bottom table: PCE.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Multivariate Results

50 100 150 0.1 0.2 0.3 0.4 wit for GDP AR1 ARMS1 VAR1 VARMS1 50 100 150 0.2 0.4 wit for PCE AR2 ARMS2 VAR2 VARMS2

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Conclusions

• A new general combination approach of multivariate predictives.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Conclusions

• A new general combination approach of multivariate predictives.
• Distributional state-space representation of the combination

scheme.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Conclusions

• A new general combination approach of multivariate predictives.
• Distributional state-space representation of the combination

scheme.

• Nonlinear Bayesian filtering for the optimal weights estimation.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Conclusions

• A new general combination approach of multivariate predictives.
• Distributional state-space representation of the combination

scheme.

• Nonlinear Bayesian filtering for the optimal weights estimation.
• Applications to macroeconomics (GDP and PCE) and finance

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Conclusions

• A new general combination approach of multivariate predictives.
• Distributional state-space representation of the combination

scheme.

• Nonlinear Bayesian filtering for the optimal weights estimation.
• Applications to macroeconomics (GDP and PCE) and finance
• Nonlinear combinations outperform (RMSPE and KLIC) the

constant and the linear time-varying weighting.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte

Outline Motivation and Previous Works Contributions of our work The Methodology Empirical Results

Conclusions

• A new general combination approach of multivariate predictives.
• Distributional state-space representation of the combination

scheme.

• Nonlinear Bayesian filtering for the optimal weights estimation.
• Applications to macroeconomics (GDP and PCE) and finance
• Nonlinear combinations outperform (RMSPE and KLIC) the

constant and the linear time-varying weighting.

• In a multivariate setup the nonlinear schemes with learning
• utperform random schemes without learning.

Monica Billio Roberto Casarin University of Venice University of Brescia Francesco Ravazzolo Combining Predictive Densities using a Bayesian Nonlinear Filte