Solving Models With Off-The-Shelf Software An Example Of Potential - - PowerPoint PPT Presentation

Introduction The model Solving the model Results Conclusion

Solving Models With Off-The-Shelf Software – An Example Of Potential Pitfalls Associated With The Use And Abuse Of Default Parameter Settings

Ric D. Herbert1 Peter J. Stemp2

1Faculty of Science and Information Technology, The University of Newcastle,

Ourimbah, NSW, Australia

2Department of Economics, Monash University, Clayton East, Vic, Australia

15th International Conference on Computing in Economics and Finance, Sydney Australia, July 2009

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Table of contents

1 Introduction 2 The model 3 Solving the model 4 Results 5 Conclusion

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Introduction

This paper is part of a project we have been working on about the computational appropriateness of solution methods for dynamic macroeconomic models. There can be a temptation to rely on default settings in proprietary software to derive solutions to the model. In this paper we show that, for the solution of non-linear dynamic models, this approach can be inappropriate.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Method

We consider versions of a simple model with stable and unstable eigenvalues, so that any dynamic solution requires the calculation of appropriate “jumps” in endogenous variables.

One version of the model, expressed in levels, is highly non-linear. A second version of the model, expressed in logarithms, is linear.

We start by showing the results that can be derived using the default settings of a typical solver. We show that this gives unsatisfactory results for the non-linear model; and then, show how the default settings and choice of solver can be adjusted to give acceptable results.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

The linear model (expressed in logarithms)

The model is given by the following set of equations: ¯ m − p = α1y − α2 ˙ p (1) y = β + (1 − γ)n (2) ˙ n = θ(δ − γ)n (3) ˙ w = η(n − ¯ n) (4) The Greek symbols denote parameters with a positive value with 0 < γ < 1.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Model variables

y = output; n = employment; ¯ n = full employment, assumed to be constant; p = price level; ¯ m = nominal money stock, assumed to be constant; and w = wage rate. The model is shocked by a change to the money stock.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Steady state

Since this model is linear, it has a unique equilibrium which satisfies the following equations: ¯ m − p = α1y (5) y = β + (1 − γ)n (6) δ − γn = w − p (7) n = ¯ n (8)

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Calibrating the model and eigenvalues

α1 = 1 α2 = 0.5 β = 0 γ = 0.5 δ = log(0.5) θ = 1 η ∈ {0.2, 0.5, 10, 100} ¯ n = 1, ¯ m = 0.1, ¯ m0 = 0 Eigenvalues η λ1 λ2 λ3 0.2 2.3

• 0.50
• 0.34

0.5 2.3

• 0.41+0.51i
• 0.41-0.51i

10.0 2.1

• 0.32+3.0i
• 0.32-3.0i

100.0 2.0

• 0.26+9.9i
• 0.26-9.9i

λ1 indicates unstable pole as η increases eigenvalues become complex and have larger imaginary parts shock to m results in p ‘jumping’ so economy moves on to the stable trajectory to new steady-state

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Linear Model Eigenvalues for η ∈ [−100, 100].

−100 −80 −60 −40 −20 20 40 60 80 100 −15 −10 −5 5 10 Eigenvalues as η varies η Real part of eigenvalue λ1 λ2 λ3 −100 −80 −60 −40 −20 20 40 60 80 100 −10 −8 −6 −4 −2 2 4 6 8 10 Eigenvalues as η varies η Imaginary part of eigenvalue λ1 λ2 λ3

Real part on LHS, imaginary part on RHS The model has a dominant real eigenvalue greater than zero for all the values of η. As η increases above 0.2 the model becomes more oscillatory.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Analytic solution

5 10 15 −0.44 −0.435 −0.43 −0.425 −0.42 −0.415 −0.41 −0.405 −0.4 −0.395 −0.39 Time p Prices with varying η 5 10 15 0.98 1 1.02 1.04 1.06 1.08 1.1 Time n Employment with varying η 5 10 15 −1.7 −1.68 −1.66 −1.64 −1.62 −1.6 −1.58 −1.56 −1.54 −1.52 −1.5 Time w Wages with varying η

The closed-form solution solved using standard matrix techniques Dynamic solution becomes more oscillatory as η increases

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

The non-linear model (expressed in levels)

An equivalent model specification can be rewritten in levels as follows: ˙ P = P log

• PN(1−γ)α1

¯ M 1

α2

(9) ˙ N = N log P(1 − γ)N−γ W θ (10) ˙ W = W log N ¯ N η (11) The variables are as for the model above except that upper case letters denote levels.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Steady-states

Some solutions to the non-linear model can be derived by using the following transformation P = exp(p) (12) N = exp(n) (13) W = exp(w) (14) There are multiple steady-states with the non-linear model and some are not economically meaningful. These include the steady-state: P = (15) N = ¯ N (16) W = (17)

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Two Point Boundary Value Problem

The computational solution to this model can be formulated as a a two point boundary value problem written as: ˙ x = f(t, x) 0 < t < tf (18) g(x(0), x(tf )) = (19) Where f(.) is either the linear model, or non-linear model and g(.) is obtained from the steady-state equation. For each initial conditions vector c, the model has a solution x(t) = x(t; c) which satisfies x(0; c) = c. Substituting into equation 19 gives: g(c, x(tf ; c)) = 0 (20) and the problem is to solve this set of non-linear equations.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Solving 2PBVP

The common approaches to solving the problem are by shooting, finite differences or projection methods. Here we will focus on on a shooting method Shooting methods for this model require

An ODE solver to generate x(t) = x(t; c) A searcher to solve g(c, x(tf ; c)) = 0

The focus of this paper is on the model solver and default parameters of the solver.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Previous Work

The nonlinear model can only be solved by computational

• means. We focus on shooting algorithms and particularly

forward (in time) and reverse shooting. In Stemp and Herbert (2006) we focussed on the oscillatory linear model (η = 100) as it may have similar dynamics to nonlinear models. We showed that with η = 100 the choice of the ODE solver has significant impact on the likely success of shooting algorithms. In Herbert and Stemp (2009) we focussed on the nonlinear model and showed that forward-shooting is likely to solve to the ‘wrong’ solution for the nonlinear model but the ‘right’ solution for the linear model.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Approach

We use the reverse shooting algorithm and compare the solution to the closed-form solution (expressed in levels). We use Matlab as the software environment. We consider 3 candidate ODE solvers which could typically be used for this model. We focus of η = 0.2 so that all eigenvalues are real.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

ODE Solvers

Runge-Kutta (Fehlberg method, Dormand-Prince 4/5 formula)

A good robust single-step solver Solver of first choice for problem like this. (default!) Matlab ode45() function

Adams-Bashforth-Moulton

A multi-step solver Not normally used for a model like this as model is not computationally expensive to evaluate. Matlab ode113() function

Stiff

This model is not obviously stiff. (Largest eigenvalue about 4 times smallest) But stiffness could be the cause of the problems. Matlab ode115s() function

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Default RK solver linear model solution

5 10 15 0.645 0.65 0.655 0.66 0.665 0.67 0.675 Time Prices [!=0.2] 5 10 15 2.7 2.75 2.8 2.85 2.9 2.95 Time Employment [!=0.2] 5 10 15 0.18 0.185 0.19 0.195 0.2 0.205 Time Wages [!=0.2] 0.64 0.66 0.68 2.6 2.8 3 0.18 0.19 0.2 0.21 Prices Phase [!=0.2] Wages Employment

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Default RK solver nonlinear model solution

5 10 15 0.645 0.65 0.655 0.66 0.665 0.67 0.675 Time Prices [!=0.2] 5 10 15 2.6 2.7 2.8 2.9 3 Time Employment [!=0.2] 5 10 15 0.18 0.185 0.19 0.195 0.2 0.205 Time Wages [!=0.2] 0.64 0.66 0.68 2.6 2.8 3 0.18 0.19 0.2 0.21 Prices Phase [!=0.2] Wages Employment

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Norm results for all methods: non-linear model when η = 0.2. (Root Mean Square Error)

Method RMSE All RMSE P RMSE N RMSE W RK Solver 0.007744 0.001996 0.021559 0.000506 RK Solver † 0.000478 0.000936 0.001303 8.60E-06 ABM Solver 0.000267 0.000521 0.000743 5.58E-06 ABM Solver † 0.000826 0.001622 0.002254 1.45E-05 ABM Solver ‡ 0.000811 0.001591 0.002212 1.42E-05 ABM Solver * 0.000133 0.000248 0.000364 5.03E-06 Stiff Solver 0.125903 0.034000 0.305671 0.007573 Stiff Solver ‡ 0.000762 0.001470 0.002083 1.61E-05 †tolerances half the defaults on solver (i.e. 10−6) ‡tolerances half the defaults on solver and refine (dense output) * max solver step size of 0.01

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Linear model with ABM limited step-size solver

5 10 15 0.645 0.65 0.655 0.66 0.665 0.67 0.675 Time Prices [!=0.2] 5 10 15 2.7 2.75 2.8 2.85 2.9 2.95 Time Employment [!=0.2] 5 10 15 0.18 0.185 0.19 0.195 0.2 0.205 Time Wages [!=0.2] 0.64 0.66 0.68 2.6 2.8 3 0.18 0.19 0.2 0.21 Prices Phase [!=0.2] Wages Employment

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Non-linear model with ABM limited step-size solver

5 10 15 0.645 0.65 0.655 0.66 0.665 0.67 0.675 Time Prices [!=0.2] 5 10 15 2.7 2.75 2.8 2.85 2.9 2.95 Time Employment [!=0.2] 5 10 15 0.18 0.185 0.19 0.195 0.2 0.205 Time Wages [!=0.2] 0.64 0.66 0.68 2.6 2.8 3 0.18 0.19 0.2 0.21 Prices Phase [!=0.2] Wages Employment

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Comments

The standard (default) software solver did not perform well. (Runge-Kutta-Fehlberg). Stiffness does not seem to be the issue. None of the solvers with default parameter settings performed well. Usual practices of reducing tolerances and using dense output also did not perform well. The technique which worked best is the least preferred practice of reducing step-size.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

Take Home Message

Comment We are a team that has extensive software development and model solving experience. We have used best-practice software and best-practice algorithms. We have shown that these can have problems solving a basically trivial model. What about the model solutions we see out there for complex models using simple software? Garbage in garbage out is the old software developers saying.

Herbert, Stemp Solving Models With Off-The-Shelf Software

Introduction The model Solving the model Results Conclusion

References

Herbert, Ric D. and Peter J. Stemp, “Solving a non-linear model: the importance of model specification for deriving a suitable solution”, Mathematics and Computers in Simulation, 2009, vol. 79, no. 9, pp. 2847–2855. Stemp, Peter J. and Ric D. Herbert, “Solving non-linear models with saddle-path instabilities”,Computational Economics, 2006, vol. 28, no. 2, pp. 211–231.

Herbert, Stemp Solving Models With Off-The-Shelf Software