Abstract
This article introduces a Toolkit for Value Function Iteration. The toolkit is implemented in Matlab and makes automatic use of the GPU and of parallel CPUs. Likely uses are Teaching, Testing Algorithms, Replication, and Research. I here provide a description of some of the main components and algorithms. I also describe the design philosophy underlying choices about how to structure the toolkit and which algorithms to use. Rather than provide simple examples, something best done online (links are given), I instead perform a replication of a classic paper from the real business cycle literature as a demonstration of the use of such a toolkit. The Toolkit, Documentation, Examples, Replications, and more can be found at vfitoolkit.com
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A pre-alpha version also exists in Julia.
In the absence of such complications methods based on the first-order conditions, like perturbation and projection methods, are typically preferable, and in the case of perturbation methods there is already a great toolkit in the form of Dynare.
Assuming of course that the model satisfies the required assumptions on the shape of the value function, etc.
Howards improvement algorithm is also sometimes called modified policy iteration.
The description of this as Case 1 is chosen as it coincides exactly with the definition of Case 1 for stochastic value function problems used in Chaps. 8 and 9 of Stokey, Lucas and Prescott (SLP)—Recursive Dynamic Economics (eg. p. 260). In their notation this is any problem that can be written as \(\textit{v}(x,z)=\sup _{y\in \Gamma (x,z)} \{ F(x,y,z) + \beta \int _Z \textit{v}(y,z')Q(z,dz')\}\). Relating the notation used in this paper with the notation of SLP with which many readers will be more familiar: d and \(a'\) are what SLP call y, a is x, and z is z. The VFI Toolkit differentiates between d and \(a'\) as the distinction makes a substantial difference for the implementation of the algorithm allowing both a faster run-time and lower memory usage.
Both of these issues could be avoided if the codes were changed, first by banning the possibility that the return function at a given state value necessarily takes value of negative infinity (regardless of decision variable), and then, after having banned such cases, by choosing a ‘small’ enough initial guess that monotonicity will drive convergence. A choice has been made that any speed boost from doing so was essentially non-existent, while the need for the user to avoid states in which the utility was negative infinity and be very careful about the initial guess was against the robustness principle underlying design of the toolkit.
These must be so that the element in row i and column j gives the probability of going from state i this period to state j next period.
By default, \(v\,f\,options.polindorval=1\), they will be the indexes, if you set \(v\,f\,options.polindorval=2\) they will be the values.
Ie. \(z_t=\rho z_{t+1} + \epsilon _t\), where \(\epsilon \sim logN(0,\sigma _{\epsilon })\), versus the more common \(log(z_t)=\rho log(z_{t-1}) + \epsilon _t\), where \(\epsilon \sim N(0,\sigma _{\epsilon })\). Both satisfy that the shock process is strictly non-negative, which can be important for theoretical reasons.
For more on linear-quadratic dynamic programming, see Díaz-Gímenez (2001). Since the log-normal innovations are independent and identically distributed, the conditional mean of the innovations is just equal to their unconditional mean; ie. because of the choice of numerical method only the mean value of the innovations is relevant. Hence the actual log-normality of the innovations is entirely ignored by the solution method used.
Hansen (1985) uses slightly different notation, \(\alpha \) he calls \(\theta , z_t\) he calls \(\lambda _t\), and \(\rho \) he calls \(\gamma \).
So \(exp(\epsilon )\) is distributed as \(N(\mu _{ln},\sigma _{ln}^2)\), where \(\mu _{ln}=log(1-\rho )-\sigma _{ln}^2/2, \sigma _{ln}=\sqrt{log(1+\frac{\sigma _{\epsilon }^2}{(1-\rho )^2}}.\)
Calculating the standard business cycle statistics involves taking logs, applying the Hodrick-Prescott filter, and then calculating standard deviations and correlations; all of which are standard functions of Matlab.
One could use the Tauchen method to discretize the (exponential of) the lognormal innovation itself and then take the log. This would involve an increase in the state space; to z and \(\epsilon \) instead of just z. In combination with the use of pure discretization it also necessitates then creating another grid for the AR(1) productivity shocks themselves, and finding the transition matrix for this shock.
Counting transitions and then normalizing is a standard and well behaved estimator for Markov transition matrices. The choice of using histogram routines to choose the grid was an arbitrary assumption.
I will not attempt any summary here of the literature on real business cycle models and their successes and failures.
References
Aldrich, E., Fernandez-Villaverde, J., Gallant, R., & Rubio-Ramirez, J. (2011). Tapping the supercomputer under your desk: Solving dynamic equilibrium models with graphics processors. Journal of Economic Dynamics and Control, 35(3), 386–393.
Aruoba, S., Fernandez-Villaverde, J., & Rubio-Ramirez, J. (2006). Comparing solution methods for dynamic equilibrium economies. Journal of Economic Dynamics and Control, 30(12), 2477–2508.
Barillas, F., & Fernandez-Villaverde, J. (2007). A generalization of the endogenous grid method. Journal of Economic Dynamics and Control, 31, 2698–2712.
Bona, J., & Santos, M. (1997). On the role of computation in economic theory. Journal of Economic Theory, 72, 241–281.
Coffman, L., & Niederle, M. (2015). Pre-analysis plans have limited upside, especially where replications are feasible. Journal of Economic Perspectives, 29(3), 81–98.
Díaz-Gímenez, J. (2001). Linear quadratic approximations: An introduction. In R. Márimon, A. Scott (Eds.), Computational methods for the study of dynamic economies (2nd ed.). Oxford: Oxford University Press.
Dziubinski, M., & Grassi, S. (2014). Heterogeneous computing in economics: A simplified approach. Computational Economics, 43(4), 485–495.
Garlappi, L., & Skoulakis, G. (2009). Numerical solutions to dynamic portfolio problems: The case for value function iteration using Taylor approximation. Computational Economics, 33, 193–207.
Hansen, G. (1985). Indivisible labor and the business cycle. Journal of Monetary Economics, 16(3), 309–327.
Kamihigashi, T., & Stachurski, J. (2015). Seeking ergodicity in dynamic economies. Working paper.
Kirkby, R. (2015). Convergence of discretized value function iteration. Computational Economics. doi:10.1007/s10614-015-9545-0
Maliar, L., & Maliar, S. (2013). Envelope condition method versus endogenous grid method for solving dynamic programming problems. Economics Letters, 120, 262–266.
Santos, M., & Peralta-Alva, A. (2005). Accuracy of simulations for stochastic dynamic models. Econometrica, 66, 409–426.
Stokey, N., Lucas, R. E., & Prescott, E. C. (1989). Recursive methods in economic dynamics. Cambridge: Harvard University Press.
Acknowledgments
I acknowledge the useful comments of seminar participants at the the 21st Conference on Economics and Finance (Taipei, 2015). Thanks to Alexander Grohe–Meyer for finding a coding bug which could have been rather embarrasing. Thanks to Iskander Karibzhanov letting me recycle some of his quadrature codes as part of the toolkit. Thanks to Erick Sager and William Peterman for helpful discussions. Thanks to an anonymous referee for helping substantially improve the readability of this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kirkby, R. A Toolkit for Value Function Iteration. Comput Econ 49, 1–15 (2017). https://doi.org/10.1007/s10614-015-9544-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10614-015-9544-1