Mathematical Methods of Operations Research

, Volume 81, Issue 3, pp 245–267 | Cite as

Dynamic programming with Hermite approximation

  • Yongyang CaiEmail author
  • Kenneth L. Judd
Original Article


Numerical dynamic programming algorithms typically use Lagrange data to approximate value functions over continuous states. Hermite data can be easily obtained from solving the Bellman equation and used to approximate the value functions. We illustrate the use of Hermite data with one-, three-, and six-dimensional examples. We find that Hermite approximation improves the accuracy in value function iteration (VFI) by one to three digits using little extra computing time. Moreover, VFI with Hermite approximation is significantly faster than VFI with Lagrange approximation for the same accuracy, and this advantage increases with the dimension of the continuous states.


Dynamic programming Value function iteration  Hermite approximation Dynamic portfolio optimization  Multi-country optimal growth model 

JEL Classification

C61 C63 


  1. Aldrich EM, Fernandez-Villaverde J, Gallant AR, Rubio- Ramrez JF (2011) Tapping the supercomputer under your desk: solving dynamic equilibrium models with graphics processors. J Econ Dyn Control 35:386–393zbMATHCrossRefGoogle Scholar
  2. Bellman R (1957) Dynamic programming. Princeton University Press, PrincetonzbMATHGoogle Scholar
  3. Bertsekas D (2005) Dynamic programming and optimal control, vol 1. Athena Scientific, BelmontzbMATHGoogle Scholar
  4. Bertsekas D (2007) Dynamic programming and optimal control, vol 2. Athena Scientific, BelmontGoogle Scholar
  5. Bisschop JJ, Meeraus A (1982) On the development of a general algebraic modeling system in a strategic planning environment. Math Program Stud 20:1–29Google Scholar
  6. Brooke A, Kendrick D, Meeraus A, Raman R (1997) GAMS language guide. Technical report, GAMS Development CorporationGoogle Scholar
  7. Byrd RH, Nocedal J, Waltz RA (2006) KNITRO: an integrated package for nonlinear optimization. Technical report, Ziena
  8. Cai Y (2010) Dynamic programming and its application in economics and finance. PhD thesis, Stanford UniversityGoogle Scholar
  9. Cai Y, Judd KL (2012) Dynamic programming with shape-preserving rational spline Hermite interpolation. Econ Lett 117(1):161–164zbMATHMathSciNetCrossRefGoogle Scholar
  10. Cai Y, Judd KL (2013) Shape-preserving dynamic programming. Math Methods Oper Res 77(3):407–421zbMATHMathSciNetCrossRefGoogle Scholar
  11. Cai Y, Judd KL (2014) Advances in numerical dynamic programming and new applications. Chapter 8. In: Schmedders Karl, Judd Kenneth L (eds) Handbook of computational economics, vol 3. Elsevier, AmsterdamGoogle Scholar
  12. Cai Y, Judd KL, Lontzek TS (2013) The social cost of stochastic and irreversible climate change. NBER working paper 18704Google Scholar
  13. Cai Y, Judd KL, Xu R (2013) Numerical solution of dynamic portfolio optimization with transaction costs. NBER working paper No. 18709Google Scholar
  14. Cai Y, Judd KL, Thain G, Wright S (2015) Solving dynamic programming problems on computational grid. Comput Econ 45(2):261–284Google Scholar
  15. Cocco JF, Gomes FJ, Maenhout PJ (2005) Consumption and portfolio choice over the life cycle. Rev Financ Stud 18(2):491–533CrossRefGoogle Scholar
  16. Czyzyk J, Mesnier MP, Moré JJ (1998) The NEOS server. IEEE Comput Sci Eng 5:68–75CrossRefGoogle Scholar
  17. Den Haan WJ, Judd KL, Juillard M (2011) Computational suite of models with heterogeneous agents II:multi-country real business cycle models. J Econ Dyn Control 35:175–177zbMATHCrossRefGoogle Scholar
  18. Dolan ED, Fourer R, Goux JP, Munson TS, Sarich J (2008) Kestrel: an interface from optimization modeling systems to the NEOS server. INFORMS J Comput 20(4):525–538zbMATHMathSciNetCrossRefGoogle Scholar
  19. Drud AS (1996) CONOPT: a system for large scale nonlinear optimization. ARKI Consulting and Development A/S, BagsvaerdGoogle Scholar
  20. Fourer R, Gay DM, Kernighan BW (1990) A modeling language for mathematical programming. Manag Sci 36:519–554zbMATHCrossRefGoogle Scholar
  21. Fourer R, Gay DM, Kernighan BW (2003) AMPL: a modeling language for mathematical programming, 2nd edn. Duxbury Press, Pacific GroveGoogle Scholar
  22. Gill P, Murray W, Saunders MA, Wright MH (1994) User’s Guide for NPSOL 5.0: a Fortran Package for Nonlinear Programming. Technical report, SOL, Stanford UniversityGoogle Scholar
  23. Gill P, Murray W, Saunders MA (2005) SNOPT: an SQP algorithm for largescale constrained optimization. SIAM Rev 47(1):99–131zbMATHMathSciNetCrossRefGoogle Scholar
  24. Griebel M, Wozniakowski H (2006) On the optimal convergence rate of universal and nonuniversal algorithms for multivariate integration and approximation. Math Comput 75(255):1259–1286zbMATHMathSciNetCrossRefGoogle Scholar
  25. Gropp W, Moré JJ (1997) Optimization environments and the NEOS server. In: Buhmann MD, Iserles A (eds) Approximation theory and optimization: tributes to M. J. D. Powell. Cambridge University Press, Cambridge, UK, pp 167–182Google Scholar
  26. Gupta A, Murray W (2005) A framework algorithm to compute optimal asset allocation for retirement with behavioral utilities. Comput Optim Appl 32(1/2):91–113zbMATHMathSciNetCrossRefGoogle Scholar
  27. Infanger G (2006) Dynamic asset allocation strategies using a stochastic dynamic programming approach. Chapter 5. In: Zenios SA, Ziemba WT (Eds) Handbook of asset and liability management, volume 1, North HollandGoogle Scholar
  28. Judd KL (1998) Numerical methods in economics. The MIT Press, CambridgezbMATHGoogle Scholar
  29. Judd KL, Maliar L, Maliar S (2011) Numerically stable and accurate stochastic simulation approaches for solving dynamic economic models. Quant Econ 2:173–210zbMATHMathSciNetCrossRefGoogle Scholar
  30. Juillard M, Villemot S (2011) Multi-country real business cycle models: accuracy tests and test bench. J Econ Dyn Control 35:178–185zbMATHMathSciNetCrossRefGoogle Scholar
  31. Malin BA, Krueger D, Kubler F (2011) Solving the multi-country real business cycle model using a Smolyak-collocation method. J Econ Dyn Control 35:229–239zbMATHMathSciNetCrossRefGoogle Scholar
  32. McCarl B, et al. (2011) McCarl expanded GAMS user guide, version 23.6. Accessed 06 Sept 2012
  33. Murtagh BA, Saunders MA (2003) MINOS 5.51 User’s Guide. Technical report, SOL, Stanford University
  34. Philbrick CR, Kitanidis PK (2001) Improved dynamic programming methods for optimal control of lumped parameter stochastic systems. Oper Res 49(3):398–412zbMATHMathSciNetCrossRefGoogle Scholar
  35. Powell WB (2007) Approximate dynamic programming: solving the curses of dimensionality. Wiley Interscience, HobokenCrossRefGoogle Scholar
  36. Powell WB, Van Roy B (2004) Approximate dynamic programming for high-dimensional dynamic resource allocation problems. Chapter 10. In: Si J, Barto AG, Powell W, Wunsch D (Eds) Handbook of learning and approximate dynamic programming. Wiley-IEEE Press, Hoboken, NJGoogle Scholar
  37. Rivlin TJ (1990) Chebyshev polynomials: from approximation theory to algebra and number theory. Wiley Interscience, New YorkzbMATHGoogle Scholar
  38. Rust J (1997) Using randomization to break the curse of dimensionality. Econometrica 65(3):487–516zbMATHMathSciNetCrossRefGoogle Scholar
  39. Rust J (2008) Dynamic programming. In: Durlauf Steven N, Blume Lawrence E (eds) New palgrave dictionary of economics, 2nd edn. Palgrave Macmillan, BasingstokeGoogle Scholar
  40. Rust J, Traub JF, Wozniakowski H (2002) Is there a curse of dimensionality for contraction fixed points in the worst case? Econometrica 70(1):285–329zbMATHMathSciNetCrossRefGoogle Scholar
  41. Schumaker L (1983) On shape-preserving quadratic spline interpolation. SIAM J Numer Anal 20:854–864zbMATHMathSciNetCrossRefGoogle Scholar
  42. Trick MA, Zin SE (1997) Spline approximations to value functions–linear programming approach. Macroecon Dyn 1:255–277zbMATHCrossRefGoogle Scholar
  43. Wang SP, Judd KL (2000) Solving a savings allocation problem by numerical dynamic programming with shape-preserving interpolation. Comput Oper Res 27(5):399–408zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Hoover InstitutionStanford UniversityStanfordUSA
  2. 2.University of ChicagoChicagoUSA
  3. 3.NBERCambridgeUSA

Personalised recommendations