Annals of Operations Research

, Volume 220, Issue 1, pp 263–278 | Cite as

On variable discounting in dynamic programming: applications to resource extraction and other economic models

  • Anna Jaśkiewicz
  • Janusz Matkowski
  • Andrzej S. NowakEmail author


This paper generalizes the classical discounted utility model introduced in Samuelson (Rev. Econ. Stud. 4:155–161, 1937) by replacing a constant discount rate with a function. The existence of recursive utilities and their constructions are based on Matkowski’s extension of the Banach Contraction Principle. The derived utilities go beyond the class of recursive utilities studied in the literature and enable a discussion on subtle issues concerning time preferences in the theory of finance, economics or psychology. Moreover, our main results are applied to the theory of optimal economic growth related with resource extraction models with unbounded utility function of consumption.


Dynamic programming Variable discounting Bellman equation Resource extraction Growth theory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alvarez, F., & Stokey, N. (1998). Dynamic programming with homogeneous functions. Journal of Economic Theory, 82, 167–189. CrossRefGoogle Scholar
  2. Becker, R. A., & Boyd, J. H. III (1997). Capital theory, equilibrium analysis and recursive utility. New York: Blackwell. Google Scholar
  3. Benzion, U., Rapoport, A., & Yagil, J. (1989). Discount rates inferred from decisions: an experimental study. Management Science 35, 270–284. CrossRefGoogle Scholar
  4. Berge, C. (1963). Topological spaces. New York: MacMillan. Google Scholar
  5. Bhattacharya, R., & Majumdar, M. (2007). Random dynamical systems: theory and applications. Cambridge: Cambridge University Press. CrossRefGoogle Scholar
  6. Blackwell, D. (1965). Discounted dynamic programming. Annals of Mathematical Statistics, 36, 226–235. CrossRefGoogle Scholar
  7. Boyd, J. H. III (1990). Recursive utility and the Ramsey problem. Journal of Economic Theory, 50, 326–345. CrossRefGoogle Scholar
  8. Boyd, J. H. III (2006). Discrete-time recursive utility. In R. A. Dana, C. Le Van, T. Mitra, & K. Nishimura (Eds.), Handbook of optimal growth 1, discrete time (pp. 251–272). New York: Springer. CrossRefGoogle Scholar
  9. Denardo, E. V. (1967). Contraction mappings in the theory underlying dynamic programming. SIAM Review, 9, 165–177. CrossRefGoogle Scholar
  10. Dugundji, J., & Granas, A. (2003). Fixed point theory. New York: Springer. Google Scholar
  11. Epstein, L., & Hynes, J. A. (1983). The rate of time preference and dynamic economic analysis. Journal of Political Economy, 91, 611–635. CrossRefGoogle Scholar
  12. Feinberg, E. A. (2002). Total reward criteria. In E. A. Feinberg & A. Shwartz (Eds.), Handbook of Markov decision processes (pp. 173–207). Boston: Kluwer. CrossRefGoogle Scholar
  13. Frederick, S., Loewenstein, G., & O’Donoghue, T. (2002). Time discounting and time preference: a critical review. Journal of Economic Literature, 11, 351–401. CrossRefGoogle Scholar
  14. Green, L., Myerson, J., & McFadden, E. (1997). Rate of temporal discounting decreases with amount of reward. Memory & Cognition, 25, 715–723. CrossRefGoogle Scholar
  15. Hernández-Lerma, O., & Lasserre, J. B. (1996). Discrete-time Markov control processes: basic optimality criteria. New York: Springer. CrossRefGoogle Scholar
  16. Hernández-Lerma, O., & Lasserre, J. B. (1999). Further topics on discrete-time Markov control processes. New York: Springer. CrossRefGoogle Scholar
  17. Hinderer, K. (1970). Foundations of non-stationary dynamic programming with discrete time parameter. In Lecture notes in oper. res., Vol. 33. New York: Springer. Google Scholar
  18. Jaśkiewicz, A., Matkowski, J., & Nowak, A. S. (2011). Persistently optimal policies in stochastic dynamic programming with generalized discounting. Working Paper. Faculty of Mathematics, Computer Science and Econometrics, University of Zielona Góra, Poland. Google Scholar
  19. Kirby, K. N. (1997). Bidding on the future: evidence against normative discounting of delayed rewards. Journal of Experimental Psychology. General, 126, 54–70. CrossRefGoogle Scholar
  20. Koopmans, T. C. (1960). Stationary ordinal utility and impatience. Econometrica, 28, 287–309. CrossRefGoogle Scholar
  21. Kreps, D. M., & Porteus, E. L. (1978). Temporal resolution of uncertainty and dynamic choice theory. Econometrica 46, 185–200. CrossRefGoogle Scholar
  22. Le Van, C., & Morhaim, L. (2002). Optimal growth models with bounded or unbounded returns: a unifying approach. Journal of Economic Theory 105, 158–187. CrossRefGoogle Scholar
  23. Le Van, C., & Vailakis, Y. (2005). Recursive utility and optimal growth with bounded or unbounded returns. Journal of Economic Theory 123, 187–209. CrossRefGoogle Scholar
  24. Lucas, R. E., & Stokey, N. (1984). Optimal growth with many consumers. Journal of Economic Theory, 32, 139–171. CrossRefGoogle Scholar
  25. Marinacci, M., & Montrucchio, L. (2010). Unique solutions for stochastic recursive utilities. Journal of Economic Theory, 145, 1776–1804. CrossRefGoogle Scholar
  26. Matkowski, J. (1975). Integral solutions of functional equations. Dissertationes Mathematicae, 127, 1–68. Google Scholar
  27. Ramsey, F. P. (1928). A mathematical theory of saving. Econometrics Journal, 38, 543–599. Google Scholar
  28. Rincón-Zapatero, J. P., & Rodriguez-Palmero, C. (2003). Existence and uniqueness of solutions to the Bellman equation in the unbounded case. Econometrica, 71, 1519–1555. CrossRefGoogle Scholar
  29. Rincón-Zapatero, J. P., & Rodriguez-Palmero, C. (2009). Corrigendum to “Existence and uniqueness of solutions to the Bellman equation in the unbounded case. Econometrica, 71, 1519–1555 (2003)”. Econometrica, 77, 317–318. CrossRefGoogle Scholar
  30. Samuelson, P. (1937). A note on measurement of utility. Review of Economic Studies 4, 155–161. CrossRefGoogle Scholar
  31. Schäl, M. (1975). Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 32, 179–196. CrossRefGoogle Scholar
  32. Stokey, N. L., Lucas, R. E., & Prescott, E. (1989). Recursive methods in economic dynamics. Cambridge: Harvard University Press. Google Scholar
  33. Strauch, R. (1966). Negative dynamic programming. Annals of Mathematical Statistics, 37, 871–890. CrossRefGoogle Scholar
  34. Thaler, R. H. (1981). Some empirical evidence on dynamic inconsistency. Economics Letters, 8, 201–207. CrossRefGoogle Scholar
  35. Uzawa, H. (1968). Time preference, the consumption function, and optimum asset holding. In J. N. Wolfe (Ed.), Value, capital and growth: papers in honor of Sir John Hicks (pp. 485–504). Edinburgh: Edinburgh University Press. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Anna Jaśkiewicz
    • 1
  • Janusz Matkowski
    • 2
    • 3
  • Andrzej S. Nowak
    • 2
    • 4
    Email author
  1. 1.Institute of Mathematics and Computer ScienceWrocław University of TechnologyWrocławPoland
  2. 2.Faculty of Mathematics, Computer Science and EconometricsUniversity of Zielona GóraZielona GóraPoland
  3. 3.Institute of MathematicsSilesian UniversityKatowicePoland
  4. 4.Institute of FinancePWSZ NysaPoland

Personalised recommendations