Optimal bayesian control of a nonlinear regression process with unknown parameters

  • Nicholas M. Kiefer
  • Yaw Nyarko
Models And Control Policies In Economics
Part of the Lecture Notes in Control and Information Sciences book series (LNCIS, volume 121)

Abstract

We have considered the decision problem facing an agent controlling a nonlinear regression process when parameters in the mean function and in the error distribution are unknown. The agent faces a tradeoff between accumulating information by varying the values of the regressors and accumulating one-period reward by following the one-period expected reward maximizing policy. We show that the problem can be brought into the dynamic programming framework and that the value function satisfies the usual functional equation. The sequence of beliefs about the unknown parameters is shown to converge almost surely. Further, the optimal action process converges to the one-period optimal action under limit beliefs.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anderson, T.W. and J. Taylor, (1976), "some Experimental Results on and Statistical Properties of Least Squares Estimates in Control Problems," Econometrica, 44:1289–1302.Google Scholar
  2. Billingsley, P., (1968), Convergence of Probability Measures, Wiley, New York.Google Scholar
  3. Blackwell, D., (1965), "Discounted Dynamic Programming," Annals of Mathematical Statistics, 36, pp. 2226–2235.Google Scholar
  4. Chung, K.L., (1974), A Course in Probability Theory, 2nd edition. Academic Press, New York.Google Scholar
  5. Diaconis, P. and D. Freedman, (1986), "On The Consistency Of Bayes Estimates," Annals of Statistics, 14, 1–26 (discussion and rejoinder 26–27).Google Scholar
  6. Easley, D. and N.M. Kiefer, (1986), "Controlling a Stochastic Process with Unknown Parameters," Cornell University working paper, forthcoming in Econometrica.Google Scholar
  7. Harkema, R., (1975), "An Analytical Comparison of Certainty Equivalence and Sequential Updating," JASA, 70, 348–350.Google Scholar
  8. Kiefer, N.M. and Y. Nyarko, "Control of a Linear Regression Process with Unknown Parameters" in W. Barnett, E. Berndt and H. White (eds.), Dynamic Econometric Modelling, New York: Cambridge University Press, 1987.Google Scholar
  9. Kiefer, N.M., "Optimal Collection of Information by Partially Informed Agents," Cornell working paper, 1988.Google Scholar
  10. Kihlstrom, R.E., (1984), "A ‘Bayesian’ Exposition of Blackwell's Theorem on the Comparison of Experiments," in Bayesian Models in Economic Theory, eds. M. Boyer and R.E. Kihlstrom, Elsevier Science Publishers B.V.Google Scholar
  11. Jordan, J.S., (1985), "The Strong Consistency of the Least Squares Control Rule and Parameter Estimates," manuscript.Google Scholar
  12. Maitra, A., (1968), "Discounted Dynamic Programming in Compact Metric Spaces," Sankhav, Ser A, 30, pp. 211–216.Google Scholar
  13. Parthasarathy, K., (1967), Probability Measures on Metric Spaces, Academic Press, New York.Google Scholar
  14. Taylor, J.B., (1974), "Asymptotic Properties of Multiperiod Control Rules in the Linear Regression Model," International Economic Review, 15, 472–484.Google Scholar
  15. Zellner, A., (1981), An Introduction to Bayesian Inference in Econometrics, Wiley: New York.Google Scholar

Copyright information

© Springer-Verlag 1989

Authors and Affiliations

  • Nicholas M. Kiefer
    • 1
  • Yaw Nyarko
    • 2
  1. 1.Department of EconomicsCornell UniversityIthaca
  2. 2.Department of EconomicsBrown UniversityProvidence

Personalised recommendations