Advertisement

Zeitschrift für Operations Research

, Volume 26, Issue 1, pp 143–155 | Cite as

Approximations and bounds for a generalized optimal stopping problem

  • W. J. Runggaldier
  • F. Spizzichino
Papers Series A: Theory

Abstract

We study a finite-horizon nonstationary Markovian decision problem, that can be interpreted as generalized optimal stopping and whose solution via the usual dynamic programming is in most practical cases not feasible from a computational point of view. Under certain assumptions, most importantly stochastic monotonicity, upper and lower bounds are obtained for optimal values and decisions using a reduced dynamic programming. From this, a suboptimal policy is derived with an upper bound on its suboptimality. Computational aspects and a particular application from optimal exploratory oil drilling are discussed.

Keywords

Drilling Lower Bound Dynamic Programming Decision Problem Practical Case 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Zusammenfassung

In der Arbeit wird ein nichtstationäres Markowsches Entscheidungsproblem mit endlichem Planungshorizont betrachtet, das als verallgemeinertes Stopp-Problem interpretiert werden kann. Die numerische Lösung des Problems mit Hilfe der üblichen Methode der dynamischen Optimierung ist in der Regel zu rechenaufwendig. Es wird deshalb eine Methode der approximativen Lösung des Problems (mit gewissen Einschränkungen) vorgeschlagen, und es werden obere und untere Schranken für den Optimalwert hergeleitet. Ferner wird eine suboptimale Politik mit einer oberen Schranke für die Suboptimalität angegeben. Abschließend wird ein praktisches Anwendungsbeispiel (optimale Versuchsbohrungen nach Öl) diskutiert, an dem auch rechentechnische Aspekte des entwickelten Lösungsverfahrens erläutert werden.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barlow, R.E., andF. Proschan: Statistical Theory of Reliability and Life Testing. New York 1975.Google Scholar
  2. Barouch, E., andG.M. Kaufman: Oil and Gas Discovery Modelled as Sampling Proportional to Random Size. Sloan School of Mgt. Working Paper, WP888-76, Dec. 1976.Google Scholar
  3. —: Estimation of Undiscovered Oil and Gas. Proc. Symposia in Appl. Math., American Math. SocietyXXI, 1977, 77–91.Google Scholar
  4. Bertsekas, D.P.: Convergence of Discretization Procedures in Dynamic Programming. IEEE Transactions AC-20, 1975, 415–419.Google Scholar
  5. -: Dynamic Programming and Stochastic Control. New York 1976.Google Scholar
  6. Daley, D.J.: Stochastically Monotone Markov Chains. Z. Wahrscheinlichkeitstheorie verw. Geb.10, 1968, 305–317.Google Scholar
  7. Gombani, A.: Sull'ottimizzazione dell'esplorazione di un giacimento di risorse esauribili. Thesis, University of Padova, 1980.Google Scholar
  8. Haggstrom, G.W.: Optimal Sequential Procedures when more than one stop is required. Ann. Math. Stat.38, 1967, 1618–1626.Google Scholar
  9. Hahnewald-Busch, A., andV. Nollau: An approximation Procedure for Stochastic Dynamic Programming in Countable State Space. Math. Operationsforsch. Statist., Ser. Optimization9, 1978, 109–117.Google Scholar
  10. —: An Approximation Procedure for Stochastic Dynamic Programming based on Clustering of State and Action Spaces. Math. Operationsforsch. Statist., Ser. Optimization10, 1979, 121–130.Google Scholar
  11. Hinderer, K.: Foundations of non-stationary Dynamic Programming with Discrete Time Parameter. Lecture Notes in Operations Research and Mathematical Economics, Vol. 33. Berlin-Heidelberg-New York 1970.Google Scholar
  12. —: On Approximate Solutions of Finite-Stage Dynamic Programs. Proceedings of the International Conference on Dynamic Programming. University of British Columbia, Vancouver. Ed. by M. Puterman. New York 1979, 289–317.Google Scholar
  13. Kalin, D.: Über Markoffsche Entscheidungsmodelle mit halbgeordnetem Zustandsraum. Methods of Operations Research33, 1979.Google Scholar
  14. Kaufman, G.M., W. Runggaldier, andZ. Livne: Predicting the Time Rate of Supply from a Petroleum Play. The Economics of Exploration for Energy Resources. Ed. by J.B. Ramsey. Greenwich, Conn., 1981.Google Scholar
  15. Langen, H.J.: Convergence of Dynamic Programming Models. Rept. Institut für Angewandte Mathematik, Univ. Bonn 1979.Google Scholar
  16. Lehmann, E.L.: Testing Statistical Hypotheses. New York-Chichester 1959.Google Scholar
  17. Nikolaev, M.: Generalized Sequential Procedures (in russian). Lietuvos Matematikos RinkinysXIX (3), 1979, 35–44.Google Scholar
  18. Runggaldier, W.J., andF. Spizzichino: Approximations and Bounds for Optimal Sequential Decisions. Quaderni dell'Istituto Matematico “G. Castelnuovo”, Univ. Roma, March 1981.Google Scholar
  19. Stoyan, D.: Qualitative Eigenschaften und Abschätzungen stochastischer Modelle. München 1977.Google Scholar
  20. Serfozo, R.F.: Monotone Optimal Policies for Markov Decision Processes. Math. Programming Study6, 1976, 202–215.Google Scholar
  21. Veinott, A.F. Jr.: Optimal Policy in a Dynamic, Single Product, Nonstationary Inventory Model with Several Demand Classes. Operations Res.13, 1965, 761–778.Google Scholar
  22. Waldmann, K.H.: Approximations of Inventory Models. Zeitschrift für OR25, 1981, 143–157.Google Scholar
  23. White III, C.C.: Monotone Control Laws for Noisy, Countable State Markov Chains. European Journal of OR5, 1980, 124–132.Google Scholar
  24. Whitt, W.: Approximations of Dynamic Programs, I. Mathematics of OR3, 1978, 231–243.Google Scholar
  25. —: Approximations of Dynamic Programs, II. Mathematics of OR4, 1979, 179–185.Google Scholar

Copyright information

© Physica-Verlag 1982

Authors and Affiliations

  • W. J. Runggaldier
    • 1
    • 2
  • F. Spizzichino
    • 3
  1. 1.Seminario MatematicoUniversità di PadovaItaly
  2. 2.Istituto per Ricerche di Dinamica dei Sistemi e Bioingegneria del CNR (LADSEB)PadovaItaly
  3. 3.Istituto Matematico “G. Castelnuovo”Università di RomaRomaItaly

Personalised recommendations