Skip to main content
SpringerLink
Log in

Approximations and bounds for a generalized optimal stopping problem

  • Papers
  • Series A: Theory
  • Published:
Zeitschrift für Operations Research Aims and scope Submit manuscript

Abstract

We study a finite-horizon nonstationary Markovian decision problem, that can be interpreted as generalized optimal stopping and whose solution via the usual dynamic programming is in most practical cases not feasible from a computational point of view. Under certain assumptions, most importantly stochastic monotonicity, upper and lower bounds are obtained for optimal values and decisions using a reduced dynamic programming. From this, a suboptimal policy is derived with an upper bound on its suboptimality. Computational aspects and a particular application from optimal exploratory oil drilling are discussed.

Zusammenfassung

In der Arbeit wird ein nichtstationäres Markowsches Entscheidungsproblem mit endlichem Planungshorizont betrachtet, das als verallgemeinertes Stopp-Problem interpretiert werden kann. Die numerische Lösung des Problems mit Hilfe der üblichen Methode der dynamischen Optimierung ist in der Regel zu rechenaufwendig. Es wird deshalb eine Methode der approximativen Lösung des Problems (mit gewissen Einschränkungen) vorgeschlagen, und es werden obere und untere Schranken für den Optimalwert hergeleitet. Ferner wird eine suboptimale Politik mit einer oberen Schranke für die Suboptimalität angegeben. Abschließend wird ein praktisches Anwendungsbeispiel (optimale Versuchsbohrungen nach Öl) diskutiert, an dem auch rechentechnische Aspekte des entwickelten Lösungsverfahrens erläutert werden.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Barlow, R.E., andF. Proschan: Statistical Theory of Reliability and Life Testing. New York 1975.

  • Barouch, E., andG.M. Kaufman: Oil and Gas Discovery Modelled as Sampling Proportional to Random Size. Sloan School of Mgt. Working Paper, WP888-76, Dec. 1976.

  • —: Estimation of Undiscovered Oil and Gas. Proc. Symposia in Appl. Math., American Math. SocietyXXI, 1977, 77–91.

    Google Scholar 

  • Bertsekas, D.P.: Convergence of Discretization Procedures in Dynamic Programming. IEEE Transactions AC-20, 1975, 415–419.

    Google Scholar 

  • -: Dynamic Programming and Stochastic Control. New York 1976.

  • Daley, D.J.: Stochastically Monotone Markov Chains. Z. Wahrscheinlichkeitstheorie verw. Geb.10, 1968, 305–317.

    Google Scholar 

  • Gombani, A.: Sull'ottimizzazione dell'esplorazione di un giacimento di risorse esauribili. Thesis, University of Padova, 1980.

  • Haggstrom, G.W.: Optimal Sequential Procedures when more than one stop is required. Ann. Math. Stat.38, 1967, 1618–1626.

    Google Scholar 

  • Hahnewald-Busch, A., andV. Nollau: An approximation Procedure for Stochastic Dynamic Programming in Countable State Space. Math. Operationsforsch. Statist., Ser. Optimization9, 1978, 109–117.

    Google Scholar 

  • —: An Approximation Procedure for Stochastic Dynamic Programming based on Clustering of State and Action Spaces. Math. Operationsforsch. Statist., Ser. Optimization10, 1979, 121–130.

    Google Scholar 

  • Hinderer, K.: Foundations of non-stationary Dynamic Programming with Discrete Time Parameter. Lecture Notes in Operations Research and Mathematical Economics, Vol. 33. Berlin-Heidelberg-New York 1970.

  • —: On Approximate Solutions of Finite-Stage Dynamic Programs. Proceedings of the International Conference on Dynamic Programming. University of British Columbia, Vancouver. Ed. by M. Puterman. New York 1979, 289–317.

    Google Scholar 

  • Kalin, D.: Über Markoffsche Entscheidungsmodelle mit halbgeordnetem Zustandsraum. Methods of Operations Research33, 1979.

  • Kaufman, G.M., W. Runggaldier, andZ. Livne: Predicting the Time Rate of Supply from a Petroleum Play. The Economics of Exploration for Energy Resources. Ed. by J.B. Ramsey. Greenwich, Conn., 1981.

  • Langen, H.J.: Convergence of Dynamic Programming Models. Rept. Institut für Angewandte Mathematik, Univ. Bonn 1979.

  • Lehmann, E.L.: Testing Statistical Hypotheses. New York-Chichester 1959.

  • Nikolaev, M.: Generalized Sequential Procedures (in russian). Lietuvos Matematikos RinkinysXIX (3), 1979, 35–44.

    Google Scholar 

  • Runggaldier, W.J., andF. Spizzichino: Approximations and Bounds for Optimal Sequential Decisions. Quaderni dell'Istituto Matematico “G. Castelnuovo”, Univ. Roma, March 1981.

  • Stoyan, D.: Qualitative Eigenschaften und Abschätzungen stochastischer Modelle. München 1977.

  • Serfozo, R.F.: Monotone Optimal Policies for Markov Decision Processes. Math. Programming Study6, 1976, 202–215.

    Google Scholar 

  • Veinott, A.F. Jr.: Optimal Policy in a Dynamic, Single Product, Nonstationary Inventory Model with Several Demand Classes. Operations Res.13, 1965, 761–778.

    Google Scholar 

  • Waldmann, K.H.: Approximations of Inventory Models. Zeitschrift für OR25, 1981, 143–157.

    Google Scholar 

  • White III, C.C.: Monotone Control Laws for Noisy, Countable State Markov Chains. European Journal of OR5, 1980, 124–132.

    Google Scholar 

  • Whitt, W.: Approximations of Dynamic Programs, I. Mathematics of OR3, 1978, 231–243.

    Google Scholar 

  • —: Approximations of Dynamic Programs, II. Mathematics of OR4, 1979, 179–185.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Seminario Matematico, Università di Padova, Italy

    W. J. Runggaldier

  2. Istituto per Ricerche di Dinamica dei Sistemi e Bioingegneria del CNR (LADSEB), Padova, Italy

    W. J. Runggaldier

  3. Istituto Matematico “G. Castelnuovo”, Università di Roma, Roma, Italy

    F. Spizzichino

Authors
  1. W. J. Runggaldier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. Spizzichino
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Research partially supported by the Consiglio Nazionale delle Ricerche (CNR), Italy, through contract n.80.02343.01 and through GNAFA.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Runggaldier, W.J., Spizzichino, F. Approximations and bounds for a generalized optimal stopping problem. Zeitschrift für Operations Research 26, 143–155 (1982). https://doi.org/10.1007/BF01917107

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01917107

Keywords

Search

Navigation