Abstract
The subject of this paper is modelling and optimal control of consumption and explorationof non‐renewable resources via piecewise‐deterministic Markov processes (PDP). Ingeneral, one assumes that new reserves are found at random times T 1 , T 2 , T 3 ,..., where theinter‐arrival times are exponentially distributed. The amount of resource found at each T i is also a random variable with a given distribution. This process falls into the category ofPDP. In this paper, we clearly show how one can apply the PDP optimal control theory tothe optimal consumption and exploration of non‐renewable resources under uncertain exploration.The advantage of using a PDP model is that no matter how complicated thedynamics, utility function and distribution functions are, as long as they satisfy some mildassumptions, one can easily transform the problem into a deterministic optimal controlproblem, which can be solved iteratively with guaranteed convergence for the value function.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
T.M. Apostol, Mathematical Analysis, Addison-Wesley, 1974.
K.J. Arrow, Optimal pricing, use and exploration of uncertain natural resource stocks, Conference on Natural Resource Pricing, Trail Lake, WY, August 1977.
G. Barles, Solutions de Viscosité des Equations de Hamilton-Jacobi, Mathematics and Applications 17, Springer, 1994.
D.P. Bertsekas, Dynamic Programming and Optimal Control, Vols.1 and 2, Athena Scientific, Belmont, MA, 1995.
G. Barles and B. Perthame, Comparison principle for Dirichlet-type Hamilton-Jacobi equations and singular perturbations of degenerated elliptic equations, Applied Mathematics and Optimization 21(1990)21-44.
D.P. Bertsekas and S.E. Shreve, Stochastic Optimal Control: The Discrete Time Case, Mathematics in Science and Engineering 139, Academic Press, 1978.
M.H.A. Davis, Markov Models and Optimizations, Monographs on Statistics and Applied Probability 49, Chapman and Hall, 1993.
S.D. Deshmukh and S.R. Pliska, Optimal consumption and exploration of nonrenewable resources under uncertainty, Econometrica 48(1980)177-200.
N.A. Derzko and S.P. Sethi, Optimal exploration and consumption of a natural resource: Stochastic case, International Journal of Policy Analysis 5(1981)185-200.
N.A. Derzko and S.P. Sethi, Optimal exploration and consumption of a natural resource: Deterministic case, Optimal Control Applications and Methods 2(1981)1-21.
M. Farid, Optimal control, piecewise-deterministic processes and viscosity solutions, Ph.D. Thesis, Department of Electrical Engineering, Imperial College, London, 1997.
P.S. Hagan, D.E. Woodward, R.E. Caflisch and J.B. Keller, Optimal pricing, use and exploration of uncertain natural resources, Applied Mathematical Finance 1(1994)87-108.
M.H.A. Davis and M. Farid, Piecewise-deterministic processes and viscosity solutions, in: Stochastic Analysis, Control, Optimization and Applications. A Volume in Honor of W.H. Fleming, eds. W.M. McEneaney, G. Yin and Q. Zhang, Systems and Control: Foundations and Applications, Birkhäuser, Boston, 1999, pp. 249-268.
H.M. Soner, Optimal control of a one-dimensional storage process, Applied Mathematics and Optimization 13(1985)175-191.
D. Vermes, Optimal control of piecewise deterministic Markov processes, Stochastics 14(1985)165-207.
Rights and permissions
About this article
Cite this article
Farid, M., Davis, M. Optimal consumption and exploration: A case studyin piecewise‐deterministic Markov modelling. Annals of Operations Research 88, 121–137 (1999). https://doi.org/10.1023/A:1018918027153
Issue Date:
DOI: https://doi.org/10.1023/A:1018918027153