Abstract
A class of stochastic control problems where the payoff depends on the running maximum of a diffusion process is described. The controller must make two kinds of decision: first, he must choose a work rate (this decision determines the rate of profit as well as the proximity of failure), and second, he must decide when to replace a deteriorated system with a new one. Preventive replacement is a realistic option if the cost for replacement after failure is larger than the cost of a preventive replacement.
We focus on the revenue and replacement cost for a single work cycle and solve the problem in two stages. First, the optimal feedback control (work rate) is determined by maximizing the payoff during a single excursion of a controlled diffusion away from the running maximum. This step involves the solution of the Hamilton-Jacobi-Bellman partial differential equation. The second step is to determine the optimal replacement set. The assumption that failure occurs only when the state is increasing restricts the optimal replacement set. This leads to a simple formula for the optimal replacement level in terms of the value function.
The authors’ research was supported by the National Science Foundation grant DMS-9006674.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
R.S. Anderson, Replacement with nonconstant operating cost, SIAM J. Control Optim., 26 (1988), pp. 1076–1098.
R.S. Anderson, Long-run average maintenance problems, preprint.
E.M. Barron, The Bellman equation for control of the running max of a diffusion and applications to look-back options, preprint.
E. Çinlar, Markov additive processes, I & II, Z. Wahrsch. Verw. Gebiete., 24 (1972), pp. 85–121.
___, Shock and wear models and Markov additive processes, in Theory and Applications of Reliability, Vol. 1, Academic Press, New York, 1977.
___, Markov and Semi-Markov models for deterioration, Reliability Theory and Models, Academic Press, New York, 1984.
C. Conrad and N.H. McClamroch, The drilling problem: A stochastic modeling and control example in manufacturing, IEEE Trans. Automat. Control, 32 (1987), pp. 947–958.
W.H. Fleming and R.W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New York, Heidelberg, Berlin, 1975.
I.I. Gihman and A.V. Skorohod, Controlled Stochastic Processes, Springer-Verlag, New York, Heidelberg, Berlin, 1979.
A.C. Heinricher and R.H. Stockbridge, Optimal control of the running max, SIAM J. Control Optim., 29 (1991).
___, Long-term average control for a continuous, monotone process, submitted for publication.
___, An infinite-dimensional LP solution to control of a continuous, monotone process, to appear in Proceedings of the Joint U.S.-French Workshop on Applied Stochastic Processes, Springer Lecture Notes in Mathematics, 1991.
___, Optimal control and replacement with state-dependent failure rate: Dynamic programming, submitted for publication.
___, Optimal control and replacement with state-dependent failure rate: An invarient measure approach, submitted for publication.
I. Karatzas and S.E. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, Heidelberg, Berlin, 1987.
T.G. Kurtz, Martingale problems for constrained Markov processes, Recent Advances in Stochastic Calculus, J.S. Baras and V. Mirelli, eds., Springer-Verlag, New York, 1990. University of Maryland (to appear).
___, A control formulation for constrained Markov processes, Proceedings of the AMS-SIAM Summer Seminar on Mathematics of Random Media (to appear).
P.L. Lions, Optimal control of diffusions and Hamilton-Jacobi equations. Part I: the dynamic programming principle and applications, Comm. Partial Differential Equations, 8 (1983), pp. 1101–1174.
H.M. Taylor, Optimal replacement under additive damage and other failure models, Naval Research Logistics Quarterly, 22 (1975), pp. 1–18.
C. Valdez-Flores and R.M. Feldman, A survey of preventive maintenance models for stochastically deteriorating single-unit systems, Naval Research Logistics Quarterly, 36 (1989), pp. 419–446.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1992 Springer-Verlag
About this paper
Cite this paper
Heinricher, A.C., Stockbridge, R.H. (1992). Optimal control and replacement with state-dependent failure rate. In: Duncan, T.E., Pasik-Duncan, B. (eds) Stochastic Theory and Adaptive Control. Lecture Notes in Control and Information Sciences, vol 184. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0113244
Download citation
DOI: https://doi.org/10.1007/BFb0113244
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-55962-7
Online ISBN: 978-3-540-47327-5
eBook Packages: Springer Book Archive