Abstract
This paper deals with the asymptotic optimality of a stochastic dynamic system driven by a singularly perturbed Markov chain with finite state space. The states of the Markov chain belong to several groups such that transitions among the states within each group occur much more frequently than transitions among the states in different groups. Aggregating the states of the Markov chain leads to a limit control problem, which is obtained by replacing the states in each group by the corresponding average distribution. The limit control problem is simpler to solve as compared with the original one. A nearly-optimal solution for the original problem is constructed by using the optimal solution to the limit problem. To demonstrate, the suggested approach of asymptotic optimal control is applied to examples of manufacturing systems of production planning.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
SETHI, S. P., and ZHANG, Q., Hierarchical Decision Making in Stochastic Manufacturing Systems, Birkhäuser, Boston, 1994.
SAKSENA, V. R., O'REILLY, J., and KOKOTOVIC, P. V., Singular Perturbations and Time-Scale Methods in Control Theory: Survey 1976–1983, Automatica, Vol. 20, pp. 273–293, 1984.
KHASMINSKII, R. Z., YIN, G., and ZHANG, Q., Asymptotic Expansions of Singularly Perturbed Systems Involving Rapidly Fluctuating Markov Chains, SIAM Journal on Applied Mathematics, Vol. 56, pp. 277–293, 1996.
KHASMINSKII, R. Z., YIN, G., and ZHANG, Q., Constructing Asymptotic Series for Probability Distribution of Markov Chains with Weak and Strong Interactions, Quarterly of Applied Mathematics (to appear).
ZHANG, Q., and YIN, G., Central Limit Theorems for Singular Perturbations of Nonstationary Finite State Markov Chains, Annals of Applied Probability, Vol. 6, pp. 650–670, 1996.
YIN, G., and ZHANG, Q., Continuous-Time Markov Chains: Limit Theory and Applications, Springer Verlag, New York, New York, 1997.
JIANG, J., and SETHI, S. P., A State Aggregation Approach to Manufacturing Systems Having Machine States with Weak and Strong Interactions, Operations Research, Vol. 39, pp. 970–978, 1991.
FLEMING, W. H., and SONER, H. M., Controlled Markov Processes and Viscosity Solutions, Springer Verlag, New York, New York, 1992.
DAVIS, M. H. A., Markov Models and Optimization, Chapman and Hall, New York, New York, 1993.
ETHIER, S. N., and KURTZ, T. G., Markov Processes: Characterization and Convergence, Wiley, New York, New York, 1986.
KIMEMIA, J. G., and GERSHWIN, S. B., An Algorithm for the Computer Control Production in Flexible Manufacturing Systems, IIE Transactions, Vol. 15, pp. 353–362, 1983.
AKELLA, R., and KUMAR, P. R., Optimal Control of Production Rate in a Failure Prone Manufacturing System, IEEE Transactions on Automatic Control, Vol. 31, pp. 106–126, 1986.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Zhang, Q., Yin, G. & Boukas, E.K. Controlled Markov Chains with Weak and Strong Interactions: Asymptotic Optimality and Applications to Manufacturing. Journal of Optimization Theory and Applications 94, 169–194 (1997). https://doi.org/10.1023/A:1022667905086
Issue Date:
DOI: https://doi.org/10.1023/A:1022667905086