Abstract
This paper concerns control of stochastic networks using state-dependent safety-stocks.
Three examples are considered: a pair of tandem queues; a simple routing model; and the Dai-Wang re-entrant line. In each case, a single policy is proposed that is independent of network load ρ•. The following conclusions are obtained for the controlled network in each case, where the finite constant K0 is independent of load: The policy is fluid-scale asymptotically optimal, and satisfies the bound
where η* is the optimal steady-state cost.
These results are based on the construction of an approximate solution to the average-cost dynamic programming equations using a perturbation of the value function for an associated fluid model.
Moreover, a new technique is introduced to obtain fluid-scale asymptotic optimality for general networks modeled in discrete time. The proposed policy is myopic with respect to a perturbation of the value function for the fluid model.
Similar content being viewed by others
References
Nicole Bäuerle, Optimal control of queueing networks: An approach via fluid models, Adv. in Appl. Probab. 34(2) (2002) 313–328.
S.L. Bell and R.J. Williams, Dynamic scheduling of a system with two parallel servers in heavy traffic with complete resource pooling: Asymptotic optimality of a continuous review threshold policy, Ann. Appl. Probab. 11 (2001) 608–649.
M. Bramson, State space collapse with application to heavy traffic limits for multiclass queueing networks, Queueing Systems: Theory and Applications 30 (1998) 89–148.
H. Chen and D.D. Yao, Fundamentals of Queueing Networks: Performance, Asymptotics, and Optimization (Springer-Verlag, New York, 2001) Stochastic Modelling and Applied Probability.
M. Chen, C. Pandit and S.P. Meyn, in search of sensitivity in network optimization, Queueing Systems: Theory and Applications 44(4) (2003) 313–363.
D.P. Connors, G. Feigin and D. Yao, Scheduling semiconductor lines using a fluid network model, IEEE Trans. on Robotics and Automation 10 (1994) 88–98.
J.G. Dai and W. Lin, Maximum pressure policies in stochastic processing networks, Operations Research, 53(2) to appear.
J.G. Dai and Y. Wang, Nonexistence of Brownian models of certain multiclass queueing networks, Queueing Systems: Theory and Applications 13 (1993) 41–46.
A. Ephremides and L. Tassiulas, Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks, IEEE Trans. Automat. Control 37 (1992) 1936–1949.
A. Gajrat, A. Hordijk and A. Ridder, Large deviations analysis of the fluid approximation for a controllable tandem queue, Ann. Appl. Probab. 13 (2003) 1423–1448.
L. Georgiadis, W. Szpankowski and L. Tassiulas, A scheduling policy with maximal stability region for ring networks with spatial reuse, Queueing Systems: Theory and Applications 19(1/2) (1995) 131–148.
P.W. Glynn and S.P. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Probab. 24(2) (1996) 916–931.
L. Green, A queueing system with general-use and limited-use servers, Oper. Res. 33(1) (1985) 168–182.
B. Hajek, Optimal control of two interacting service stations, IEEE Trans. Automat. Control, AC-29 (1984) 491–499.
J. M. Harrison, Brownian Motion and Stochastic Flow Systems (Robert E. Krieger Publishing Co. Inc., Malabar, FL, 1990) Reprint of the 1985 original.
J.M. Harrison and M.J. López, Heavy traffic resource pooling in parallel-server systems, Queueing Systems: Theory and Applications 33 (1999) 339–368.
S.G. Henderson, S.P. Meyn and V. B. Tadić, Performance evaluation and policy selection in multiclass networks, Discrete Event Dynamic Systems: Theory and Applications, 13(1/2) (2003) 149–189. Special issue on learning, optimization and decision making (invited).
F.P. Kelly and C.N. Laws, Dynamic routing in open queueing networks: Brownian models, cut constraints and resource pooling, QUESTA, 13 (1993) 47–86.
L. Kleinrock, Queueing Systems Volume 1: Theory (Wiley, 1975).
G. Koole, On the pathwise optimal Bernoulli routing policy for homogeneous parallel servers, Math. Operations Res. 21 (1996) 469–476.
V.G. Kulkarni, Modeling and Analysis of Stochastic Systems (Texts in Statistical Science Series. Chapman and Hall Ltd., London, 1995).
H.J. Kushner, Heavy Traffic Analysis of Controlled Queueing and Communication Networks (Springer-Verlag, New York, 2001). Stochastic Modelling and Applied Probability.
C.N. Laws, Dynamic routing in queueing networks, PhD thesis, Cambridge University, Cambridge, UK, 1990.
C. Maglaras, Dynamic scheduling in multiclass queueing networks: Stability under discrete-review policies, Queueing Systems: Theory and Applications 31 (1999) 171–206.
A. Mandelbaum and A.L. Stolyar, Scheduling flexible servers with convex delay costs: heavy-traffic optimality of the generalized cμ-rule, Operations Res. 52(6) (2004) 836–855.
S.P. Meyn, The policy iteration algorithm for average reward Markov decision processes with general state space, IEEE Trans. Automat. Control 42(12) (1997) 1663–1680.
S.P. Meyn, Stability and optimization of queueing networks and their fluid models, in: Mathematics of Stochastic Manufacturing Systems (Williamsburg, VA, 1996) pp. 175–199. Amer. Math. Soc., Providence, RI, 1997.
S.P. Meyn, Sequencing and routing in multiclass queueing networks. Part I: Feedback regulation, SIAM J. Control Optim. 40(3) (2001) 741–776.
S.P. Meyn, Sequencing and routing in multiclass queueing networks. Part II: Workload relaxations, SIAM J. Control Optim. 42(1) (2003) 178–217.
S.P. Meyn, Workload models for Stochastic networks: Value functions and performance evaluation, in IEEE Transactions on Automatic Control, July 2005.
S.P. Meyn and R.L. Tweedie, Markov Chains and Stochastic Stability (Springer-Verlag, London, 1993). online: http://black.csl.uiuc.edu/meyn/pages/book.html.
T. Osogami, M. Harchol-Balter, A. Scheller-Wolf and L. Zhang, Exploring threshold-base policies for load sharing, in: Proceedings of the Forty-second Annual Allerton Conference on Communication, Control, and Computing, 2004.
S. Stidham, Jr. and R. Weber, A survey of Markov decision models for control of networks of queues, Queueing Systems: Theory and Applications 13(1/3) (1993) 291–314.
A. L. Stolyar, Maxweight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic, Adv. Appl. Probab. 14(1) (2004) 1–53.
V. Tadic and S.P. Meyn, Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes, in: American Control Conference, 2003.
L. Tassiulas and A. Ephremides, Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks, IEEE Trans. Automat. Control 37(12) (1992) 1936–1948.
Y.-C. Teh and A.R. Ward, Critical thresholds for dynamic routing in queueing networks, Queueing Systems: Theory and Applications 42(3) (2002) 297–316.
M.H. Veatch, Fluid analysis of arrival routing, IEEE Trans. Automat. Control AC-46 (2001) 1254–1257.
G. Weiss, Optimal draining of a fluid re-entrant line, in: Volume 71 of IMA volumes in Mathematics and its Applications, eds. Frank Kelly and Ruth Williams (New York, Springer-Verlag, 1995) pp. 91–103.
R.J. Williams, Diffusion approximations for open multiclass queueing networks: sufficient conditions involving state space collapse, Queueing Systems: Theory and Applications, 30(1/2) (1998) 27–88.
Author information
Authors and Affiliations
Corresponding author
Additional information
AMS subject classification: Primary: 90B35, 68M20, 90B15; Secondary: 93E20, 60J20.
This paper is based upon work supported by the National Science Foundation under Award Nos. ECS 02 17836 and DMI 00 85165.
Rights and permissions
About this article
Cite this article
Meyn, S. Dynamic Safety-Stocks for Asymptotic Optimality in Stochastic Networks. Queueing Syst 50, 255–297 (2005). https://doi.org/10.1007/s11134-005-0732-x
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/s11134-005-0732-x