Abstract
A general model with multiple input flows (classes) and several flexible multi-server pools is considered. We propose a robust, generic scheme for routing new arrivals, which optimally balances server pools’ loads, without the knowledge of the flow input rates and without solving any optimization problem. The scheme is based on Shadow routing in a virtual queueing system. We study the behavior of our scheme in the Halfin–Whitt (or, QED) asymptotic regime, when server pool sizes and the input rates are scaled up simultaneously by a factor r growing to infinity, while keeping the system load within \(O(\sqrt{r}\,)\) of its capacity.
The main results are as follows. (i) We show that, in general, a system in a stationary regime has at least \(O(\sqrt{r}\,)\) average queue lengths, even if the so called null-controllability (Atar et al., Ann. Appl. Probab. 16, 1764–1804, 2006) on a finite time interval is possible; strategies achieving this \(O(\sqrt{r}\,)\) growth rate we call order-optimal. (ii) We show that some natural algorithms, such as MaxWeight, that guarantee stability, are not order-optimal. (iii) Under the complete resource pooling condition, we prove the diffusion limit of the arrival processes into server pools, under the Shadow routing. (We conjecture that result (iii) leads to order-optimality of the Shadow routing algorithm; a formal proof of this fact is an important subject of future work.) Simulation results demonstrate good performance and robustness of our scheme.
Similar content being viewed by others
References
Aksin, Z., Armony, M., Mehrotra, V.: The modern call center: A multi-disciplinary perspective on operations management research. Prod. Oper. Manag. 16, 655–688 (2007)
Armony, M.: Dynamic routing in large-scale service systems with heterogenous servers. Queueing Syst., Theory Appl. 51, 287–329 (2005)
Armony, M., Maglaras, C.: Contact centers with a call-back option and real-time delay information. Oper. Res. 52, 527–545 (2004)
Armony, M., Maglaras, C.: On customer contact centers with a call-back option: Customer decisions, routing rules and system design. Oper. Res. 52, 271–292 (2004)
Atar, R.: A diffusion model of scheduling control in queueing systems with many servers. Ann. Appl. Probab. 15, 820–852 (2005)
Atar, R.: Scheduling control for queueing systems with many servers: Asymptotic optimality in heavy traffic. Ann. Appl. Probab. 15, 2606–2650 (2005)
Atar, R., Mandelbaum, A., Reiman, M.: Scheduling a multi-class queue with many exponential servers: Asymptotic optimality in heavy-traffic. Ann. Appl. Probab. 14, 1084–1134 (2004)
Atar, R., Mandelbaum, A., Shaikhet, G.: Queueing systems with many servers: Null controllability in heavy traffic. Ann. Appl. Probab. 16, 1764–1804 (2006)
Bassamboo, A., Zeevi, A.: On a data-driven method for staffing large call centers. Oper. Res. 57, 714–726 (2009)
Bassamboo, A., Harrison, J.M., Zeevi, A.: Design and control of a large call center: Asymptotic analysis of an LP-based method. Oper. Res. 54, 419–435 (2006)
Bassamboo, A., Harrison, J.M., Zeevi, A.: Dynamic routing and admission control in high-volume service systems: Asymptotic analysis via multi-scale fluid limits. Queueing Syst., Theory Appl. 51, 249–285 (2006)
Bell, S.L., Williams, R.J.: Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: Asymptotic optimality of a threshold policy. Ann. Appl. Probab. 11, 608–649 (2001)
Chung, K.L.: A Course in Probability Theory, 3rd edn. Academic Press, New York (2001)
Dai, J.G., Tezcan, T.: State space collapse in many server limits of parallel server systems. Technical report, School of Industrial and Systems Engineering, Georgia Institute of Technology (2005)
Dai, J.G., Tezcan, T.: Dynamic control of parallel server systems in many server heavy traffic. Queueing Syst., Theory Appl. 59, 95–134 (2008)
Gamarnik, D., Momcilovic, P.: Steady-state analysis of a multi-server queue in the Halfin–Whitt regime. Adv. Appl. Probab. 40, 548–577 (2008)
Gans, N., Koole, G., Mandelbaum, A.: Telephone call centers: Tutorial, review and research prospects. Manuf. Serv. Oper. Manag. 5, 79–141 (2003)
Garnett, O., Mandelbaum, A., Reiman, M.: Designing a call center with impatient customers. Manuf. Serv. Oper. Manag. 48, 566–583 (2002)
Gurvich, I., Whitt, W.: Scheduling flexible servers with convex delay costs in many-server service systems. Manuf. Serv. Oper. Manag. 11, 237–253 (2007)
Gurvich, I., Whitt, W.: Service-level differentiation in many-server service systems via queue-ratio routing. Oper. Res. 58, 316–328 (2010)
Gurvich, I., Armony, M., Mandelbaum, A.: Staffing and control of large-scale service systems with multiple customer classes and fully flexible servers. Manag. Sci. 54, 279–294 (2008)
Halfin, S., Whitt, W.: Heavy-traffic limits for queues with many exponential servers. Oper. Res. 29, 567–588 (1981)
Harrison, J.M., López, M.J.: Heavy traffic resource pooling in parallel-server systems. Queueing Syst., Theory Appl. 33, 339–368 (1999)
Harrison, J.M., Zeevi, A.: Dynamic scheduling of a multiclass queue in the Halfin and Whitt heavy traffic regime. Oper. Res. 52, 243–257 (2004)
Jennings, O., de Vericourt, F.: Nurse staffing and bed capacity: A queueing perspective. Technical report, Duke University, The Fuqua School of Business (2008)
Kaspi, H., Ramanan, K.: Law of large numbers limits for many server queues. Working paper (2007)
Mandelbaum, A., Stolyar, A.L.: Scheduling flexible servers with convex delay costs: Heavy-traffic optimality of the generalized cμ-rule. Oper. Res. 52, 836–855 (2004)
Mandelbaum, A., Massey, W.A., Reiman, M.: Strong approximations for Markovian service networks. Queueing Syst., Theory Appl. 30, 149–201 (1998)
Perry, O., Whitt, W.: Responding to unexpected overloads in large-scale service systems. Manag. Sci. 55, 1353–1367 (2009)
Puhalskii, A., Reiman, M.: The multiclass GI/PH/N queue in the Halfin–Whitt regime. Adv. Appl. Probab. 32, 564–595 (2000)
Randhawa, R.S., Kumar, S.: Usage restriction and subscription services: Operational benefits with rational users. Manuf. Serv. Oper. Manag. 10, 429–447 (2008)
Reed, J.E.: The G/GI/N queue in the Halfin–Whitt regime I: Infinite server queue system equations. Ann. Appl. Probab. 19, 2211–2269 (2009)
Shakkottai, S., Stolyar, A.L.: Scheduling for multiple flows sharing a time-varying channel: The exponential rule. Anal. Methods Appl. Probab., Am. Math. Soc. Transl. Ser. 2 207, 185–202 (2002)
Stolyar, A.L.: Maxweight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic. Ann. Appl. Probab. 14, 1–53 (2004)
Stolyar, A.L.: Maximizing queueing network utility subject to stability: Greedy primal-dual algorithm. Queueing Syst., Theory Appl. 50, 401–457 (2005)
Stolyar, A.L.: Optimal routing in output-queued flexible server systems. Probab. Eng. Inf. Sci. 19, 141–189 (2005)
Stolyar, A.L., Viswanathan, H.: Self-organizing dynamic fractional frequency reuse in ofdma systems. In: Proceeding of INFOCOM’2008 (2008)
Tezcan, T., Dai, J.G.: Dynamic control of N-systems with many servers: Asymptotic optimality of a static priority policy in heavy traffic. Oper. Res. 58, 94–110 (2010)
Whitt, W.: A diffusion approximation for the G/GI/n/m queue. Oper. Res. 52, 922–941 (2004)
Whitt, W.: Heavy-traffic limits for the \(G/H_{2}^{*}/n/m\) queue. Math. Oper. Res. 30, 1–27 (2005)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Stolyar, A.L., Tezcan, T. Control of systems with flexible multi-server pools: a shadow routing approach. Queueing Syst 66, 1–51 (2010). https://doi.org/10.1007/s11134-010-9183-0
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11134-010-9183-0
Keywords
- Queueing networks
- Large flexible server pools
- Routing and scheduling
- Shadow routing
- Many server asymptotics
- Halfin–Whitt regime
- Diffusion limit
- Order-optimality