Queueing Systems

, Volume 85, Issue 1–2, pp 31–65 | Cite as

Pull-based load distribution among heterogeneous parallel servers: the case of multiple routers

  • Alexander L. Stolyar


The model is a service system, consisting of several large server pools. A server’s processing speed and buffer size (which may be finite or infinite) depend on the pool. The input flow of customers is split equally among a fixed number of routers, which must assign customers to the servers immediately upon arrival. We consider an asymptotic regime in which the total customer arrival rate and pool sizes scale to infinity simultaneously, in proportion to a scaling parameter n, while the number of routers remains fixed. We define and study a multi-router generalization of the pull-based customer assignment (routing) algorithm PULL, introduced in Stolyar (Queueing Syst 80(4): 341–361, 2015) for the single-router model. Under the PULL algorithm, when a server becomes idle it sends a “pull-message” to a randomly uniformly selected router; each router operates independently—it assigns an arriving customer to a server according to a randomly uniformly chosen available (at this router) pull-message, if there is any, or to a randomly uniformly selected server in the entire system otherwise. Under Markov assumptions (Poisson arrival process and independent exponentially distributed service requirements), and under subcritical system load, we prove asymptotic optimality of PULL: as \(n\rightarrow \infty \), the steady-state probability of an arriving customer experiencing blocking or waiting vanishes. Furthermore, PULL has an extremely low router–server message exchange rate of one message per customer. These results generalize some of the single-router results in Stolyar (2015).


Large-scale heterogeneous service systems Multiple routers (dispatchers) Pull-based load distribution Load balancing Fluid limits Asymptotic optimality 

Mathematics Subject Classification

90B15 60K25 


  1. 1.
    Badonnel, R., Burgess, M.: Dynamic pull-based load balancing for autonomic servers. Netw. Oper. Manag. Symp. NOMS 2008, 751–754 (2008)Google Scholar
  2. 2.
    Bramson, M., Lu, Y., Prabhakar, B.: Asymptotic independence of queues under randomized load balancing. Queueing Syst. 71, 247–292 (2012)CrossRefGoogle Scholar
  3. 3.
    Bramson, M., Lu, Y., Prabhakar, B.: Decay of tails at equlibrium for FIFO join the shortest queue networks. Ann. Appl. Probab. 23, 1841–1878 (2013)CrossRefGoogle Scholar
  4. 4.
    Eschenfeldt, P., Gamarnik, D.: Join the shortest queue with many servers. The heavy traffic asymptotics (2015)Google Scholar
  5. 5.
    Grimmett, G., Stirzaker, D.: Probability and Random Processes, 3rd edn. Oxford University Press, Oxford (2001)Google Scholar
  6. 6.
    Liggett, T.M.: Interacting Particle Systems. Springer, New York (1985)CrossRefGoogle Scholar
  7. 7.
    Lu, Y., Xie, Q., Kliot, G., Geller, A., Larus, J., Greenberg, A.: Join-idle-queue: a novel load balancing algorithm for dynamically scalable web services. Perform. Eval. 68, 1057–1071 (2011)CrossRefGoogle Scholar
  8. 8.
    Mitzenmacher, M.: The power of two choices in randomized load balancing. IEEE Trans. Parallel Distrib. Syst. 12(10), 1094–1104 (2001)CrossRefGoogle Scholar
  9. 9.
    Mukherjee, D., Borst, S., van Leeuwaarden, J., Whiting, P.: Universality of load balancing schemes on diffusion scale (2015)Google Scholar
  10. 10.
    Stolyar, A.L.: Large-scale heterogeneous service systems with general packing constraints. Advances in Applied Probab. 49(1), (2017, to appear)Google Scholar
  11. 11.
    Stolyar, A.L.: Pull-based load distribution in large-scale heterogeneous service systems. Queueing Syst. 80(4), 341–361 (2015)Google Scholar
  12. 12.
    Vvedenskaya, N., Dobrushin, R., Karpelevich, F.: Queueing system with selection of the shortest of two queues: an asymptotic approach. Problems Inform. Transm. 32(1), 20–34 (1996)Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Lehigh UniversityBethlehemUSA

Personalised recommendations