Queueing Systems

, Volume 88, Issue 3–4, pp 243–278 | Cite as

Asymptotics of insensitive load balancing and blocking phases



We study a single class of traffic acting on a symmetric set of processor-sharing queues with finite buffers, and we consider the case where the load scales with the number of servers. We address the problem of giving robust performance bounds based on the study of the asymptotic behaviour of the insensitive load balancing schemes, which have the desirable property that the stationary distribution of the resulting stochastic network depends on the distribution of job-sizes only through its mean. It was shown for small systems with losses that they give good estimates of performance indicators, generalizing henceforth Erlang formula, whereas optimal policies are already theoretically and computationally out of reach for networks of moderate size. We characterize the response of symmetric systems under those schemes at different scales and show that three amplitudes of deviations can be identified according to whether \(\rho < 1\), \(\rho = 1\), or \(\rho > 1\). A central limit scaling takes place for a sub-critical load; for \(\rho =1\), the number of free servers scales like \(n^{ {\theta \over \theta +1}}\) (\(\theta \) being the buffer depth and n being the number of servers) and is of order 1 for super-critical loads. This further implies the existence of different phases for the blocking probability. Before a (refined) critical load \(\rho _c(n)=1-a n^{- {\theta \over \theta +1}}\), the blocking is exponentially small and becomes of order \( n^{- {\theta \over \theta +1}}\) at \(\rho _c(n)\). This generalizes the well-known quality-and-efficiency-driven regime, or Halfin—Whitt regime, for a one-dimensional queue and leads to a generalized staffing rule for a given target blocking probability.


Insensitive load balancing Blocking phases Mean-field scalings QED-Jagerman–Halfin–Whitt regime 

Mathematics Subject Classification

60K25 60F05 60F10 



This work was partially supported by the Basque Center for Applied Mathematics BCAM and the Bizkaia Talent and European Commission through COFUND programme, under the project titled “High-dimensional stochastic networks and particles systems”, awarded in the 2014 Aid Programme with request Reference Number AYD-000-273, and by the STIC-AmSud Project No. 14STIC03. We thank the referees for their insightful comments which have improved the quality of the paper.


  1. 1.
    Alzer, H., Baricz, A.: Functional inequalities for the incomplete gamma function. J. Math. Anal. Appl. 385(1), 167–178 (2012)CrossRefGoogle Scholar
  2. 2.
    Atar, R.: A diffusion regime with nondegenerate slowdown. Oper. Res. 60(2), 490–500 (2012)CrossRefGoogle Scholar
  3. 3.
    Benäim, M.: Dynamics of stochastic approximation algorithms. Lecture Notes in Math. Springer, Berlin, Séminaire de Probabilités XXXIII (1999)Google Scholar
  4. 4.
    Bonald, T., Jonckheere, M., Proutière, A.: Insensitive load balancing. SIGMETRICS Perform. Eval. Rev. 32(1), 367–377 (2004)CrossRefGoogle Scholar
  5. 5.
    Bonald, T., Massoulié, L., Proutière, A., Virtamo, J.: A queueing analysis of max-min fairness, proportional fairness and balanced fairness. Queueing Syst. Theory Appl. 53(1–2), 65–84 (2006)CrossRefGoogle Scholar
  6. 6.
    Bonald, T., Proutière, A.: Insensitive bandwidth sharing in data networks. Queueing Syst. Theory Appl. 44(1), 69–100 (2003)CrossRefGoogle Scholar
  7. 7.
    Bramson, M., Lu, Y., Prabakhar, B.: Asymptotic independence of queues under randomized load balancing. Queueing Syst. 71, 247–292 (2012)CrossRefGoogle Scholar
  8. 8.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Hoboken (2006)Google Scholar
  9. 9.
    Eschenfeldt, P., Gamarnik, D.: Join the shortest queue with many servers. In: The Heavy Traffic Asymptotics (2015)Google Scholar
  10. 10.
    Foss, S., Stolyar, A.: Large-scale join-idle-queue system with general service times. ArXiv e-prints May (2016)Google Scholar
  11. 11.
    Graham, C.: Chaoticity on path space for a queueing network with selection of the shortest queue among several. J. Appl. Probab. 37(1), 198–211 (2000)CrossRefGoogle Scholar
  12. 12.
    Halfin, S., Whitt, W.: Heavy-traffic limits for queues with many exponential servers. Oper. Res. 29(3), 567–588 (1981)CrossRefGoogle Scholar
  13. 13.
    Hordijk, A.: Insensitive bounds for performance measures. In: Proceedings of the 12th International Teletraffic Congress (ITC 12), Torino (1988)Google Scholar
  14. 14.
    Hordijk, A., Koole, G.: On the optimality of the generalized shortest queue policy. Probab. Eng. Inf. Sci. 4(4), 477–487 (1990)CrossRefGoogle Scholar
  15. 15.
    Hordijk, A., Ridder, A.: Insensitive bounds for the stationary distribution of non-reversible Markov chains. J. Appl. Probab. 25(1), 9–20 (1988)CrossRefGoogle Scholar
  16. 16.
  17. 17.
    Jagerman, D.L.: Some properties of the Erlang loss function. Bell Syst. Tech. J. 53(3), 525–551 (1974)CrossRefGoogle Scholar
  18. 18.
    Janssen, A.J.E.M., Van Leeuwaarden, J.S.H., Zwart, B.: Gaussian expansions and bounds for the Poisson distribution applied to the Erlang B formula. Adv. Appl. Probab. 40(1), 122–143 (2008)CrossRefGoogle Scholar
  19. 19.
    Janssen, A.J.E.M., van Leeuwaarden, J.S.H., Zwart, A.P.: Corrected asymptotics for a multi-server queue in the Halfin–Whitt regime. Queueing Syst. Theory Appl. 58(4), 261–301 (2008)CrossRefGoogle Scholar
  20. 20.
    Janssen, A.J.E.M., van Leeuwaarden, J.S.H., Zwart, A.P.: Refining square-root safety staffing by expanding Erlang C. Oper. Res. 59(6), 1512–1522 (2011)CrossRefGoogle Scholar
  21. 21.
    Jonckheere, M.: Insensitive versus efficient dynamic load balancing in networks without blocking. Queueing Syst. 54(3), 193–202 (2006)CrossRefGoogle Scholar
  22. 22.
    Jonckheere, M., Mairesse, J.: Towards an Erlang formula for multiclass networks. Queueing Syst. 66(1), 53–78 (2010)CrossRefGoogle Scholar
  23. 23.
    Kipnis, C., Landim, C.: Scaling Limits of Interacting Particle Systems. Grundlehren der Mathematischen Wissenschaften. Springer, Berlin (2013)Google Scholar
  24. 24.
    Knessl, C., Yao, H.: On the finite capacity shortest queue problem. Prog. Appl. Math. 2(1), 1–34 (2011)CrossRefGoogle Scholar
  25. 25.
    Le Boudec, J.Y.: The stationary behaviour of fluid limits of reversible processes is concentrated on stationary points. Netw. Hetegnereous Med. 8(2), 1529–540 (2013)Google Scholar
  26. 26.
    Leino, J., Virtamo, J.: Insensitive load balancing in data networks. Comput. Netw. 50(8), 1059–1068 (2006)CrossRefGoogle Scholar
  27. 27.
    Lu, Y., Xie, Q., Kliot, G., Geller, A., Larus, J.R., Greenberg, A.: Join-idle-queue: a novel load balancing algorithm for dynamically scalable web services. Perform. Eval. 68(11), 1056–1071 (2011)CrossRefGoogle Scholar
  28. 28.
    Maman, S.: Uncertainty in the demand for service: the case of call centers and emergency departments. Master’s Thesis, Technion, April (2009)Google Scholar
  29. 29.
    Mitzenmacher, M.: The power of two choices in randomized load balancing. Ph.D. Thesis (1996)Google Scholar
  30. 30.
    Mukherjee, D., Borst, S.C., van Leeuwaarden, J.S.H., Whiting, P.A.: Universality of load balancing schemes on the diffusion scale. J. Appl. Probab. 53(4), 1111–1124 (2016). 12CrossRefGoogle Scholar
  31. 31.
    Mukhopadhyay, A., Karthik, A., Mazumdar, R.R., Guillemin, F.: Mean field and propagation of chaos in multi-class heterogeneous loss models. Perform. Eval. 91, 117–131 (2015). (Special issue: performance 2015)CrossRefGoogle Scholar
  32. 32.
    Nemes, G.: An explicit formula for the coefficients in Laplace’s method. Constr. Approx. 38(3), 471–487 (2013). arXiv:1207.5222 CrossRefGoogle Scholar
  33. 33.
    Pla, V., Virtamo, J., Martinez-Bauset, J.: Optimal robust policies for bandwidth allocation and admission control in wireless networks. Comput. Netw. 52(17), 3258–3272 (2008)CrossRefGoogle Scholar
  34. 34.
    Robert, P.: Stochastic Networks and Queues. Springer, Berlin (2003)CrossRefGoogle Scholar
  35. 35.
    Sagitov, S.: Weak Convergence of Probability Measures. (2013)
  36. 36.
    Sparaggis, P.D., Towsley, D., Cassandras, C.G.: Extremal properties of the shortest/longest non-full queue policies in finite-capacity systems with state-dependent service rates. J. Appl. Probab. 30(1), 223–236 (1993)CrossRefGoogle Scholar
  37. 37.
    Stolyar, A.: Pull-based load distribution among heterogeneous parallel servers: the case of multiple routers. ArXiv e-prints (2015)Google Scholar
  38. 38.
    Vvedenskaya, N.D., Dobrushin, R.L., Karpelevich, F.I.: Queueing system with selection of the shortest of two queues: an asymptotic approach. Probl. Inf. Transm. 32(1), 15–27 (1996)Google Scholar
  39. 39.
    Whitt, W.: Heavy traffic approximations for service systems with blocking. Bell Syst. Tech. J. 63(4), 689–708 (1984)Google Scholar
  40. 40.
    Whittle, P.: Partial balance and insensitivity. J. Appl. Probab. 22(1), 168–176 (1985)CrossRefGoogle Scholar
  41. 41.
    Xie, Q., Dong, X., Lu, Y., Srikant, R.: Power of d choices for large-scale bin packing: a loss model. In: Proceedings of the 2015 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS’15, pp. 321–334. ACM, New York (2015)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Instituto de cálculoUniversidad de Buenos Aires and ConicetBuenos AiresArgentina
  2. 2.LAAS-CNRSUniversité de Toulouse, CNRSToulouseFrance

Personalised recommendations