Queueing Systems

, Volume 81, Issue 2–3, pp 99–169 | Cite as

Asymptotically optimal index policies for an abandonment queue with convex holding cost

  • M. LarrañagaEmail author
  • U. Ayesta
  • I. M. Verloop


We investigate a resource allocation problem in a multi-class server with convex holding costs and user impatience under the average cost criterion. In general, the optimal policy has a complex dependency on all the input parameters and state information. Our main contribution is to derive index policies that can serve as heuristics and are shown to give good performance. Our index policy attributes to each class an index, which depends on the number of customers currently present in that class. The index values are obtained by solving a relaxed version of the optimal stochastic control problem and combining results from restless multi-armed bandits and queueing theory. They can be expressed as a function of the steady-state distribution probabilities of a one-dimensional birth-and-death process. For linear holding cost, the index can be calculated in closed-form and turns out to be independent of the arrival rates and the number of customers present. In the case of no abandonments and linear holding cost, our index coincides with the \(c\mu \)-rule, which is known to be optimal in this simple setting. For general convex holding cost, we derive properties of the index value in limiting regimes: we consider the behavior of the index (i) as the number of customers in a class grows large, which allows us to derive the asymptotic structure of the index policies, (ii) as the abandonment rate vanishes, which allows us to retrieve an index policy proposed for the multi-class M/M/1 queue with convex holding cost and no abandonments, and (iii) as the arrival rate goes to either 0 or \(\infty \), representing light-traffic and heavy-traffic regimes, respectively. We show that Whittle’s index policy is asymptotically optimal in both light-traffic and heavy-traffic regimes. To obtain further insights into the index policy, we consider the fluid version of the relaxed problem and derive a closed-form expression for the fluid index. The latter is shown to coincide with the index values for the stochastic model in asymptotic regimes. For arbitrary convex holding cost the fluid index can be seen as the \(Gc\mu /\theta \)-rule; that is, including abandonments into the generalized \(c\mu \)-rule (\(Gc\mu \)-rule). Numerical experiments for a wide range of parameters have shown that the Whittle index policy and the fluid index policy perform very well for a broad range of parameters.


Whittle index Restless bandits Abandonments Fluid model Optimal scheduling 

Mathematics Subject Classification

68M20 60K25 90C40 



The authors would like to thank O.J. Boxma and A.J.E.M. Janssen for the proof of Lemma 1. The authors are grateful to the two anonymous referees for their valuable comments which helped improve the readability and focus of the paper. The PhD fellowship of Maialen Larrañaga is funded by a research grant of the Foundation Airbus Group ( A shorter version of this paper was published in the Proceedings of ACM Sigmetrics 2014 [30].


  1. 1.
    Abramowitz, M., Stegun, A.I.: Handbook of Mathematical Functions, with Formulas. Graphs and Mathematical Tables. Dover Publications Inc, New York (1965)Google Scholar
  2. 2.
    Anderson, D.F.: Introduction to Stochastic Processes with Applications in the Biosciences. University of Wisconsin at Madison, Available at, (2013)
  3. 3.
    Ansell, P.S., Glazebrook, K.D., Niño-Mora, J., O’Keeffe, M.: Whittle’s index policy for a multi-class queueing system with convex holding costs. Math. Methods Oper. Res. 57(1), 21–39 (2003)CrossRefGoogle Scholar
  4. 4.
    Argon, N.T., Ziya, S., Righter, R.: Scheduling impatient jobs in a clearing system with insights on patient triage in mass-casualty incidents. Probab. Eng. Inf. Sci. 22(3), 301–332 (2010)Google Scholar
  5. 5.
    Ata, B., Tongarlak, M.H.: On scheduling a multiclass queue with abandonments under general delay costs. Queueing Syst. 74, 65–104 (2013)CrossRefGoogle Scholar
  6. 6.
    Atar, R., Giat, C., Shimkin, N.: The \(c\mu /\theta \) rule for many-server queues with abandonment. Oper. Res. 58(5), 1427–1439 (2010)CrossRefGoogle Scholar
  7. 7.
    Atar, R., Giat, C., Shimkin, N.: On the asymptotic optimality of the \(c\mu /\theta \) rule under ergodic cost. Queueing Syst. 67, 127–144 (2011)CrossRefGoogle Scholar
  8. 8.
    Avram, F., Bertsimas, D., Richard, M.: Optimization of multiclass queuing networks: a linear control approach. In: Kelly, F.P., Williams, R.J. (eds.) Stochastic Networks, pp. 199–234. Springer, New York (1995)CrossRefGoogle Scholar
  9. 9.
    Ayesta, U., Jacko, P., Novak, V.: A nearly-optimal index rule for scheduling of users with abandonment. In: IEEE Infocom, pp. 2849–2857 (2011)Google Scholar
  10. 10.
    Bäuerle, N.: Asymptotic optimality of tracking policies in stochastic networks. Ann. Appl. Probab. 10, 1065–1083 (2000)CrossRefGoogle Scholar
  11. 11.
    Bäuerle, N., Rieder, U.: Optimal control of single-server fluid networks. Queueing Syst. 35, 185–200 (2000)CrossRefGoogle Scholar
  12. 12.
    Bertsekas., D.P.: Dynamic Programming and Optimal Control. Athena Scientific, Belmont (2005)Google Scholar
  13. 13.
    Bhulai, S., Blok, H., Spieksma, F.M.: \(k\) Computing queues with customer abandonment: optimality of a generalized \(c\mu \)-rule by the smoothed rate truncation method. Work in progress (2014)Google Scholar
  14. 14.
    Bhulai, S., Brooms, A.C., Spieksma, F.M.: On structural properties of the value function for an unbounded jump markov process with an application to a processor sharing retrial queue. Queueing Syst. 76, 425–446 (2014)CrossRefGoogle Scholar
  15. 15.
    Bispo, C.F.: The single-server scheduling problem with convex costs. Queueing Syst. 73, 261–294 (2013)CrossRefGoogle Scholar
  16. 16.
    Buyukkoc, C., Varaya, P., Walrand, J.: The \(c\mu \) rule revisited. Adv. Appl. Probab. 17, 237–238 (1985)CrossRefGoogle Scholar
  17. 17.
    Chaichian, M., Demichev, M.: Path Integrals in Physics Volume 1: Stochastic Process and Quantum Mechanics. Taylor & Francis, New York (2001)CrossRefGoogle Scholar
  18. 18.
    Dai, J.G.: On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models. Ann. Appl. Probab. 5, 49–77 (1995)CrossRefGoogle Scholar
  19. 19.
    Dai, J.G., He, S.: Many-server queues with customer abandonment: a survey of diffusion and fluid approximations. J. Syst. Sci. Syst. Eng. 21, 1–36 (2012)CrossRefGoogle Scholar
  20. 20.
    Down, D.G., Koole, G., Lewis, M.E.: Dynamic control of a single server system with abandonments. Queueing Syst. 67, 63–90 (2011)CrossRefGoogle Scholar
  21. 21.
    Gajrat, A., Hordijk, A.: Fluid approximation of a controlled multiclass tandem network. Queueing Syst. 35, 349–380 (2000)CrossRefGoogle Scholar
  22. 22.
    Gelenbe, E., Mitrani, I.: Analysis and Synthesis of Computer Systems. Academic Press, London (1980)Google Scholar
  23. 23.
    Gittins, J.C., Glazebrook, K., Weber, R.: Multi-Armed Bandit Allocation Indices. Wiley, London (2011)CrossRefGoogle Scholar
  24. 24.
    Glazebrook, K.D., Ansell, P.S., Dunn, R.T., Lumley, R.R.: On the optimal allocation of service to impatient tasks. J. Appl. Probab. 41, 51–72 (2004)CrossRefGoogle Scholar
  25. 25.
    Glazebrook, K.D., Hodge, D.J., Kirkbride, C., Minty, R.J.: Stochastic scheduling: a short history of index policies and new approaches to index generation for dynamic resource allocation. J. Sched. (2013). doi: 10.1007/s10951-013-0325-1
  26. 26.
    Glazebrook, K.D., Kirkbride, C., Ouenniche, J.: Index policies for the admission control and routing of impatient customers to heterogeneous service stations. Oper. Res. 57, 975–989 (2009)CrossRefGoogle Scholar
  27. 27.
    Hasenbein, J., Perry, D. (eds).: Special issue on queueing systems with abandonments. Queuing Syst. 75(2–4), 111–384 (2013)Google Scholar
  28. 28.
    Kim, J., Ward, A.: Dynamic scheduling of a GI/GI/1+GI queue with multiple customer classes. Queueing Syst. 75(2–4), 339–384 (2013)CrossRefGoogle Scholar
  29. 29.
    Larrañaga, M., Ayesta, U., Verloop, I.M.: Dynamic fluid-based scheduling in a multi-class abandonment queue. Perform. Eval. 70, 841–858 (2013)CrossRefGoogle Scholar
  30. 30.
    Larrañaga, M., Ayesta, U., Verloop, I.M.: Index policies for a multi-class queue with convex holding cost and abandonment. In: Proceedings of ACM SIGMETRICS, 2014Google Scholar
  31. 31.
    Larrañaga, M., Ayesta, U., Verloop, I.M.: Stochastic and fluid index policies for resource allocation problems. In: Proceedings of IEEE Infocom, 2015Google Scholar
  32. 32.
    Mandelbaum, A., Stolyar, S.: Scheduling flexible servers with convex delay costs: heavy-traffic optimality of the generalized \(c\mu \)-rule. Oper. Res. 52(6), 836–855 (2004)CrossRefGoogle Scholar
  33. 33.
    Niño-Mora, J.: Dynamic priority allocation via restless bandit marginal productivity indices. TOP 15(2), 161–198 (2007)CrossRefGoogle Scholar
  34. 34.
    Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (2005)Google Scholar
  35. 35.
    van Mieghem, J.A.: Dynamic scheduling with convex delay costs: the generalized \(c\mu \) rule. Ann. Appl. Probab. 5(3), 808–833 (1995)Google Scholar
  36. 36.
    Verloop, I.M.: Asymptotically optimal priority policies for indexable and non-indexable restless bandits. CNRS Technical Report, hal-00743781, April 2015Google Scholar
  37. 37.
    Verloop, I.M., Núñez-Queija, R.: Asymptotically optimal parallel resource assignment with interference. Queueing Syst. 65(1), 43–92 (2010)CrossRefGoogle Scholar
  38. 38.
    Weber, R.R., Weiss, G.: On an index policy for restless bandits. J. Appl. Probab. 27, 637–648 (1990)CrossRefGoogle Scholar
  39. 39.
    Weiss, G.: On optimal draining of reentrant fluid lines. In: Kelly, F.P., Williams, R.J. (eds.) Stochastic Networks, pp. 91–103. Springer, New York (1995)CrossRefGoogle Scholar
  40. 40.
    Whittle, P.: Restless bandits: activity allocation in a changing world. J. Appl. Probab. 25, 287–298 (1988)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • M. Larrañaga
    • 1
    • 2
    • 5
    Email author
  • U. Ayesta
    • 2
    • 3
    • 4
    • 5
  • I. M. Verloop
    • 1
    • 5
  1. 1.CNRS, IRITToulouseFrance
  2. 2.CNRS, LAASToulouseFrance
  3. 3.IKERBASQUE, Basque Foundation for ScienceBilbaoSpain
  4. 4.UPV/EHU, University of the Basque CountryDonostiaSpain
  5. 5.Univ. de Toulouse, INP, LAASToulouseFrance

Personalised recommendations